chore: add comprehensive API wrapper documentation

- Add docs/ directory with 6 comprehensive documentation files:
  * INDEX.md - Main navigation and documentation index
  * GETTING_STARTED.md - Setup and configuration guide
  * API_REFERENCE.md - Complete API documentation
  * WRAPPERS.md - High-level wrappers guide
  * CUSTOM_REQUESTS.md - Advanced usage patterns
  * TROUBLESHOOTING.md - Problem-solving guide

- Update README.md with enhanced project information

- Update development.md with expanded documentation context

Extends previous session's documentation work with complete
developer and user-facing documentation.

Closes previous documentation issue.
This commit is contained in:
BarnacleBoy 2026-04-18 01:44:49 +00:00
parent 14346ed02e
commit fd1a1566e9
8 changed files with 4909 additions and 61 deletions

View file

@ -1,37 +1,788 @@
# Things to think about do
## Debates
1. Hating the name `jobs` and `run_job`. Requests is an
existing package dependency though. `tasks` seems to broard. I'm sure
there's probably an already existing name that I can't articulate how
to find. `apirequests`, `rqsts`, `apicalls`?
2. Should these `job` be called with dot notation? Or with slashes? Or
something else? Is it okay to have these as strings?
3. Should we do validation in the `jobs` or let the server do that all?
4. `wrapped` is an awkward name. What's more proper? But also obvious?
5. Should have consistent format, ie `create_{noun}`, and
`delete_{noun}`, rather than just verbing ie `comment`, `post` as this
is ambiguousetc
6. Known/displayed entity names or as-is/DB names, ie: `trust` or
`reltrust`? We call trust as reltrust in the database and in the
backend, relfollow for follow, but don't have relblock, instead it's
userblock. It's a bit inconsistent and not very obvious at all.
Feels helpful to make it more obvious inside the wrapper but is also
potentially misleading.
7. It makes more sense to organise like: `user.follow` rather than
`follow.follow`, and `post.vote` instead of `vote.vote`, but what about
`post.comment.vote` vs `comment.vote`, what about
`post.comment.comment`, they seem weird.
## ToDo:
1. Make more jobs
2. Make more wrappers
3. Write more documentation
Mute
Block
Branch Mute
# TrustCafé API Wrapper - Development Guide
This document covers development considerations, known limitations, and contribution guidelines for the TrustCafé API wrapper.
## Table of Contents
1. [Project Structure](#project-structure)
2. [Design Decisions](#design-decisions)
3. [Known Limitations](#known-limitations)
4. [Missing Features](#missing-features)
5. [Future Enhancements](#future-enhancements)
6. [Contributing](#contributing)
7. [Testing Guide](#testing-guide)
8. [Extending the Wrapper](#extending-the-wrapper)
## Project Structure
```
trustcafe-api-wrapper/
├── README.md # Main documentation
├── documentation.md # Basic usage documentation
├── development.md # This file - development notes
├── pyproject.toml # Project configuration
├── setup.py # Legacy setup file
├── development.md
├── .gitlab-ci.yml # CI/CD configuration
└── src/
└── trustcafeapiwrapper/
├── __init__.py # Package exports
├── apiclient.py # Core API client (218 lines)
├── jobs/ # API job functions
│ ├── __init__.py
│ ├── post/
│ ├── comment/
│ ├── userprofile/
│ ├── follow/
│ ├── vote/
│ ├── reaction/
│ ├── notification/
│ ├── trust/
│ ├── branch/
│ └── feed/
├── wrappers/ # High-level wrappers
│ ├── __init__.py
│ ├── post/
│ └── comment/
└── utils/ # Helper functions
├── __init__.py
├── make_comment_sk.py
├── make_post_sk.py
├── get_post_pksk.py
├── get_userprofile_pksk_from_slug.py
├── get_user_slug_from_path.py
├── get_entity_from_str.py
├── get_parent_pksk_from_path.py
└── get_child_spksk_from_paths.py
```
## Design Decisions
### 1. Naming Conventions
**Jobs vs. Wrappers:**
| Aspect | Jobs | Wrappers |
|--------|------|----------|
| Naming | Verb (`create_x`, `get_x`) | Noun phrase (`create_post`, `update_post`) |
| Pattern | Inconsistent | Standardized (`create_noun`, `update_noun`) |
| Usage | Flexible | Optimized for common cases |
| Type | Function | Function returning dict with job spec |
**Question: Should these be called `jobs`, `tasks`, `apirequests`, `rqsts`, or `apicalls`?**
- **Chosen:** `jobs` - Existing dependency (requests)
- **Alternatives considered:** `tasks` (too broad), `apirequests` (confusing with HTTP library)
**Thread with discussion:** See GitLab issues
### 2. Job Execution Method
**Question: Should jobs be called with `dot notation` (e.g., `userprofile.get`) or `slashes` (e.g., `userprofile/get`)?**
- **Chosen:** Dot notation for module style
- **Flexibility:** Supports both strings for dynamic loading and functions for static use
- **Pattern:**
```python
# String-based (dynamic) - works for variable jobs
API.run_job('userprofile.get', username)
# Function-based (static) - good for explicit calls
API.run_job(userprofile_get, username)
```
### 3. Parameter Validation
**Question: Should validation happen in jobs or let the server do it?**
- **Chosen:** Server-side validation with pydantic at APIClient level
- **Reasoning:**
- Server provides authoritative validation
- Reduced redundancy
- Known API responses
- **Trade-off:** Misuse symptoms appear when server rejects invalid data
### 4. Wrapper Architecture
**Question: What's the proper name for the wrapper system?**
- **Chosen:** `wrapped()` as the calling method
- **Alternatives considered:** None after review
- **Pros:** Self-documenting parameter handling
- **Cons:** Awkward name
### 5. Verbosity in Noun Names
**Question: Use `post.vote` vs `vote.cast`?**
- **Chosen:** `post.vote` for entity-based paths
- **Rationale:**
- Better semantic structure: `object.action`
- Clearer context (what are we voting on?)
- Consistent with TrustCafé's own endpoints
**Examples:**
- ✅ `post.comment` (comment on a post)
- ✅ `comment.vote` (vote on a comment)
- ⚠️ vs `comment.comment` (commenting on a comment)
### 6. Database vs. Display Names
**Question: Use `trust` or RelTrust in code?**
- **Chosen:** Display names (`trust`, `follow`, `block`)
- **Misleading risk:** Database uses RelTrust, RelFollow, UserBlock
- **Plan:** Internal mapping in utility functions
**Database Inconsistencies:**
- `trust``RelTrust`
- `follow``RelFollow`
- `block``UserBlock` (not `RelBlock`)
- `mute``Mute`
### 7. Entity Hierarchy Organization
**Question: Organize as `user.follow` or `follow.follow`?**
- **Chosen:** `user.follow` (object-centric) for most cases
- **Complex cases:** `comment.vote`, `post.comment.comment` (both possible)
**Plan:**
- Simple cases: Entity-based (`user.follow`, `post.comment`)
- Complex cases: Action-centric (`comment.vote`)
- Organize by common pattern, not dogmatically
## Known Limitations
### 1. Incomplete Job Coverage
**Status:** Partial coverage of API endpoints
**Currently Implemented:**
- ✅ Post operations: create, get, update, listall, listpublic, listbybranch, listbyuserprofile, listremoved
- ✅ Comment operations: create, listtbypostid
- ✅ UserProfile: get
- ✅ Follow: follow
- ✅ Vote: votecast
- ✅ Reaction: reacttosomething, getbyparent, listbyparent
- ✅ Notification: listnotifications, markallasread
- ✅ Trust: createorupdate, listbyusersinit, listbyuserhas
- ✅ Branch: get, listbyname
- ✅ Feed: cafefeed, following
- ⚠️ Block: job created but not connected to server
- ⚠️ Mute: job created but not connected to server
**Missing/Incomplete:**
- ❌ Post status operations (delete, pin, etc.)
- ❌ Full post metadata operations
- ❌ Advanced comment features
- ❌ User management operations
- ❌ Search functionality
- ❌ Moderation operations
- ❌ Analytics endpoints
---
### 2. Enums Not Strictly Typed
**Issue:** Some parameter types are not enforced
**Example:**
```python
# Acceptable (but should be):
API.run_job('vote.votecast', {..., "voteType": "UP"}) # Should error?
# Better to use type hints:
vote_type: Literal["up", "down"]
```
**Affected APIs:**
- voteType (should be "up" or "down")
- reactionType (should match TrustCafé enum values)
- trustType (should be known set)
**Solution:** Add pydantic models for requests and responses
---
### 3. No Async Support
**Issue:** All requests are synchronous (blocking)
**Impact:**
- Serial request execution
- Long waits for multiple calls
- Limited throughput
**Solution:**
- Add async/await support using `aiohttp`
- Provide sync wrapper that calls async
- Consider for future major version
---
### 4. Limited Timeout Configuration
**Issue:** Hardcoded 20-second timeout
**Current Code:**
```python
# apiclient.py line 87
response = session.request(method.upper(), url, json=data, headers=headers, timeout=20)
```
**Problem:**
- No way to configure timeout per request
- Default works but is a guess
- Network issues hard to diagnose without flexibility
**Solution:**
```python
# Add timeout parameter
def make_request(self, method, endpoint, path, data=None,
authenticate=True, query_params=None,
timeout=20): # Add timeout parameter
```
---
### 5. No Detailed Error Types
**Issue:** All errors are generic `Exception` or `ValueError`
**Current Code:**
```python
except Exception as e:
raise ConnectionError(f"An error occurred while making the request: {e}")
```
**Limitations:**
- No authentication-specific errors
- No validation errors
- No permission errors
- Error type doesn't give context
**Solution:** Create custom error classes:
```python
class APIError(Exception):
"""Base API error"""
class AuthenticationError(APIError):
"""Authentication failures"""
class ValidationError(APIError):
"""Invalid parameter values"""
```
---
### 6. Token File Handling
**Issue:** Token file stored in working directory (influenced by os.getcwd())
**Problem:**
- Cross-platform issues
- Permission problems
- Can't control location from outside
**Current Code:**
```python
token_data_path = f"token_data_{self.environment}.json"
```
**Solution:**
```python
# User-configurable path
import os
default_path = os.path.join(os.getcwd(), f"token_data_{self.environment}.json")
API.handle_token(token_data_path=os.getenv("TRUSTCAFE_TOKEN_PATH", default_path))
```
---
### 7. Database-Display Name Mismatch
**Issue:** Internal vs. external naming不一致
**Examples:**
- Display: `trust` ↔ Database: `RelTrust`
- Display: `follow` ↔ Database: `RelFollow`
- Display: `block` ↔ Database: `UserBlock` (not `RelBlock`)
- Display: `mute` ↔ Database: `Mute`
**Solution:**
- Map most common names (trust, follow, mute)
- Add mappings in utility functions
- Document naming inconsistencies
- Consider renaming display names for consistency
---
### 8. Unused Dependencies
**Cleanup opportunity:** Some dependencies may not be used
**Current Dependencies:**
```toml
dependencies = [
"dotenv>=0.9.9", # Used in development.md example
"pydantic>=2.12.5", # Used for SecretStr
"requests>=2.33.1", # Used for HTTP requests
"simplejson>=3.20.2", # Used for JSON parsing
]
```
**Potential Removal:**
- `simplejson`: Can use Python's built-in json
- **Note:** Keep for now to minimize risk
---
### 9. Hard-coded Endpoints
**Issue:** API base URLs are hard-coded
**Current Code:**
```python
endpoints = {
"alpha": {
"audrey": "https://eso1of8gqd.execute-api.us-east-1.amazonaws.com/alpha/",
# ...
},
"production": {
# ...
}
}
```
**Problem:**
- Environment changes will break
- No hot-reload
- Same URLs across versions
**Solution:**
```python
# Could be environment variables
endpoints = {
"alpha": {
"audrey": os.getenv("AUDREY_ALPHA_URL", "default-alpha-url"),
},
# ...
}
```
## Missing Features
### High Priority
1. **Complete Post Operations**
- Post status: pin, unpin, archive, delete
- Post editing: editing mode handling
- Post metadata bulk operations
- Post history/versioning
2. **User Management Jobs**
- Block/unblock users
- Mute/unmute users
- User profile management
- User settings
3. **Search Functionality**
- Full-text search
- Advanced filters
- Query builder
4. **Comment Enhancements**
- Comment editing
- Comment deletion
- Comment threading
- Comment moderation
---
### Medium Priority
5. **Reaction UI**
- Multiple response types (heart, fire, thumbs-up, etc.)
- Reaction history
- Reaction statistics
6. **Trust System**
- Trust ranking algorithms
- Trust network visualization
- Block followers
- Trust relationships with others' trust
7. **Notification Enhancements**
- Notification callbacks/webhooks
- Notification types documentation
- Notification preferences
- Notification filtering
8. **Feed Customization**
- Filter classes
- Schedule-based feeds
- Feed templates
---
### Low Priority
9. **Analytics**
- User engagement metrics
- Content popularity
- API usage analytics
- Performance metrics
10. **Batch Operations**
- Bulk post creation
- Bulk comment actions
- Bulk follow operations
11. **Spatial/Media**
- Location support
- Media attachments handling
- Image/video upload
## Future Enhancements
### Architecture Improvements
1. **Type Safety**
- Pydantic models for all inputs and outputs
- Strict type checking
- Auto-completion support
- Better IDE integration
2. **Async Support**
- Async wrappers for all operations
- Pooling for concurrency
- Automatic retries with exponential backoff
- Connection pooling
3. **Configuration Management**
- Centralized config file
- Environment-specific configs
- Secrets management integration
- Hot-reload capability
4. **Better Testing**
- Comprehensive test suite
- Integration tests
- Mock API server
- Property-based testing
### Developer Features
5. **CLI Tool**
- `trustcafe` command-line interface
- Interactive workflows
- Dry-run mode
- Export/import operations
6. **Code Generation**
- Auto-generate jobs from API spec
- Wrapper generators
- Type definitions from OpenAPI spec
7. **Plugin System**
- Extension points
- Custom jobs
- Custom middleware
- Hooks system
---
### User Experience Improvements
8. **Better Documentation**
- API sandbox/demo site
- Data models and schemas
- Migration guides
- FAQ section
9. **Developer Experience**
- SDK-style TypeScript version
- Better examples
- Quick start guides
- Video tutorials
10. **Monitoring and observability**
- Request/response logging
- Metrics and tracing
- Health checks
- Alerting hooks
## Contributing
### Adding New Jobs
```python
# 1. Create job file in src/trustcafeapiwrapper/jobs/
# Example: src/trustcafeapiwrapper/jobs/example/my_job.py
from trustcafeapiwrapper.apiclient import APIClient
def my_job(API: APIClient, param1: str, param2: int) -> dict:
"""
Create your job here.
Args:
API: The APIClient instance
param1: Description of param1
param2: Description of param2
Returns:
dict: Whatever the API returns
"""
response = API.make_request(
"GET",
"endpoint",
f"path/to/resource/{param1}",
authenticate=True
)
return response
# 2. Export from __init__.py (if needed)
# Keep __init__.py empty per current structure
```
### Adding New Wrappers
```python
# Example: src/trustcafeapiwrapper/wrappers/example/my_wrapper.py
from trustcafeapiwrapper.utils.get_parent_pksk_from_path import get_parent_pksk_from_path
def my_wrapper(
param1: str,
param2: int,
optional_param: str = None
):
"""
Create a wrapper that makes a job easier.
Args:
param1: Required parameter description
param2: Required parameter description
optional_param: Optional parameter with default
Returns:
dict: Job specification for wrapped execution
"""
parent_pksk = get_parent_pksk_from_path("/")
return {
"job_function": "example.my_job",
"payload": {
"param1": param1,
"param2": param2,
"optional_param": optional_param,
}
}
# 3. Export from __init__.py (if needed)
```
### Testing
```python
# Create test file: tests/test_my_job.py
import unittest
from unittest.mock import patch
from trustcafeapiwrapper.apiclient import APIClient
from trustcafeapiwrapper.jobs.example.my_job import my_job
class TestMyJob(unittest.TestCase):
def setUp(self):
self.api_client = APIClient(
client_id="test-id",
client_secret="test-secret",
debug=False
)
@patch('trustcafeapiwrapper.apiclient.requests.Session.request')
def test_my_job(self, mock_request):
mock_request.return_value.json.return_value = {"result": "success"}
mock_request.return_value.raise_for_status.return_value = None
result = my_job(self.api_client, "param1", 123)
self.assertEqual(result["result"], "success")
if __name__ == '__main__':
unittest.main()
```
### Code Style
- Follow PEP 8
- Use type hints
- Write comprehensive docstrings
- Add error handling
- Include examples in docstrings
### Testing Requirements
- All new features must have tests
- Test edge cases
- Mock external API calls
- Achieve 80%+ test coverage
## Testing Guide
### Current Test Structure
```
tests/
├── apiclient.py # API client tests
├── test_api_client.py # Legacy test file
```
### Running Tests
```bash
# Run all tests
pytest
# Run specific test file
pytest tests/apiclient.py
# Run with coverage
pytest --cov=src/trustcafeapiwrapper
# Run with verbose output
pytest -v
# Run specific test
pytest tests/apiclient.py::TestAPIClient::test_initialization
```
### Test Best Practices
1. **Mock External Calls**
```python
@patch('trustcafeapiwrapper.apiclient.requests.Session.request')
def test_implementation(self, mock_request):
mock_request.return_value.json.return_value = {"mock": "response"}
# Test logic
```
2. **Test All Paths**
```python
def test_error_handling():
# Success case
assert result == expected
# Error cases
with pytest.raises(ValueError):
handle_error_param
```
3. **Keep Tests Independent**
- Each test should work alone
- No shared state between tests
- Use setUp for common setup
4. **Clear Test Names**
```python
# Good
def test_userprofile_get_with_invalid_username_raises_error(self):
# Test description
# Bad
def test_invalid(self):
# Unclear what this tests
```
## Extending the Wrapper
### Adding Configuration
```python
# Add to pyproject.toml
[tool.poetry.dependencies]
trustcafeapiwrapper = { path = "." }
[project.optional-dependencies]
dev = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"httpx>=0.24.0",
]
test = [
"pytest>=7.0.0",
]
```
### Adding New Endpoints
Edit `src/trustcafeapiwrapper/apiclient.py`:
```python
endpoints = {
"alpha": {
# ... existing endpoints ...
"newendpoint": "https://api.example.com/alpha/",
},
}
```
### Adding Utility Functions
```python
# src/trustcafeapiwrapper/utils/my_utilty.py
def my_utility_function(input_data: str) -> dict:
"""Utility function for common data transformations"""
# Implementation
return transformed_data
```
---
## Getting Started with Development
1. **Fork the repository**
- Add your fork as a remote
- Keep pull requests to main
2. **Set up development environment**
```bash
# Clone your fork
git clone https://gitlab.com/your-username/trustcafe-api-wrapper.git
cd trustcafe-api-wrapper
# Install dependencies
pip install -e ".[dev]"
# Run tests
pytest
```
3. **Create feature branch**
```bash
git checkout -b feature/your-feature-name
```
4. **Make changes**
- Add code
- Add tests
- Update documentation
5. **Submit PR**
- Branch must be up-to-date with main
- Include helpful description
- Reference related issues
---
## Resources
- **Stavanger AI**: See the development.md debate notes
- **GitLab Repository**: https://gitlab.com/trustcafe/trustcafe-api-wrapper
- **WikiTribune Team**: Contact via company channels
- **Packaging**: pyproject.toml for dependencies and metadata
---
**Stay Updated**: Read development.md for ongoing decisions and discussions. This file is intentionally loose to capture evolving thoughts and debates as the project progresses.