trustcafe-api-wrapper/development.md

# TrustCafé API Wrapper - Development Guide

This document covers development considerations, known limitations, and contribution guidelines for the TrustCafé API wrapper.

## Table of Contents

1. [Project Structure](#project-structure)
2. [Design Decisions](#design-decisions)
3. [Known Limitations](#known-limitations)
4. [Missing Features](#missing-features)
5. [Future Enhancements](#future-enhancements)
6. [Contributing](#contributing)
7. [Testing Guide](#testing-guide)
8. [Extending the Wrapper](#extending-the-wrapper)

## Project Structure

```
trustcafe-api-wrapper/
├── README.md                    # Main documentation
├── documentation.md             # Basic usage documentation
├── development.md              # This file - development notes
├── pyproject.toml             # Project configuration
├── setup.py                   # Legacy setup file
├── development.md
├── .gitlab-ci.yml            # CI/CD configuration
└── src/
    └── trustcafeapiwrapper/
        ├── __init__.py           # Package exports
        ├── apiclient.py          # Core API client (218 lines)
        ├── jobs/                 # API job functions
        │   ├── __init__.py
        │   ├── post/
        │   ├── comment/
        │   ├── userprofile/
        │   ├── follow/
        │   ├── vote/
        │   ├── reaction/
        │   ├── notification/
        │   ├── trust/
        │   ├── branch/
        │   └── feed/
        ├── wrappers/            # High-level wrappers
        │   ├── __init__.py
        │   ├── post/
        │   └── comment/
        └── utils/               # Helper functions
            ├── __init__.py
            ├── make_comment_sk.py
            ├── make_post_sk.py
            ├── get_post_pksk.py
            ├── get_userprofile_pksk_from_slug.py
            ├── get_user_slug_from_path.py
            ├── get_entity_from_str.py
            ├── get_parent_pksk_from_path.py
            └── get_child_spksk_from_paths.py
```

## Design Decisions

### 1. Naming Conventions

**Jobs vs. Wrappers:**

| Aspect | Jobs | Wrappers |
|--------|------|----------|
| Naming | Verb (`create_x`, `get_x`) | Noun phrase (`create_post`, `update_post`) |
| Pattern | Inconsistent | Standardized (`create_noun`, `update_noun`) |
| Usage | Flexible | Optimized for common cases |
| Type | Function | Function returning dict with job spec |

**Question: Should these be called `jobs`, `tasks`, `apirequests`, `rqsts`, or `apicalls`?**
- **Chosen:** `jobs` - Existing dependency (requests)
- **Alternatives considered:** `tasks` (too broad), `apirequests` (confusing with HTTP library)

**Thread with discussion:** See GitLab issues

### 2. Job Execution Method

**Question: Should jobs be called with `dot notation` (e.g., `userprofile.get`) or `slashes` (e.g., `userprofile/get`)?**
- **Chosen:** Dot notation for module style
- **Flexibility:** Supports both strings for dynamic loading and functions for static use
- **Pattern:**

```python
# String-based (dynamic) - works for variable jobs
API.run_job('userprofile.get', username)

# Function-based (static) - good for explicit calls
API.run_job(userprofile_get, username)
```

### 3. Parameter Validation

**Question: Should validation happen in jobs or let the server do it?**
- **Chosen:** Server-side validation with pydantic at APIClient level
- **Reasoning:**
  - Server provides authoritative validation
  - Reduced redundancy
  - Known API responses
- **Trade-off:** Misuse symptoms appear when server rejects invalid data

### 4. Wrapper Architecture

**Question: What's the proper name for the wrapper system?**
- **Chosen:** `wrapped()` as the calling method
- **Alternatives considered:** None after review
- **Pros:** Self-documenting parameter handling
- **Cons:** Awkward name

### 5. Verbosity in Noun Names

**Question: Use `post.vote` vs `vote.cast`?**
- **Chosen:** `post.vote` for entity-based paths
- **Rationale:**
  - Better semantic structure: `object.action`
  - Clearer context (what are we voting on?)
  - Consistent with TrustCafé's own endpoints

**Examples:**
- ✅ `post.comment` (comment on a post)
- ✅ `comment.vote` (vote on a comment)
- ⚠️ vs `comment.comment` (commenting on a comment)

### 6. Database vs. Display Names

**Question: Use `trust` or RelTrust in code?**
- **Chosen:** Display names (`trust`, `follow`, `block`)
- **Misleading risk:** Database uses RelTrust, RelFollow, UserBlock
- **Plan:** Internal mapping in utility functions

**Database Inconsistencies:**
- `trust` ↔ `RelTrust`
- `follow` ↔ `RelFollow`
- `block` ↔ `UserBlock` (not `RelBlock`)
- `mute` ↔ `Mute`

### 7. Entity Hierarchy Organization

**Question: Organize as `user.follow` or `follow.follow`?**
- **Chosen:** `user.follow` (object-centric) for most cases
- **Complex cases:** `comment.vote`, `post.comment.comment` (both possible)

**Plan:**
- Simple cases: Entity-based (`user.follow`, `post.comment`)
- Complex cases: Action-centric (`comment.vote`)
- Organize by common pattern, not dogmatically

## Known Limitations

### 1. Incomplete Job Coverage

**Status:** Partial coverage of API endpoints

**Currently Implemented:**
- ✅ Post operations: create, get, update, listall, listpublic, listbybranch, listbyuserprofile, listremoved
- ✅ Comment operations: create, listtbypostid
- ✅ UserProfile: get
- ✅ Follow: follow
- ✅ Vote: votecast
- ✅ Reaction: reacttosomething, getbyparent, listbyparent
- ✅ Notification: listnotifications, markallasread
- ✅ Trust: createorupdate, listbyusersinit, listbyuserhas
- ✅ Branch: get, listbyname
- ✅ Feed: cafefeed, following
- ⚠️ Block: job created but not connected to server
- ⚠️ Mute: job created but not connected to server

**Missing/Incomplete:**
- ❌ Post status operations (delete, pin, etc.)
- ❌ Full post metadata operations
- ❌ Advanced comment features
- ❌ User management operations
- ❌ Search functionality
- ❌ Moderation operations
- ❌ Analytics endpoints

---

### 2. Enums Not Strictly Typed

**Issue:** Some parameter types are not enforced

**Example:**
```python
# Acceptable (but should be):
API.run_job('vote.votecast', {..., "voteType": "UP"})  # Should error?

# Better to use type hints:
vote_type: Literal["up", "down"]
```

**Affected APIs:**
- voteType (should be "up" or "down")
- reactionType (should match TrustCafé enum values)
- trustType (should be known set)

**Solution:** Add pydantic models for requests and responses

---

### 3. No Async Support

**Issue:** All requests are synchronous (blocking)

**Impact:**
- Serial request execution
- Long waits for multiple calls
- Limited throughput

**Solution:**
- Add async/await support using `aiohttp`
- Provide sync wrapper that calls async
- Consider for future major version

---

### 4. Limited Timeout Configuration

**Issue:** Hardcoded 20-second timeout

**Current Code:**
```python
# apiclient.py line 87
response = session.request(method.upper(), url, json=data, headers=headers, timeout=20)
```

**Problem:**
- No way to configure timeout per request
- Default works but is a guess
- Network issues hard to diagnose without flexibility

**Solution:**
```python
# Add timeout parameter
def make_request(self, method, endpoint, path, data=None,
                 authenticate=True, query_params=None,
                 timeout=20):  # Add timeout parameter
```

---

### 5. No Detailed Error Types

**Issue:** All errors are generic `Exception` or `ValueError`

**Current Code:**
```python
except Exception as e:
    raise ConnectionError(f"An error occurred while making the request: {e}")
```

**Limitations:**
- No authentication-specific errors
- No validation errors
- No permission errors
- Error type doesn't give context

**Solution:** Create custom error classes:
```python
class APIError(Exception):
    """Base API error"""

class AuthenticationError(APIError):
    """Authentication failures"""

class ValidationError(APIError):
    """Invalid parameter values"""
```

---

### 6. Token File Handling

**Issue:** Token file stored in working directory (influenced by os.getcwd())

**Problem:**
- Cross-platform issues
- Permission problems
- Can't control location from outside

**Current Code:**
```python
token_data_path = f"token_data_{self.environment}.json"
```

**Solution:**
```python
# User-configurable path
import os

default_path = os.path.join(os.getcwd(), f"token_data_{self.environment}.json")

API.handle_token(token_data_path=os.getenv("TRUSTCAFE_TOKEN_PATH", default_path))
```

---

### 7. Database-Display Name Mismatch

**Issue:** Internal vs. external naming不一致

**Examples:**
- Display: `trust` ↔ Database: `RelTrust`
- Display: `follow` ↔ Database: `RelFollow`
- Display: `block` ↔ Database: `UserBlock` (not `RelBlock`)
- Display: `mute` ↔ Database: `Mute`

**Solution:**
- Map most common names (trust, follow, mute)
- Add mappings in utility functions
- Document naming inconsistencies
- Consider renaming display names for consistency

---

### 8. Unused Dependencies

**Cleanup opportunity:** Some dependencies may not be used

**Current Dependencies:**
```toml
dependencies = [
    "dotenv>=0.9.9",       # Used in development.md example
    "pydantic>=2.12.5",    # Used for SecretStr
    "requests>=2.33.1",    # Used for HTTP requests
    "simplejson>=3.20.2",  # Used for JSON parsing
]
```

**Potential Removal:**
- `simplejson`: Can use Python's built-in json
- **Note:** Keep for now to minimize risk

---

### 9. Hard-coded Endpoints

**Issue:** API base URLs are hard-coded

**Current Code:**
```python
endpoints = {
    "alpha": {
        "audrey": "https://eso1of8gqd.execute-api.us-east-1.amazonaws.com/alpha/",
        # ...
    },
    "production": {
        # ...
    }
}
```

**Problem:**
- Environment changes will break
- No hot-reload
- Same URLs across versions

**Solution:**
```python
# Could be environment variables
endpoints = {
    "alpha": {
        "audrey": os.getenv("AUDREY_ALPHA_URL", "default-alpha-url"),
    },
    # ...
}
```

## Missing Features

### High Priority

1. **Complete Post Operations**
   - Post status: pin, unpin, archive, delete
   - Post editing: editing mode handling
   - Post metadata bulk operations
   - Post history/versioning

2. **User Management Jobs**
   - Block/unblock users
   - Mute/unmute users
   - User profile management
   - User settings

3. **Search Functionality**
   - Full-text search
   - Advanced filters
   - Query builder

4. **Comment Enhancements**
   - Comment editing
   - Comment deletion
   - Comment threading
   - Comment moderation

---

### Medium Priority

5. **Reaction UI**
   - Multiple response types (heart, fire, thumbs-up, etc.)
   - Reaction history
   - Reaction statistics

6. **Trust System**
   - Trust ranking algorithms
   - Trust network visualization
   - Block followers
   - Trust relationships with others' trust

7. **Notification Enhancements**
   - Notification callbacks/webhooks
   - Notification types documentation
   - Notification preferences
   - Notification filtering

8. **Feed Customization**
   - Filter classes
   - Schedule-based feeds
   - Feed templates

---

### Low Priority

9. **Analytics**
   - User engagement metrics
   - Content popularity
   - API usage analytics
   - Performance metrics

10. **Batch Operations**
    - Bulk post creation
    - Bulk comment actions
    - Bulk follow operations

11. **Spatial/Media**
    - Location support
    - Media attachments handling
    - Image/video upload

## Future Enhancements

### Architecture Improvements

1. **Type Safety**
   - Pydantic models for all inputs and outputs
   - Strict type checking
   - Auto-completion support
   - Better IDE integration

2. **Async Support**
   - Async wrappers for all operations
   - Pooling for concurrency
   - Automatic retries with exponential backoff
   - Connection pooling

3. **Configuration Management**
   - Centralized config file
   - Environment-specific configs
   - Secrets management integration
   - Hot-reload capability

4. **Better Testing**
   - Comprehensive test suite
   - Integration tests
   - Mock API server
   - Property-based testing

### Developer Features

5. **CLI Tool**
   - `trustcafe` command-line interface
   - Interactive workflows
   - Dry-run mode
   - Export/import operations

6. **Code Generation**
   - Auto-generate jobs from API spec
   - Wrapper generators
   - Type definitions from OpenAPI spec

7. **Plugin System**
   - Extension points
   - Custom jobs
   - Custom middleware
   - Hooks system

---

### User Experience Improvements

8. **Better Documentation**
   - API sandbox/demo site
   - Data models and schemas
   - Migration guides
   - FAQ section

9. **Developer Experience**
   - SDK-style TypeScript version
   - Better examples
   - Quick start guides
   - Video tutorials

10. **Monitoring and observability**
    - Request/response logging
    - Metrics and tracing
    - Health checks
    - Alerting hooks

## Contributing

### Adding New Jobs

```python
# 1. Create job file in src/trustcafeapiwrapper/jobs/
# Example: src/trustcafeapiwrapper/jobs/example/my_job.py

from trustcafeapiwrapper.apiclient import APIClient

def my_job(API: APIClient, param1: str, param2: int) -> dict:
    """
    Create your job here.

    Args:
        API: The APIClient instance
        param1: Description of param1
        param2: Description of param2

    Returns:
        dict: Whatever the API returns
    """
    response = API.make_request(
        "GET",
        "endpoint",
        f"path/to/resource/{param1}",
        authenticate=True
    )
    return response

# 2. Export from __init__.py (if needed)
# Keep __init__.py empty per current structure
```

### Adding New Wrappers

```python
# Example: src/trustcafeapiwrapper/wrappers/example/my_wrapper.py

from trustcafeapiwrapper.utils.get_parent_pksk_from_path import get_parent_pksk_from_path

def my_wrapper(
    param1: str,
    param2: int,
    optional_param: str = None
):
    """
    Create a wrapper that makes a job easier.

    Args:
        param1: Required parameter description
        param2: Required parameter description
        optional_param: Optional parameter with default

    Returns:
        dict: Job specification for wrapped execution
    """
    parent_pksk = get_parent_pksk_from_path("/")

    return {
        "job_function": "example.my_job",
        "payload": {
            "param1": param1,
            "param2": param2,
            "optional_param": optional_param,
        }
    }

# 3. Export from __init__.py (if needed)
```

### Testing

```python
# Create test file: tests/test_my_job.py

import unittest
from unittest.mock import patch
from trustcafeapiwrapper.apiclient import APIClient
from trustcafeapiwrapper.jobs.example.my_job import my_job

class TestMyJob(unittest.TestCase):
    def setUp(self):
        self.api_client = APIClient(
            client_id="test-id",
            client_secret="test-secret",
            debug=False
        )

    @patch('trustcafeapiwrapper.apiclient.requests.Session.request')
    def test_my_job(self, mock_request):
        mock_request.return_value.json.return_value = {"result": "success"}
        mock_request.return_value.raise_for_status.return_value = None

        result = my_job(self.api_client, "param1", 123)
        self.assertEqual(result["result"], "success")

if __name__ == '__main__':
    unittest.main()
```

### Code Style

- Follow PEP 8
- Use type hints
- Write thorough docstrings
- Add error handling
- Include examples in docstrings

### Testing Requirements

- All new features must have tests
- Test edge cases
- Mock external API calls
- Achieve 80%+ test coverage

## Testing Guide

### Current Test Structure

```
tests/
├── apiclient.py        # API client tests
├── test_api_client.py   # Legacy test file
```

### Running Tests

```bash
# Run all tests
pytest

# Run specific test file
pytest tests/apiclient.py

# Run with coverage
pytest --cov=src/trustcafeapiwrapper

# Run with verbose output
pytest -v

# Run specific test
pytest tests/apiclient.py::TestAPIClient::test_initialization
```

### Test Best Practices

1. **Mock External Calls**
   ```python
   @patch('trustcafeapiwrapper.apiclient.requests.Session.request')
   def test_implementation(self, mock_request):
       mock_request.return_value.json.return_value = {"mock": "response"}
       # Test logic
   ```

2. **Test All Paths**
   ```python
   def test_error_handling():
       # Success case
       assert result == expected

       # Error cases
       with pytest.raises(ValueError):
           handle_error_param
   ```

3. **Keep Tests Independent**
   - Each test should work alone
   - No shared state between tests
   - Use setUp for common setup

4. **Clear Test Names**
   ```python
   # Good
   def test_userprofile_get_with_invalid_username_raises_error(self):
       # Test description

   # Bad
   def test_invalid(self):
       # Unclear what this tests
   ```

## Extending the Wrapper

### Adding Configuration

```python
# Add to pyproject.toml
[tool.poetry.dependencies]
trustcafeapiwrapper = { path = "." }

[project.optional-dependencies]
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "httpx>=0.24.0",
]

test = [
    "pytest>=7.0.0",
]
```

### Adding New Endpoints

Edit `src/trustcafeapiwrapper/apiclient.py`:

```python
endpoints = {
    "alpha": {
        # ... existing endpoints ...
        "newendpoint": "https://api.example.com/alpha/",
    },
}
```

### Adding Utility Functions

```python
# src/trustcafeapiwrapper/utils/my_utilty.py

def my_utility_function(input_data: str) -> dict:
    """Utility function for common data transformations"""

    # Implementation

    return transformed_data
```

---

## Getting Started with Development

1. **Fork the repository**
   - Add your fork as a remote
   - Keep pull requests to main

2. **Set up development environment**
   ```bash
   # Clone your fork
   git clone https://gitlab.com/your-username/trustcafe-api-wrapper.git
   cd trustcafe-api-wrapper

   # Install dependencies
   pip install -e ".[dev]"

   # Run tests
   pytest
   ```

3. **Create feature branch**
   ```bash
   git checkout -b feature/your-feature-name
   ```

4. **Make changes**
   - Add code
   - Add tests
   - Update documentation

5. **Submit PR**
   - Branch must be up-to-date with main
   - Include helpful description
   - Reference related issues

---

## Resources

- **Stavanger AI**: See the development.md debate notes
- **GitLab Repository**: https://gitlab.com/trustcafe/trustcafe-api-wrapper
- **WikiTribune Team**: Contact via company channels
- **Packaging**: pyproject.toml for dependencies and metadata

---

**Stay Updated**: Read development.md for ongoing decisions and discussions. This file is intentionally loose to capture evolving thoughts and debates as the project progresses.