trustcafe-api-wrapper/development.md

TrustCafé API Wrapper - Development Guide

This document covers development considerations, known limitations, and contribution guidelines for the TrustCafé API wrapper.

Table of Contents

1. Project Structure
2. Design Decisions
3. Known Limitations
4. Missing Features
5. Future Enhancements
6. Contributing
7. Testing Guide
8. Extending the Wrapper

Project Structure

trustcafe-api-wrapper/
├── README.md                    # Main documentation
├── documentation.md             # Basic usage documentation
├── development.md              # This file - development notes
├── pyproject.toml             # Project configuration
├── setup.py                   # Legacy setup file
├── development.md
├── .gitlab-ci.yml            # CI/CD configuration
└── src/
    └── trustcafeapiwrapper/
        ├── __init__.py           # Package exports
        ├── apiclient.py          # Core API client (218 lines)
        ├── jobs/                 # API job functions
        │   ├── __init__.py
        │   ├── post/
        │   ├── comment/
        │   ├── userprofile/
        │   ├── follow/
        │   ├── vote/
        │   ├── reaction/
        │   ├── notification/
        │   ├── trust/
        │   ├── branch/
        │   └── feed/
        ├── wrappers/            # High-level wrappers
        │   ├── __init__.py
        │   ├── post/
        │   └── comment/
        └── utils/               # Helper functions
            ├── __init__.py
            ├── make_comment_sk.py
            ├── make_post_sk.py
            ├── get_post_pksk.py
            ├── get_userprofile_pksk_from_slug.py
            ├── get_user_slug_from_path.py
            ├── get_entity_from_str.py
            ├── get_parent_pksk_from_path.py
            └── get_child_spksk_from_paths.py

Design Decisions

1. Naming Conventions

Jobs vs. Wrappers:

Aspect	Jobs	Wrappers
Naming	Verb (create_x, get_x)	Noun phrase (create_post, update_post)
Pattern	Inconsistent	Standardized (create_noun, update_noun)
Usage	Flexible	Optimized for common cases
Type	Function	Function returning dict with job spec

Question: Should these be called jobs, tasks, apirequests, rqsts, or apicalls?

• Chosen: jobs - Existing dependency (requests)
• Alternatives considered: tasks (too broad), apirequests (confusing with HTTP library)

Thread with discussion: See GitLab issues

2. Job Execution Method

Question: Should jobs be called with dot notation (e.g., userprofile.get) or slashes (e.g., userprofile/get)?

• Chosen: Dot notation for module style
• Flexibility: Supports both strings for dynamic loading and functions for static use
• Pattern:

# String-based (dynamic) - works for variable jobs
API.run_job('userprofile.get', username)

# Function-based (static) - good for explicit calls
API.run_job(userprofile_get, username)

3. Parameter Validation

Question: Should validation happen in jobs or let the server do it?

• Chosen: Server-side validation with pydantic at APIClient level
• Reasoning:
  • Server provides authoritative validation
  • Reduced redundancy
  • Known API responses
• Trade-off: Misuse symptoms appear when server rejects invalid data

4. Wrapper Architecture

Question: What's the proper name for the wrapper system?

• Chosen: wrapped() as the calling method
• Alternatives considered: None after review
• Pros: Self-documenting parameter handling
• Cons: Awkward name

5. Verbosity in Noun Names

Question: Use post.vote vs vote.cast?

• Chosen: post.vote for entity-based paths
• Rationale:
  • Better semantic structure: object.action
  • Clearer context (what are we voting on?)
  • Consistent with TrustCafé's own endpoints

Examples:

• ✅ post.comment (comment on a post)
• ✅ comment.vote (vote on a comment)
• ⚠️ vs comment.comment (commenting on a comment)

6. Database vs. Display Names

Question: Use trust or RelTrust in code?

• Chosen: Display names (trust, follow, block)
• Misleading risk: Database uses RelTrust, RelFollow, UserBlock
• Plan: Internal mapping in utility functions

Database Inconsistencies:

• trust ↔ RelTrust
• follow ↔ RelFollow
• block ↔ UserBlock (not RelBlock)
• mute ↔ Mute

7. Entity Hierarchy Organization

Question: Organize as user.follow or follow.follow?

• Chosen: user.follow (object-centric) for most cases
• Complex cases: comment.vote, post.comment.comment (both possible)

Plan:

• Simple cases: Entity-based (user.follow, post.comment)
• Complex cases: Action-centric (comment.vote)
• Organize by common pattern, not dogmatically

Known Limitations

1. Incomplete Job Coverage

Status: Partial coverage of API endpoints

Currently Implemented:

• ✅ Post operations: create, get, update, listall, listpublic, listbybranch, listbyuserprofile, listremoved
• ✅ Comment operations: create, listtbypostid
• ✅ UserProfile: get
• ✅ Follow: follow
• ✅ Vote: votecast
• ✅ Reaction: reacttosomething, getbyparent, listbyparent
• ✅ Notification: listnotifications, markallasread
• ✅ Trust: createorupdate, listbyusersinit, listbyuserhas
• ✅ Branch: get, listbyname
• ✅ Feed: cafefeed, following
• ⚠️ Block: job created but not connected to server
• ⚠️ Mute: job created but not connected to server

Missing/Incomplete:

• ❌ Post status operations (delete, pin, etc.)
• ❌ Full post metadata operations
• ❌ Advanced comment features
• ❌ User management operations
• ❌ Search functionality
• ❌ Moderation operations
• ❌ Analytics endpoints

---

2. Enums Not Strictly Typed

Issue: Some parameter types are not enforced

Example:

# Acceptable (but should be):
API.run_job('vote.votecast', {..., "voteType": "UP"})  # Should error?

# Better to use type hints:
vote_type: Literal["up", "down"]

Affected APIs:

• voteType (should be "up" or "down")
• reactionType (should match TrustCafé enum values)
• trustType (should be known set)

Solution: Add pydantic models for requests and responses

---

3. No Async Support

Issue: All requests are synchronous (blocking)

Impact:

• Serial request execution
• Long waits for multiple calls
• Limited throughput

Solution:

• Add async/await support using aiohttp
• Provide sync wrapper that calls async
• Consider for future major version

---

4. Limited Timeout Configuration

Issue: Hardcoded 20-second timeout

Current Code:

# apiclient.py line 87
response = session.request(method.upper(), url, json=data, headers=headers, timeout=20)

Problem:

• No way to configure timeout per request
• Default works but is a guess
• Network issues hard to diagnose without flexibility

Solution:

# Add timeout parameter
def make_request(self, method, endpoint, path, data=None,
                 authenticate=True, query_params=None,
                 timeout=20):  # Add timeout parameter

---

5. No Detailed Error Types

Issue: All errors are generic Exception or ValueError

Current Code:

except Exception as e:
    raise ConnectionError(f"An error occurred while making the request: {e}")

Limitations:

• No authentication-specific errors
• No validation errors
• No permission errors
• Error type doesn't give context

Solution: Create custom error classes:

class APIError(Exception):
    """Base API error"""

class AuthenticationError(APIError):
    """Authentication failures"""

class ValidationError(APIError):
    """Invalid parameter values"""

---

6. Token File Handling

Issue: Token file stored in working directory (influenced by os.getcwd())

Problem:

• Cross-platform issues
• Permission problems
• Can't control location from outside

Current Code:

token_data_path = f"token_data_{self.environment}.json"

Solution:

# User-configurable path
import os

default_path = os.path.join(os.getcwd(), f"token_data_{self.environment}.json")

API.handle_token(token_data_path=os.getenv("TRUSTCAFE_TOKEN_PATH", default_path))

---

7. Database-Display Name Mismatch

Issue: Internal vs. external naming不一致

Examples:

• Display: trust ↔ Database: RelTrust
• Display: follow ↔ Database: RelFollow
• Display: block ↔ Database: UserBlock (not RelBlock)
• Display: mute ↔ Database: Mute

Solution:

• Map most common names (trust, follow, mute)
• Add mappings in utility functions
• Document naming inconsistencies
• Consider renaming display names for consistency

---

8. Unused Dependencies

Cleanup opportunity: Some dependencies may not be used

Current Dependencies:

dependencies = [
    "dotenv>=0.9.9",       # Used in development.md example
    "pydantic>=2.12.5",    # Used for SecretStr
    "requests>=2.33.1",    # Used for HTTP requests
    "simplejson>=3.20.2",  # Used for JSON parsing
]

Potential Removal:

• simplejson: Can use Python's built-in json
• Note: Keep for now to minimize risk

---

9. Hard-coded Endpoints

Issue: API base URLs are hard-coded

Current Code:

endpoints = {
    "alpha": {
        "audrey": "https://eso1of8gqd.execute-api.us-east-1.amazonaws.com/alpha/",
        # ...
    },
    "production": {
        # ...
    }
}

Problem:

• Environment changes will break
• No hot-reload
• Same URLs across versions

Solution:

# Could be environment variables
endpoints = {
    "alpha": {
        "audrey": os.getenv("AUDREY_ALPHA_URL", "default-alpha-url"),
    },
    # ...
}

Missing Features

High Priority

1. Complete Post Operations

   • Post status: pin, unpin, archive, delete
   • Post editing: editing mode handling
   • Post metadata bulk operations
   • Post history/versioning

2. User Management Jobs

   • Block/unblock users
   • Mute/unmute users
   • User profile management
   • User settings

3. Search Functionality

   • Full-text search
   • Advanced filters
   • Query builder

4. Comment Enhancements

   • Comment editing
   • Comment deletion
   • Comment threading
   • Comment moderation

---

Medium Priority

5. Reaction UI

   • Multiple response types (heart, fire, thumbs-up, etc.)
   • Reaction history
   • Reaction statistics

6. Trust System

   • Trust ranking algorithms
   • Trust network visualization
   • Block followers
   • Trust relationships with others' trust

7. Notification Enhancements

   • Notification callbacks/webhooks
   • Notification types documentation
   • Notification preferences
   • Notification filtering

8. Feed Customization

   • Filter classes
   • Schedule-based feeds
   • Feed templates

---

Low Priority

9. Analytics

   • User engagement metrics
   • Content popularity
   • API usage analytics
   • Performance metrics

10. Batch Operations

    • Bulk post creation
    • Bulk comment actions
    • Bulk follow operations

11. Spatial/Media

    • Location support
    • Media attachments handling
    • Image/video upload

Future Enhancements

Architecture Improvements

1. Type Safety

   • Pydantic models for all inputs and outputs
   • Strict type checking
   • Auto-completion support
   • Better IDE integration

2. Async Support

   • Async wrappers for all operations
   • Pooling for concurrency
   • Automatic retries with exponential backoff
   • Connection pooling

3. Configuration Management

   • Centralized config file
   • Environment-specific configs
   • Secrets management integration
   • Hot-reload capability

4. Better Testing

   • Comprehensive test suite
   • Integration tests
   • Mock API server
   • Property-based testing

Developer Features

5. CLI Tool

   • trustcafe command-line interface
   • Interactive workflows
   • Dry-run mode
   • Export/import operations

6. Code Generation

   • Auto-generate jobs from API spec
   • Wrapper generators
   • Type definitions from OpenAPI spec

7. Plugin System

   • Extension points
   • Custom jobs
   • Custom middleware
   • Hooks system

---

User Experience Improvements

8. Better Documentation

   • API sandbox/demo site
   • Data models and schemas
   • Migration guides
   • FAQ section

9. Developer Experience

   • SDK-style TypeScript version
   • Better examples
   • Quick start guides
   • Video tutorials

10. Monitoring and observability

    • Request/response logging
    • Metrics and tracing
    • Health checks
    • Alerting hooks

Contributing

Adding New Jobs

# 1. Create job file in src/trustcafeapiwrapper/jobs/
# Example: src/trustcafeapiwrapper/jobs/example/my_job.py

from trustcafeapiwrapper.apiclient import APIClient

def my_job(API: APIClient, param1: str, param2: int) -> dict:
    """
    Create your job here.

    Args:
        API: The APIClient instance
        param1: Description of param1
        param2: Description of param2

    Returns:
        dict: Whatever the API returns
    """
    response = API.make_request(
        "GET",
        "endpoint",
        f"path/to/resource/{param1}",
        authenticate=True
    )
    return response

# 2. Export from __init__.py (if needed)
# Keep __init__.py empty per current structure

Adding New Wrappers

# Example: src/trustcafeapiwrapper/wrappers/example/my_wrapper.py

from trustcafeapiwrapper.utils.get_parent_pksk_from_path import get_parent_pksk_from_path

def my_wrapper(
    param1: str,
    param2: int,
    optional_param: str = None
):
    """
    Create a wrapper that makes a job easier.

    Args:
        param1: Required parameter description
        param2: Required parameter description
        optional_param: Optional parameter with default

    Returns:
        dict: Job specification for wrapped execution
    """
    parent_pksk = get_parent_pksk_from_path("/")

    return {
        "job_function": "example.my_job",
        "payload": {
            "param1": param1,
            "param2": param2,
            "optional_param": optional_param,
        }
    }

# 3. Export from __init__.py (if needed)

Testing

# Create test file: tests/test_my_job.py

import unittest
from unittest.mock import patch
from trustcafeapiwrapper.apiclient import APIClient
from trustcafeapiwrapper.jobs.example.my_job import my_job

class TestMyJob(unittest.TestCase):
    def setUp(self):
        self.api_client = APIClient(
            client_id="test-id",
            client_secret="test-secret",
            debug=False
        )

    @patch('trustcafeapiwrapper.apiclient.requests.Session.request')
    def test_my_job(self, mock_request):
        mock_request.return_value.json.return_value = {"result": "success"}
        mock_request.return_value.raise_for_status.return_value = None

        result = my_job(self.api_client, "param1", 123)
        self.assertEqual(result["result"], "success")

if __name__ == '__main__':
    unittest.main()

Code Style

• Follow PEP 8
• Use type hints
• Write thorough docstrings
• Add error handling
• Include examples in docstrings

Testing Requirements

• All new features must have tests
• Test edge cases
• Mock external API calls
• Achieve 80%+ test coverage

Testing Guide

Current Test Structure

tests/
├── apiclient.py        # API client tests
├── test_api_client.py   # Legacy test file

Running Tests

# Run all tests
pytest

# Run specific test file
pytest tests/apiclient.py

# Run with coverage
pytest --cov=src/trustcafeapiwrapper

# Run with verbose output
pytest -v

# Run specific test
pytest tests/apiclient.py::TestAPIClient::test_initialization

Test Best Practices

1. Mock External Calls

   @patch('trustcafeapiwrapper.apiclient.requests.Session.request')
   def test_implementation(self, mock_request):
       mock_request.return_value.json.return_value = {"mock": "response"}
       # Test logic
   

2. Test All Paths

   def test_error_handling():
       # Success case
       assert result == expected
   
       # Error cases
       with pytest.raises(ValueError):
           handle_error_param
   

3. Keep Tests Independent

   • Each test should work alone
   • No shared state between tests
   • Use setUp for common setup

4. Clear Test Names

   # Good
   def test_userprofile_get_with_invalid_username_raises_error(self):
       # Test description
   
   # Bad
   def test_invalid(self):
       # Unclear what this tests
   

Extending the Wrapper

Adding Configuration

# Add to pyproject.toml
[tool.poetry.dependencies]
trustcafeapiwrapper = { path = "." }

[project.optional-dependencies]
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "httpx>=0.24.0",
]

test = [
    "pytest>=7.0.0",
]

Adding New Endpoints

Edit src/trustcafeapiwrapper/apiclient.py:

endpoints = {
    "alpha": {
        # ... existing endpoints ...
        "newendpoint": "https://api.example.com/alpha/",
    },
}

Adding Utility Functions

# src/trustcafeapiwrapper/utils/my_utilty.py

def my_utility_function(input_data: str) -> dict:
    """Utility function for common data transformations"""

    # Implementation

    return transformed_data

---

Getting Started with Development

1. Fork the repository

   • Add your fork as a remote
   • Keep pull requests to main

2. Set up development environment

   # Clone your fork
   git clone https://gitlab.com/your-username/trustcafe-api-wrapper.git
   cd trustcafe-api-wrapper
   
   # Install dependencies
   pip install -e ".[dev]"
   
   # Run tests
   pytest
   

3. Create feature branch

   git checkout -b feature/your-feature-name
   

4. Make changes

   • Add code
   • Add tests
   • Update documentation

5. Submit PR

   • Branch must be up-to-date with main
   • Include helpful description
   • Reference related issues

---

Resources

• Stavanger AI: See the development.md debate notes
• GitLab Repository: https://gitlab.com/trustcafe/trustcafe-api-wrapper
• WikiTribune Team: Contact via company channels
• Packaging: pyproject.toml for dependencies and metadata

---

Stay Updated: Read development.md for ongoing decisions and discussions. This file is intentionally loose to capture evolving thoughts and debates as the project progresses.