Skip to main content

​
Developer Documentation - Vault API Platform

​
Getting Started

​
Local Development Setup

  1. Environment Setup 
    git clone <repository-url>
cd vault
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
  2. Database Configuration 
    # Create local settings (copy from settings template)
cp api/settings_local.py.template api/settings_local.py

# Run migrations
./dj.sh mig

# Create superuser
./dj.sh sudo
  3. Environment Variables Create .env file in project root: 
    DEBUG=True
SECRET_KEY=dev-key-change-in-production
NOTION_TOKEN=your-notion-integration-token
API_BASE_URL=http://localhost:8000
DATABASE_URL=sqlite:///db.sqlite3
  4. Start Development Server 
    ./dj.sh run
# Server available at http://localhost:8000

​
Development Workflow

​
Project Structure

vault/
├── api/                    # Django project root
│   ├── settings.py         # Main settings
│   ├── urls.py            # Root URL configuration
│   └── manage.py          # Django management
├── core/                  # Base services
│   ├── views.py           # Server stats, health checks
│   ├── app_registry.py    # Multi-app configuration
│   └── templates/         # Server stats templates
├── trace/                 # Feature Specification Management
│   ├── views.py           # FSM API endpoints
│   ├── models.py          # FSM data models
│   ├── services.py        # Notion integration
│   └── templates/         # Admin dashboards
├── club/                  # Community Analytics
│   ├── views.py           # Analytics API endpoints
│   ├── services.py        # Data processing
│   ├── models.py          # Export tracking
│   └── templates/         # Analytics dashboards
├── static/data/           # JSON data storage
├── requirements.txt       # Python dependencies
├── dj.sh                 # Django command shortcuts
└── README.md             # Main documentation

​
Services Architecture

​
1. Core Service

Purpose: System monitoring and health checks Key Components:
  • BaseServerStatsView - Comprehensive system metrics
  • HealthCheckView - Simple health endpoint
  • ApplicationContextMiddleware - Multi-app context injection
Development Notes:
  • Uses psutil for system metrics
  • Custom JSON encoder handles Path objects
  • Auto-refresh dashboards every 30 seconds

​
2. Trace Service (FSM)

Purpose: Feature Specification Management via Notion Key Components:
  • ApplicationTraceabilityService - Notion API integration
  • BaseService - Standardized export functionality
  • Multi-app filtering via registry pattern
Data Flow:

​
3. Club Service

Purpose: Community analytics and data processing Key Components:
  • ClubTraceabilityService - External API integration
  • Async data export with concurrent processing
  • Feature-specific statistics and analytics
Data Sources:
  • External APIs (hero metrics, feature stats)
  • Static JSON file processing
  • Database export tracking

​
Multi-Application System

​
Registry Pattern

# Application configuration
applications = {
    'skyflow': {
        'name': 'Skyflow',
        'features': ['flow', 'clout', 'kudos'],
        'filters': {
            'tier': lambda data: custom_skyflow_filter(data),
            'feature': lambda data: skyflow_feature_filter(data)
        }
    }
}

​
Request Flow

​
API Development

​
Creating New Endpoints

  1. Add to views.py: 
    class NewEndpointView(TracerViewBase):  # or BaseClubAPIView
    @method_decorator(cache_page(300))
    def get(self, request, app_name=None):
        # Validate app context automatically handled by base class
        data = get_your_data()
        data = self.apply_app_filters(data, 'your_data_type')
        return self.build_response(data)
  2. Add URL pattern: 
    # Multi-app route
path('public/<str:app_name>/new-endpoint/', views.NewEndpointView.as_view()),
# Default route
path('public/new-endpoint/', views.NewEndpointView.as_view()),
  3. Update registry filters (if needed): 
    'your_data_type_filter': lambda data: [item for item in data if custom_logic(item)]

​
Response Standards

Multi-App Response: 
{
    "data": [...],
    "meta": {
        "application": "skyflow",
        "endpoint": "/trace/public/skyflow/tiers/",
        "total_items": 5,
        "custom_field": "value"
    }
}
Error Response: 
{
    "error": "User-friendly message",
    "detail": "Technical details for debugging",
    "available_apps": ["skyflow", "nexus"]  # For app validation errors
}

​
Testing

​
Running Tests

# All tests
./dj.sh test

# Specific app
./dj.sh test trace
./dj.sh test club

# Specific test class
./dj.sh test trace.tests.PublicTiersViewTestCase

# With coverage
coverage run --source='.' api/manage.py test
coverage report

​
Test Structure

class YourViewTestCase(TestCase):
    def setUp(self):
        self.client = Client()
        # Setup test data

    def test_multi_app_endpoint(self):
        response = self.client.get('/trace/public/skyflow/tiers/')
        self.assertEqual(response.status_code, 200)
        data = response.json()
        self.assertEqual(data['meta']['application'], 'skyflow')

    def test_invalid_app_handling(self):
        response = self.client.get('/trace/public/invalid/tiers/')
        self.assertEqual(response.status_code, 404)
        data = response.json()
        self.assertIn('Unknown application', data['error'])

​
Manual Testing

# Test multi-app endpoints
python api/manage.py test_multi_app

# List registered applications
python api/manage.py list_apps

# Check specific endpoints
curl http://localhost:8000/health/
curl http://localhost:8000/trace/public/skyflow/tiers/
curl http://localhost:8000/club/public/nexus/features/

​
Data Management

​
Static Data Files

Location: static/data/ Trace Service Files:
  • features.json - Feature specifications
  • gateways.json - Gateway configurations
  • tiers.json - Tier definitions
  • tags.json - Tag metadata
  • fsm_index.json - Service index with history
Club Service Files:
  • hero-metrics.json - Key community metrics
  • {feature}-most-present.json - Feature statistics
  • club_index.json - Service index with history

​
Data Export Services

Trace Service: 
# Export from Notion
python api/manage.py refresh_trace_data

# Manual export
from trace.services import ApplicationTraceabilityService
service = ApplicationTraceabilityService(notion_token='...')
service.run_full_export()
Club Service: 
# Export from external APIs
python api/manage.py refresh_club

# Manual export
from club.services import ClubTraceabilityService
service = ClubTraceabilityService()
await service.run_full_export_async()

​
Admin Dashboards

​
Creating Admin Views

  1. View Function: 
    @staff_member_required
def admin_your_feature(request):
    try:
        data = get_your_admin_data()
        context = {'data': data}
        return render(request, 'your_app/admin/feature.html', context)
    except Exception as e:
        context = {'error': str(e)}
        return render(request, 'your_app/admin/feature.html', context)
  2. Template: Follow existing patterns in templates/ directories
  3. URL Configuration: 
    path('admin/your-feature/', views.admin_your_feature, name='admin_your_feature'),

​
Dashboard Features

  • Auto-refresh: Templates include JavaScript for periodic refresh
  • Error handling: Graceful error display with retry options
  • Responsive design: Works on desktop and mobile
  • Interactive elements: Search, filtering, data manipulation

​
Caching Strategy

​
Cache Keys

  • club_data_{filename} - Club service data files
  • club_available_features - Feature list cache
  • trace_matrix_data - Trace service data cache
  • App-specific: club_data_{app_name}_{filename}

​
Cache Management

# Manual cache clearing
from django.core.cache import cache
cache.clear()

# Service-specific clearing
cache.delete_many(['club_data_hero-metrics.json', 'club_available_features'])

# Pattern-based (if supported)
cache.delete_pattern('club_*')

​
Monitoring & Debugging

​
Health Monitoring

  1. Basic Health: /health/
    • Database connectivity
    • Cache functionality
    • Basic service status
  2. Detailed Health: /club/admin/service-health/
    • File system status
    • Cache performance
    • Database statistics
  3. System Stats: /stats/
    • CPU, memory, disk usage
    • Django application info
    • Process information

​
Debugging Tools

# Django shell
./dj.sh shell

# Check for issues
./dj.sh check

# View database content
./dj.sh shell -c "from trace.models import *; print(APILog.objects.count())"

# Test specific views
from django.test import Client
client = Client()
response = client.get('/trace/public/skyflow/tiers/')
print(response.status_code, response.content)

​
Logging

import logging
logger = logging.getLogger(__name__)

# In views
logger.info(f"Processing request for app: {app_name}")
logger.error(f"Failed to process data: {str(e)}")

# View logs
tail -f api/logs/django.log

​
Deployment

​
Production Settings

# settings_production.py
DEBUG = False
ALLOWED_HOSTS = ['your-domain.com']

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USER'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST'),
        'PORT': os.environ.get('DB_PORT'),
    }
}

# Cache configuration
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': os.environ.get('REDIS_URL'),
    }
}

​
Deployment Checklist

  • Environment variables configured
  • Database migrations run
  • Static files collected
  • Cache backend configured
  • Monitoring alerts set up
  • Backup strategy implemented
  • SSL certificates installed
  • Rate limiting configured

​
Docker Setup

FROM python:3.11
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["gunicorn", "api.wsgi:application", "--bind", "0.0.0.0:8000"]

​
Contributing

​
Code Style

  • Follow Django best practices
  • Use type hints where helpful
  • Write docstrings for complex functions
  • Keep views thin, services thick

​
Git Workflow

  1. Create feature branch
  2. Make changes
  3. Write/update tests
  4. Update documentation
  5. Submit pull request

​
Code Review

  • Ensure backward compatibility
  • Verify multi-app functionality
  • Check admin dashboard updates
  • Test with different applications
Last Updated: January 2025
Django Version: 4.2+
Python Version: 3.9+