Skip to main content

Notion Business Intelligence CLI

Command-line tools for managing and monitoring your Notion Business Intelligence Platform.

Overview

The CLI provides direct access to data refresh operations, system health monitoring, and business intelligence export capabilities. These tools are designed for operational management, automated workflows, and administrative tasks.

Installation

The CLI is included with your Django application. Ensure your environment is properly configured:
# Verify Django management commands are available
python manage.py --help

# Or use the convenience script
./dj.sh

Configuration

Set required environment variables:
export NOTION_TOKEN="your_notion_integration_token"
export NOTION_FEATURES_DB_ID="your_features_database_id"
export NOTION_IO_DB_ID="your_gateways_database_id"
export NOTION_TIER_DB_ID="your_tiers_database_id"
export NOTION_TAG_DB_ID="your_tags_database_id"
export NOTION_KEYWORDS_DB_ID="your_keywords_database_id"

Command Reference

Data Refresh Commands

Comprehensive Refresh

# Refresh all data and generate analytics
./dj.sh rf_all

# Full refresh with options
./dj.sh rf all --force --clear-cache --preload-lookups --generate-analytics

# Dry run to see what would be refreshed
./dj.sh rf all --dry-run

Targeted Data Refresh

# Refresh only core data (no analytics)
./dj.sh rf_data

# Refresh specific data types
./dj.sh rf features
./dj.sh rf keywords
./dj.sh rf gateways
./dj.sh rf tiers
./dj.sh rf tags

# Force refresh even if data appears recent
./dj.sh rf features --force

Analytics-Only Refresh

# Regenerate analytics without fetching new data
./dj.sh rf_analytics

# Refresh specific analytics
./dj.sh rf seo
./dj.sh rf content
./dj.sh rf business

Cache Management

# Refresh lookup tables and clear caches
./dj.sh rf_cache

# Clear cache for specific data type
./dj.sh rf features --clear-cache

System Monitoring

Health Checks

# Basic system health check
./dj.sh health

# Detailed output includes:
# - Database connectivity status
# - Lookup cache status
# - API connectivity test
# - Service availability

Performance Monitoring

# Check via Django shell for detailed metrics
./dj.sh shell -c "
from vault.api.trace.notion.repository import new_notion_repository
import asyncio
repo = new_notion_repository(notion_token='your_token', ...)
print(asyncio.run(repo.get_lookup_stats()))
"

Data Export

Business Intelligence Reports

# Export business insights to JSON
./dj.sh export_business

# Export to specific file
./dj.sh export_business --output /path/to/report.json

# Export SEO analytics
./dj.sh export_seo --output seo_report.json

# Export feature analytics
./dj.sh export_features --output features_report.json

Raw Data Export

# Export database contents
./dj.sh dump

# Export to specific location
python manage.py dumpdata > /backup/database_backup.json

Usage Patterns

Daily Operations

Morning Data Sync
# Quick refresh for updated data
./dj.sh rf_data --preload-lookups
Weekly Full Refresh
# Complete refresh with analytics
./dj.sh rf_all --force --clear-cache
Health Monitoring
# Check system status
./dj.sh health

Development Workflow

Testing Changes
# Dry run to verify refresh logic
./dj.sh rf all --dry-run

# Refresh specific component during development
./dj.sh rf features --force
Performance Testing
# Preload lookups for performance testing
./dj.sh rf cache
./dj.sh rf_analytics

Production Deployment

Post-Deployment Verification
# Verify system health after deployment
./dj.sh health

# Refresh data to ensure connectivity
./dj.sh rf_data
Scheduled Maintenance
# Full system refresh (suitable for cron)
./dj.sh rf_all --clear-cache --generate-analytics

Command Options

Global Options

  • --force: Force refresh even if data appears recent
  • --dry-run: Show what would be done without executing
  • --clear-cache: Clear relevant caches after operation
  • --preload-lookups: Load lookup tables for better performance
  • --generate-analytics: Generate fresh analytics after data refresh

Output Formats

Most commands provide structured output:
  • Success operations show green checkmarks
  • Warnings appear in yellow
  • Errors display in red
  • Progress indicators for long operations

Troubleshooting

Common Issues

Authentication Errors
# Verify token configuration
echo $NOTION_TOKEN

# Test connectivity
./dj.sh health
Cache Issues
# Clear all caches
./dj.sh rf_cache

# Force refresh specific data
./dj.sh rf features --force --clear-cache
Performance Issues
# Preload lookups for better performance
./dj.sh rf cache

# Check lookup table status
./dj.sh health

Error Diagnostics

Database Connectivity
# Check database configuration in health output
./dj.sh health

# Verify database IDs are configured
./dj.sh shell -c "from django.conf import settings; print(settings.NOTION_FEATURES_DB_ID)"
Service Issues
# Test individual services
./dj.sh rf features --dry-run
./dj.sh rf seo --dry-run

Automation

Cron Jobs

Daily Data Refresh
# Add to crontab for daily 6 AM refresh
0 6 * * * /path/to/dj.sh rf_data --preload-lookups

# Weekly full refresh (Sundays at 2 AM)
0 2 * * 0 /path/to/dj.sh rf_all --force --clear-cache
Health Monitoring
# Hourly health checks
0 * * * * /path/to/dj.sh health || echo "Health check failed" | mail admin@company.com

CI/CD Integration

Deployment Pipeline
# Post-deployment data refresh
- name: Refresh Notion data
  run: ./dj.sh rf_data --preload-lookups

# Health verification
- name: Verify system health
  run: ./dj.sh health

Performance Considerations

Optimization Strategies

Lookup Preloading
# Preload frequently accessed data
./dj.sh rf cache
Targeted Refreshes
# Refresh only changed data types
./dj.sh rf features  # Instead of rf_all
Cache Management
# Strategic cache clearing
./dj.sh rf seo --clear-cache  # Only clear SEO caches

Resource Usage

  • Memory: Lookup preloading increases memory usage but improves performance
  • Network: Full refreshes make multiple API calls to Notion
  • Time: Complete refresh typically takes 2-5 minutes depending on data volume

Security

Best Practices

  • Store API tokens in environment variables, not code
  • Use restricted Notion integration tokens when possible
  • Monitor API usage through health checks
  • Regularly rotate API tokens

Access Control

# Verify token permissions through health check
./dj.sh health

# Test with minimal permissions first
./dj.sh rf features --dry-run

Integration Examples

Shell Scripts

#!/bin/bash
# daily_refresh.sh

set -e

echo "Starting daily Notion refresh..."

# Health check first
if ! ./dj.sh health; then
    echo "Health check failed, aborting refresh"
    exit 1
fi

# Refresh data
./dj.sh rf_data --preload-lookups

# Generate analytics
./dj.sh rf_analytics

echo "Daily refresh completed successfully"

Python Scripts

import subprocess
import sys

def refresh_notion_data():
    """Refresh Notion data with error handling"""
    try:
        # Health check
        result = subprocess.run(['./dj.sh', 'health'],
                              capture_output=True, text=True, check=True)

        # Data refresh
        result = subprocess.run(['./dj.sh', 'rf_data', '--preload-lookups'],
                              capture_output=True, text=True, check=True)

        print("Notion data refreshed successfully")
        return True

    except subprocess.CalledProcessError as e:
        print(f"Refresh failed: {e.stderr}")
        return False

if __name__ == "__main__":
    success = refresh_notion_data()
    sys.exit(0 if success else 1)

Development

Adding Custom Commands

Create new management commands in vault/api/trace/management/commands/:
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    help = 'Custom Notion operation'

    def handle(self, *args, **options):
        # Your custom logic here
        pass

Extending the CLI

Add new shortcuts to dj.sh:
# Add to dj.sh case statement
my_custom_command)
    $MANAGE my_custom_command "${@:2}"
    ;;

Support

For issues with CLI commands:
  1. Check command syntax with ./dj.sh (shows usage)
  2. Run health check: ./dj.sh health
  3. Try dry run mode: ./dj.sh rf all --dry-run
  4. Check Django logs for detailed error messages
  5. Verify environment configuration and API tokens

Version Compatibility

  • Django 3.2+
  • Python 3.8+
  • Notion API version 2022-06-28
  • Redis (recommended for caching)
The CLI tools are designed to be backward compatible and will gracefully handle missing configuration or API changes.