What We Learned
When adding three AI services to Traceo’s MCP server (semantic search, LLM quality analysis, and RAG document Q&A), the single design decision that paid off most was constructor injection on the OpenAI client. Every service accepts an optionalclient parameter — pass None and it creates a real OpenAI client; pass a mock and the service never touches the network.
The mock client (MockOpenAIClient) uses a deterministic hash to generate embeddings: the same text always produces the same 1536-dimension vector. For chat completions, it returns pre-canned JSON matching the expected analysis schema. This means tests verify real business logic — chunking boundaries, similarity ranking, structured output parsing, tenant isolation — without a single API call.
Why It Matters
AI features are the most expensive thing to test in CI. A real OpenAI embedding call costs tokens and takes 200-500ms. Multiply by 32 tests across every PR and you get slow builds, flaky failures on rate limits, and a monthly bill that grows with commit frequency. Constructor injection collapses all of that to zero. The alternative — mocking at the HTTP layer withresponses or httpx_mock — is brittle. It couples tests to the SDK’s internal request format, which changes across versions. Constructor injection mocks at the service boundary: the contract is “give me an object with .embeddings.create() and .chat.completions.create().” That’s stable.
The Pattern
Every AI service class takesclient=None in __init__. When None, it imports and instantiates the real SDK client. When provided, it stores whatever was passed. Tests inject a mock that satisfies the same async interface. The mock lives in conftest.py as a shared fixture, so every test file gets it for free.
This same pattern extends to any external API dependency: payment processors, notification services, third-party data providers. The service owns the client lifecycle; tests own the mock.