Verification Traceability Matrix (Current Baseline)
A. Core Config / Nests
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| CFG-1 | Default config is valid, workspace_root set, sync/assembly fields parse | Unit load+validate | pkg.LoadConfig used in multiple tests (e.g. TestNestOlly_DefaultConfig_ServicesGraph) | Covered (indirect) |
| CFG-2 | GetDefaultConfigPath prefers nests layout (nests/olly.yaml) | Code path execution (manual + tests) | internal/pkg/config.go::GetDefaultConfigPath, used in CLI / engine, sanity checked via nest tests path setup | Partially covered (behavior validated via usage, not directly asserted anymore) |
| NEST-1 | nests/olly.yaml services and deps are correct (uuid-lib → none, is-lib → uuid-lib) | Config-level integration test | engine/tests/nest_olly_integration_test.go::TestNestOlly_DefaultConfig_ServicesGraph | Covered |
| NEST-2 | nests/olly.yaml yields correct project graph (topological order: uuid-lib, is-lib) | Config-level integration test | Same as NEST-1 (graph check) | Covered |
| NEST-3 | Engine can consume nests/olly.yaml via core.Engine.LoadProject and build correct project | Engine-level integration test | engine/tests/engine_nest_olly_integration_test.go::TestEngine_NestOlly_LoadProject | Covered |
B. Project Model & Orchestration
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| PROJ-1 | Project.ScanRepositories normalizes relative paths with WorkspaceRoot and is deterministic | Unit tests | internal/project/project_test.go::TestScanRepositories_* | Covered |
| PROJ-2 | Project.RunAcrossServices is deterministic in unordered mode (sorted by name) | Unit + integration tests | internal/project/project_test.go::TestRunAcrossServices_UnorderedDeterministic, engine/tests/engine_project_integration_test.go::TestEngineAndProject_Integration_RunAcrossServices | Covered |
| PROJ-3 | Project.RunAcrossServices honors dependency ordering when ordered=true | Unit + integration tests | project_test.go::TestRunAcrossServices_OrderedByDependencies, engine_project_integration_test.go::TestEngineAndProject_Integration_OrderedByDependencies | Covered |
| PROJ-4 | Project.RunAcrossServices supports filters and error propagation | Unit tests | project_test.go::TestRunAcrossServices_FilterBehavior, ..._ErrorPropagation | Covered |
| PROJ-5 | Dependency graph TopologicalOrder yields stable order & detects cycles | Unit + usage in integrations | internal/project/model.go tests (topo tests if present) + all tests above that rely on order; assembler and nest tests indirectly exercise this | Mostly covered (strong via usage) |
C. Core Engine Surface
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| ENG-1 | core.Engine.RunAcrossServices validates inputs and delegates correctly | Unit tests | internal/core/engine_test.go::TestEngineRunAcrossServices_* | Covered |
| ENG-2 | core.Engine.RunAcrossServices preserves deterministic behavior from project layer | Integration tests | engine/tests/engine_project_integration_test.go::* | Covered |
| ENG-3 | core.Engine.LoadProject can construct a project from config and expose services graph | Engine-level integration | engine/tests/engine_nest_olly_integration_test.go::TestEngine_NestOlly_LoadProject | Covered |
| ENG-4 | core.Engine.Assemble and core.Engine.Sync are reachable from CLI and accept options | CLI smoke + code inspection | engine/cli_main_test.go::TestAssembleCommand_NoConfig, TestSyncCommand_NoConfig (wiring & error surface) | Wiring covered, end-to-end assemble/sync behavior not yet verified via engine-only tests |
D. Assembler (Option B)
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| ASM-1 | Assembler.calculateBuildOrder respects service dependency DAG | Unit test | internal/assembler/assembler_test.go::TestCalculateBuildOrder_RespectsDependencies | Covered |
| ASM-2 | Assembler.calculateBuildOrder detects cycles | Unit test | assembler_test.go::TestCalculateBuildOrder_DetectsCycle | Covered |
| ASM-3 | Full AssembleWorkspace behavior over repos and builds | Not yet covered | No direct integration test yet | Not covered (by design for now) |
E. Sync Engine (Option B)
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| SYNC-1 | Pattern matching picks included files and excludes correctly | Unit test | internal/sync/engine_test.go::TestMatchesSyncAndExcludePatterns | Covered |
| SYNC-2 | discoverSyncCandidates walks a tree, applies patterns/excludes deterministically | Unit test with temp dir | engine_test.go::TestDiscoverSyncCandidates_SmallTree | Covered |
| SYNC-3 | SynchronizeWorkspace dry-run mode computes ops and reports success without side effects | Integration test with temp dir | engine_test.go::TestSynchronizeWorkspace_DryRun_SmallTree | Covered |
| SYNC-4 | Non-dry-run sync actually mutates filesystem | Not yet covered | No current test | Not covered (intentionally kept out of fast deterministic suite) |
F. CLI / UX (Option C)
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| CLI-1 | olly assemble routes via core.Engine.Assemble and propagates config path / log level | Code inspection + smoke test (error path) | engine/main.go::newAssembleCommand, cli_main_test.go::TestAssembleCommand_NoConfig | Wiring covered, happy-path not covered |
| CLI-2 | olly sync routes via core.Engine.Sync and preserves summary output | Code inspection + smoke test (error path) | engine/main.go::newSyncCommand, cli_main_test.go::TestSyncCommand_NoConfig | Wiring + output path covered for error case |
| CLI-3 | project subcommands (info, services, graph) use core.Engine.LoadProject | Code inspection + existing behavior | engine/main.go::newProject*Command | Covered by design; no dedicated tests yet |
| CLI-4 | run services uses core.Engine.RunAcrossServices | Code inspection | engine/main.go::newRunServicesCommand | Wiring covered, not yet under test |
G. Observability / Logging
| Req ID | Requirement / Behavior | Verification Method | Test / Artifact | Status |
|---|---|---|---|---|
| OBS-1 | pkg.Logger can be constructed and used in sync engine paths | Integration tests | internal/sync/engine_test.go tests construct real loggers; dry-run logs are emitted | Covered (for sync paths) |
| OBS-2 | Higher-level observability (metrics/workflows) integration | Separate existing tests | engine/tests/observability_integration_test.go::* | Covered (existing baseline) |
Summary
- Truth testing: nests/olly.yaml is now covered both at config level and engine level.
- Core orchestration: Project + engine orchestration are well-covered by unit + integration tests.
- Assembler/Sync: Key deterministic behaviors are covered; destructive/full operations are intentionally left for future, slower or opt-in tests.
- CLI: Wiring through core.Engine is in place and smoke-tested for error paths; full happy-path E2Es are a future enhancement, not required for this baseline.