Test Pipeline

This document describes the DBTools testing infrastructure, including test categories, execution methods, and shared test utilities.

Overview

DBTools uses a dual-tier testing strategy:

Headless Tests - Run via dotnet test without Revit, validating build artifacts and cloud integrations
Revit Integration Tests - Run inside a live Revit process using the ricaun.RevitTest adapter

Source: invoke-revit-tests.sh:1-6

Test Categories

Category System

Tests are categorized using NUnit's [Category] attribute to enable selective execution:

Category	Description	DA-Compatible
`RequiresRevitUI`	Needs UIApplication, UIDocument, TaskDialog, or STA thread	No
`RequiresActiveDocument`	Needs an open Revit document with elements	No
`Local`	Runs only in local Revit environment	No
`Slow`	Takes >5 seconds, may be excluded from quick runs	Yes
`Integration`	Requires external dependencies (network, cloud)	Yes

Source: testing/TestSupport/TestCategories.cs:14-42

DA vs Local Execution

DA (Design Automation): Filter out UI-dependent tests:

dotnet test --filter "Category!=RequiresRevitUI&Category!=RequiresActiveDocument"

Local: Run all tests inside Revit

Source: testing/TestSupport/TestCategories.cs:10-12

Test Project Organization

Directory Structure

testing/
  DBTools.BuildArtifacts.Tests/   # Headless artifact validation
  DBTools.DA.Tests/               # Design Automation tests (cloud/headless)
  TestSupport/                    # Shared test infrastructure
    GM/                           # GM-specific test doubles
    SGT/                          # SGT-specific test doubles
    TestHost/                     # Revit host integration
  RevitTestModels/                # Test .rvt files by year
    2024/GM/, 2024/SGT/
    2025/GM/, 2025/SGT/
    2026/GM/, 2026/SGT/

src/Tools/
  Common/GM/Tests/                # GM integration tests
  Structural/SGT/Tests/           # SGT integration tests
  Testing/VTC/Tests/              # VTC tests
  Common/TDV/Tests/               # TDV tests

Headless Test Projects

DBTools.BuildArtifacts.Tests

Validates build output without Revit:

Assembly existence and validity per Revit year
Target framework correctness (net48 for 2024, net8.0 for 2025+)
Forbidden reference detection (RevitAPI, AdWindows)
Namespace presence verification
Embedded XAML resource validation

Source: testing/DBTools.BuildArtifacts.Tests/AssemblyManifestTests.cs:14-20

Run command:

dotnet test testing/DBTools.BuildArtifacts.Tests/DBTools.BuildArtifacts.Tests.csproj -c Release

DBTools.DA.Tests

Design Automation and APS integration tests:

Links APS test files from tool directories (e.g., GM/Tests/APS/)
Uses shared test utilities via <Compile Include> links
Runs against Autodesk cloud services

Source: testing/DBTools.DA.Tests/DBTools.DA.Tests.csproj:43-48

Run command:

dotnet test testing/DBTools.DA.Tests/DBTools.DA.Tests.csproj -c Release

Configuration: Uses da.runsettings for APS bucket configuration:

<TestRunParameters>
  <Parameter name="APS_BUCKET" value="dbtools-tests" />
  <Parameter name="APS_REGION" value="US" />
</TestRunParameters>

Source: testing/da.runsettings:1-8

Tool-Specific Test Projects

Each tool has its own test project under src/Tools/*/Tests/:

Project	Target Frameworks	Revit Years
`DBTools.GM.Tests`	net48, net8.0-windows	2024, 2025, 2026
`DBTools.SGT.Tests`	net48, net8.0-windows	2024, 2025, 2026
`DBTools.TDV.Tests`	net48, net8.0-windows	2024, 2025, 2026
`DBTools.VTC.Tests`	net48, net8.0-windows	2024, 2025, 2026

Source: src/Tools/Common/GM/Tests/DBTools.GM.Tests.csproj:4, src/Tools/Structural/SGT/Tests/DBTools.SGT.Tests.csproj:4

Key dependencies:

ricaun.RevitTest.TestAdapter - Revit test execution
ricaun.Revit.UI.Tasks - UI task scheduling
NUnit 3 with test adapter

Source: src/Tools/Common/GM/Tests/DBTools.GM.Tests.csproj:35-44

Test Runner: invoke-revit-tests.sh

The primary test runner script for Revit integration tests.

Prerequisites

Required: gum (interactive terminal UI tool)

# Install via Homebrew (macOS/Linux)
brew install gum

# Or Debian/Ubuntu
sudo apt install gum

# Or Go
go install github.com/charmbracelet/gum@latest

Source: invoke-revit-tests.sh:8-17

Basic Usage

# Run all GM tests with smart mode (recommended)
bash invoke-revit-tests.sh --smart --tool GM

# Run SGT tests for Revit 2025
bash invoke-revit-tests.sh --smart --tool SGT -y 2025

# Run specific test by filter
bash invoke-revit-tests.sh -f "FullyQualifiedName~SomeTestName"

# Run specific fixture
bash invoke-revit-tests.sh --smart --fixture GmAdapterTests

Source: invoke-revit-tests.sh:121-160, invoke-revit-tests.sh:649-663

Command-Line Options

Test Execution

Flag	Description	Default
`-y, --year YEAR`	Revit year: 2024, 2025, or 2026	2026
`--debug, -d`	Use Debug configuration	Auto-detect
`--release, -r`	Use Release configuration	Auto-detect
`-f, --filter "..."`	VSTest filter (must include FullyQualifiedName)	None
`-v, --verbosity N`	0=quiet, 1=normal, 2=verbose	2
`--timeout-minutes N`	Revit test timeout	4

Source: invoke-revit-tests.sh:125-152

Revit Instance Modes

Flag	Behavior
`--close`	Open a fresh isolated Revit test instance and close it after tests (default)
`--reuse`	Rejected for isolated worktree runs
`--persist`	Rejected for isolated worktree runs

Source: invoke-revit-tests.sh:356 Source: invoke-revit-tests.sh:1132

Smart Mode (Recommended)

bash invoke-revit-tests.sh --smart --tool GM

Smart mode checks if the build has changed since the last successful test, warns if so, and always uses --close mode to prevent leaked Revit instances. For isolated launches, the runner now stages a stable worktree-local payload under .artifacts/revit-test-host/<year>/payload and temporarily rewrites the single DBTools.Loader.dll AppLoader entry to point at that stable path for the run. Because the path stays fixed across runs, Revit trust decisions are reusable and the repeated "Always Load" prompt no longer appears.

Source: invoke-revit-tests.sh:153 Source: invoke-revit-tests.sh:190 Source: invoke-revit-tests.sh:1056

Tool/Fixture Shortcuts

Flag	Description
`--tool TOOL`	Run tests for GM, SGT, TDV, or VTC
`--fixture FIXTURE`	Run tests for specific fixture class

Source: invoke-revit-tests.sh:129-131, invoke-revit-tests.sh:651-663

Running Single Tests

Recommended (via test runner):

# Exact match
bash invoke-revit-tests.sh --smart -y 2026 -f "FullyQualifiedName=DBTools.GM.Tests.GmAdapterTests.SomeTest"

# Partial match (contains)
bash invoke-revit-tests.sh --smart -f "FullyQualifiedName~SomeTestName"

Headless projects only:

dotnet test testing/DBTools.BuildArtifacts.Tests/DBTools.BuildArtifacts.Tests.csproj \
  -c Release --filter "FullyQualifiedName~Metadata"

Source: invoke-revit-tests.sh:691-697, AGENTS.md

Quiet Output and Run Bundles

Test execution is quiet by default in the terminal. The runner writes the full vstest console stream to a per-run bundle on disk, then prints only the compact summary, failed test names, and artifact paths to the terminal.

Each run creates .artifacts/revit-test-runs/<timestamp>-<year>-<tool-or-filter>/ with:

console.log - full raw vstest console output
results.trx - structured test results from the TRX logger
vstest.diag.log - test platform diagnostics
summary.json - counts, exit code, duration, failed tests, and artifact paths
failures.md - aggregated failure report for bulk triage
failed/*.md - one artifact per failed test with captured failure details

This layout is designed for bulk triage loops: run the full tool suite once, read failures.md, apply production-first fixes under the test-audit rule, rebuild once, and rerun the full suite.

Source: invoke-revit-tests.sh:97-117 Source: invoke-revit-tests.sh:960-969 Source: invoke-revit-tests.sh:1008-1015 Source: invoke-revit-tests.sh:1041-1283

Test Discovery

The runner includes built-in test discovery that scans source files on every invocation (no cached registry file):

# List all available tests
bash invoke-revit-tests.sh --discover

# Show test count summary
bash invoke-revit-tests.sh --discover --summary

# Filter by tool
bash invoke-revit-tests.sh --discover --tool GM

# Show only DA-compatible tests
bash invoke-revit-tests.sh --discover --da-only

# Show only local-only tests
bash invoke-revit-tests.sh --discover --local-only

Source: invoke-revit-tests.sh:397-467

Test History

The runner tracks test execution history and stores the latest bundle paths for each filter:

# Show test run history
bash invoke-revit-tests.sh --history

# Show tests not run in last N days
bash invoke-revit-tests.sh --show-stale 7

# Show tests that failed on last run
bash invoke-revit-tests.sh --show-failed

# Show error details for failed tests
bash invoke-revit-tests.sh --get-errors "GM"

Source: invoke-revit-tests.sh:148-152 Source: invoke-revit-tests.sh:472-560 Source: invoke-revit-tests.sh:563-634

History is stored at: .artifacts/test-history.json

--get-errors "PATTERN" replays the structured failures.md report from the latest matching failed run instead of showing only a short snippet.

Source: invoke-revit-tests.sh:58-60 Source: invoke-revit-tests.sh:532-551

Parallel Agent Safety

Multiple agents are prevented from running tests for the same Revit year simultaneously via per-year file locks (.artifacts/.revit-locks/revit-{YEAR}.lock). No explicit session management is needed — the locking is automatic.

Source: invoke-revit-tests.sh:1144

TestSupport Infrastructure

Shared test utilities in testing/TestSupport/:

Common Test Doubles

Located in CommonTestDoubles.cs:

Class	Purpose
`RecordingNotifier`	Records error/success banner invocations
`RecordingOverlay`	Records progress overlay method calls
`InlineExecutor`	Runs actions inline (no async dispatch)
`InlineTransactionRunner`	Executes without Revit transactions
`InlineTransactionGroupService`	Executes without transaction groups

Source: testing/TestSupport/CommonTestDoubles.cs:15-339

Test Host Integration

RevitHost (TestHost/RevitHost.cs):

Manages Revit context for async test execution
Handles inline vs queued execution modes
Integrates with RevitTestTaskBinder

Source: testing/TestSupport/TestHost/RevitHost.cs:12-109

RevitAmbientScope (TestHost/RevitAmbientScope.cs):

Provides ambient UIApplication access during tests

Test Path Resolution

TestPathResolver (TestPathResolver.cs):

Resolves test model paths from dbtools.testparams.json
Walks up from assembly location to find .artifacts/dbtools.testparams.json
Falls back to %APPDATA%/DBTools/dbtools.testparams.json (legacy)
invoke-revit-tests.sh writes the canonical file to .artifacts/ and mirrors it to %APPDATA%/DBTools/ so RevitTest temp-assembly runs can still resolve models

// Get model path
var path = TestPathResolver.ResolveTestModelPath("2026", "GM", "gm_test_model.rvt");

Source: testing/TestSupport/TestPathResolver.cs:10-66 Source: invoke-revit-tests.sh:867-956

Test Logging

TestLoggingBridge (TestLoggingBridge.cs):

Centralizes test logging setup
Bridges to production logging when Revit is running
Falls back to console mirroring in standalone mode

[OneTimeSetUp]
public void GlobalSetup()
{
    TestLoggingBridge.Initialize();
}

[OneTimeTearDown]
public void GlobalTeardown()
{
    TestLoggingBridge.Shutdown();
}

Source: testing/TestSupport/TestLoggingBridge.cs:14-143

Tool-Specific Test Doubles

GM (TestSupport/GM/):

GmTestDataBuilder - Builds test data for GM
GmRealisticTestDoubles - Production-like test doubles
GmShellViewModelTestFactory - Creates shell ViewModels for testing

SGT (TestSupport/SGT/):

SgtTestDataBuilder - Builds test data for SGT
SgtRealisticTestDoubles - Production-like test doubles
InMemorySgtUiStateStore - In-memory UI state store

Test Fixture Base Classes

GmTestFixture

Base class for GM integration tests:

[NonParallelizable]
public abstract class GmTestFixture
{
    protected UIApplication UiApp { get; private set; }
    protected Document? Doc { get; private set; }
    protected IRevitCallGate? Gate { get; private set; }
    protected ITransactionRunner? TxRunner { get; private set; }

    [OneTimeSetUp]
    public virtual void GlobalSetup(UIApplication app)
    {
        // Opens gm_test_model.rvt
        // Initializes call gate and transaction runner
    }
}

Features:

Opens tool-specific test model automatically
Provides ResolveTypeIds(), ResolveStyleIds(), ResolveMaterialIds() helpers
Manages document lifecycle

Source: src/Tools/Common/GM/Tests/GMTestFixture.cs:14-128

Test Models

Located in testing/RevitTestModels/:

RevitTestModels/
  model_info.txt
  2024/
    GM/gm_test_model.rvt
    SGT/sgt_test_model.rvt
    SGT/_LinkedModels/sgt_test_linked_model.rvt
  2025/
    GM/gm_test_model.rvt
    SGT/sgt_test_model.rvt
    SGT/_LinkedModels/sgt_test_linked_model.rvt
  2026/
    GM/gm_test_model.rvt
    SGT/sgt_test_model.rvt
    SGT/_LinkedModels/sgt_test_linked_model.rvt

Each year has its own model versions for forward compatibility testing.

Configuration Files

.runsettings

Revit Tests: Generated at runtime by invoke-revit-tests.sh:

<RunSettings>
  <NUnit>
    <Version>2026</Version>
    <Language>ENU</Language>
    <Open>true</Open>
    <Close>false</Close>
    <Verbosity>2</Verbosity>
    <Timeout>4</Timeout>
  </NUnit>
</RunSettings>

Source: invoke-revit-tests.sh:867-937

DA Tests: testing/da.runsettings for APS configuration.

Source: testing/da.runsettings:1-8

Test Parameters (dbtools.testparams.json)

Written to .artifacts/dbtools.testparams.json and mirrored to %APPDATA%/DBTools/dbtools.testparams.json so both repo-local and temp-hosted RevitTest executions can resolve the same model root:

{
  "ModelsDirectory": "C:\\...\\csharp\\testing\\RevitTestModels",
  "Year": 2026
}

Source: invoke-revit-tests.sh:939-956

Build Integration

Tests depend on artifacts from build.sh:

# Build all (required before running tests)
bash build.sh BuildAll

# Build tests specifically
bash build.sh BuildTests

The test runner validates artifact freshness:

Checks .artifacts/metadata/build.json for source fingerprint
Fails if artifacts are stale relative to source

Source: invoke-revit-tests.sh:733-841

Guardrails (from AGENTS.md)

Prime Directive

When a test fails, investigate and fix PRODUCTION CODE first. Do not change tests or infrastructure to simulate missing production behavior.

Forbidden Test Patterns

Use test doubles only for TRUE externals (Revit API, filesystem, network)
Never mock your own services/orchestrators
Never assert only that mocks were called
Every test must assert at least one real outcome

Legitimate vs Illegitimate Stubbing

Legitimate (external):

Revit API
File system
Network
Time/Random

Illegitimate (your code):

Your own services
Orchestrators/handlers
Anything in src/ you wrote

Quick Reference

Task	Command
Run GM tests (smart mode)	`bash invoke-revit-tests.sh --smart --tool GM`
Run SGT tests for 2025	`bash invoke-revit-tests.sh --smart --tool SGT -y 2025`
Run single test	`bash invoke-revit-tests.sh --smart -f "FullyQualifiedName~TestName"`
Run headless artifact tests	`dotnet test testing/DBTools.BuildArtifacts.Tests/ -c Release`
Run DA tests	`dotnet test testing/DBTools.DA.Tests/ -c Release`
List all tests	`bash invoke-revit-tests.sh --discover`
Show failed tests	`bash invoke-revit-tests.sh --show-failed`
Build before tests	`bash build.sh BuildAll`

Build Pipeline - Build system and artifact generation
Architecture Overview - High-level system architecture
Project References - Project structure and dependencies

Verification Status

Check	Status
Source anchors for all claims	Yes
UNVERIFIED markers where needed	No (all verified)
Cross-references added	Yes
Examples tested	N/A
No assumptions without evidence	Yes

Verified by: docs-run-20260124-020412
Date: 2026-01-24

Table of Contents