dsh-testgen > Automated unit-test generation for DeepSeek Harness: a /testgen command and a generate_tests tool that scaffold tests, run your project's test runner, and fix failures until they pass — bounded, observable, and honest about what it changed. English | 中文 --- ## Why this plugin DeepSeek Harness agents write code constantly — and leave it untested. dsh-testgen closes that loop inside the harness: - One command for humans — /testgen src/utils/math.ts generates, runs, and fixes in a single step. - One tool for agents — the agent itself can call generate_tests after writing code, keeping the loop in the session. - Two generators, no dead ends — behavioral tests through your composed LLM when available; deterministic structural smoke tests (zero LLM, works offline) otherwise. - A real fix loop — failures are parsed from vitest / jest / mocha / node:test output and fed back to the LLM, up to a bounded number of iterations, only ever rewriting the test files it generated. No duplicated wheels: the ecosystem has git tools and eval harnesses, but no plugin that generates, runs, and fixes unit tests as a first-class harness surface. ## Features | | | |---|---| | ⚡ /testgen slash command | [options] <file-or-glob>, --json, --help; rendered directly in the Web UI | | 🛠️ generate_tests model tool | structured JSON input/output, cooperative cancellation, never runs in parallel | | 🧠 LLM generator | streams through ctx.llm, honors your provider/model, bounded source truncation | | 🧩 Template generator | zero-dependency smoke tests scaffolded from exported symbols — no API key needed | | 🔁 Generate → run → fix loop | per-framework failure parsing, bounded by maxIterations | | 🧪 Runner autodetect | vitest / jest / mocha detected from the project; node --test as the universal fallback | | 🔥 Hot-reloaded config | testgen: section in settings.yaml; edits apply on the next invocation, no restart | | 🧰 Lifecycle-clean | registrations and in-flight runs unwind with the plugin fiber; refuses to overwrite user tests | | 📐 Typed end to end | TypeScript, schemastery config, stable error codes, 78 unit tests including a real end-to-end run | ## Demo Terminal-style demo (deterministic template generator + real node --test run): In the Web UI, the plugin shows up in Settings → Plugins → Plugin list with an active status (real dsh web boot, 134 plugins loaded): ## Installation Requires Node ≥ 22 and dsh (DeepSeek Harness CLI). sh # Install straight from GitHub dsh plugin --profile web add github:bujue600-arch/dsh-testgen dsh plugin add installs the package into the profile and reconciles the bundle layer stack — the plugin declares a dsh.bundle manifest ("dsh": { "bundle": { "patch": "./cordis.patch.yml" } }), so no hand-editing is required. It works for any profile: web, headless, or your own. (An npm package will be published with the first stable dsh release it targets; pinning to a release tag is recommended once available.) > Note for git/file installs: pnpm exits non-zero on missing peers for linked dependencies; that is expected — the harness resolves the peer packages (@deepseek-ai/dsh-*, cordis) from the profile's own module fallback at boot. The plugin loads normally. Verify it loaded (Web): Settings → Plugins → Plugin list shows the testgen entry with an active status dot. ## Usage ### Slash command /testgen [options] <file-or-glob> [more targets…] Options: --runner <vitest|jest|node-test|mocha|auto> framework (default: auto) --generator <llm|template|auto> generator (default: auto) --iterations <n> fix-loop bound (default: 3) --model <provider/model> generation model override --no-run generate only, do not run --json machine-readable report -h, --help show this help sh /testgen src/utils/math.ts /testgen --runner vitest "src/**/*.ts" /testgen --generator template --no-run src/app.ts /testgen --model deepseek-official/deepseek-chat src/parser.ts ### Model tool The agent can call it directly — same pipeline, structured result: generate_tests({ target: "src/utils/math.ts", runner: "vitest", maxIterations: 3 }) ### What gets written For src/utils/math.ts, tests land in src/utils/__tests__/math.test.ts (framework-dependent; node:test uses .test.mts). Existing test files are never overwritten — a target that already has a test file is skipped with a warning. ## Configuration All settings live under the testgen: key of your harness settings document ($DSH_HOME/settings.yaml) and are hot-reloaded — edit, save, done. Layering: schema defaults → composition entry (cordis.patch.yml) → your settings section. yaml testgen: runner: auto # auto | vitest | jest | node-test | mocha generator: auto # auto | llm | template maxIterations: 3 # generate → run → fix loop bound (0 disables fixing) timeoutSec: 120 # per-run wall-clock timeout autoRun: true # run the suite after generation includeGlobs: - '**/*.{ts,tsx,js,jsx}' excludeGlobs: - '**/node_modules/**' - '**/dist/**' - '**/*.test.*' - '**/*.spec.*' - '**/__tests__/**' testDir: __tests__ # generated tests land here, next to the target model: # optional provider/model override provider: deepseek-official model: deepseek-chat maxSourceChars: 60000 # source characters fed to the LLM per target You can also override any of these per profile by patching the testgen row in the profile's cordis.patch.yml (a patch replaces the row's whole config). See docs/config.md for the full reference. ## Input / output specification Every surface — command, tool, engine — speaks the same contract. The tool's canonical output schema is the JSON projection of TestgenReport; docs/io-spec.md documents it in full. ts interface TestgenReport { status: 'passed' | 'fixed' | 'generated' | 'failed' | 'skipped' targets: { path: string; language: string }[] generated: GeneratedTest[] // path, framework, generator, testCount runs: TestRun[] // per-iteration exit code, summary, failures warnings: string[] stats: { generatedFiles: number; passed: number; failed: number; iterations: number } elapsedMs: number } ## Architecture Everything is a plugin, and this plugin stays inside its box: dsh-testgen ├── src/ │ ├── index.ts # cordis entry: name / inject / Config / apply │ ├── schema.ts # schemastery config + settings namespace schema │ ├── settings.ts # hot-reload aware effective-config resolution │ ├── command.ts # /testgen grammar + handler │ ├── tool.ts # generate_tests definition (input/output schemas) │ ├── report.ts # plain / markdown / JSON rendering │ ├── errors.ts # stable TESTGEN_* error codes (HarnessError) │ └── engine/ # pure, framework-free core (fully unit-tested) │ ├── resolve.ts # path/glob → SourceTarget[], language detection │ ├── template.ts # deterministic smoke-test generator │ ├── generate-llm.ts # ctx.llm streaming, prompt assembly, extraction │ ├── runner.ts # framework detect, spawn, output parsing │ └── pipeline.ts # generate → run → fix orchestration ├── cordis.patch.yml # bundle patch: one `testgen` insert row ├── docs/ # io-spec, config reference ├── examples/fixture/ # demo project └── test/ # 78 unit tests (vitest) - Non-invasive: one insert row in the composition; no core patches, no monkey-patching. Removable with dsh plugin --profile web remove dsh-testgen. - Composable: hard-injects only commands and tools (present in every shipped profile); settings, llm, and the agent session are consumed opportunistically, so a profile without a settings provider or LLM adapter still works. - Honest side effects: writes test files only, refuses to overwrite existing ones, and fixes only files it generated. Test processes run with your project's runner, from the workspace root, with a tree-killing timeout and abort wiring. ## Development sh pnpm install pnpm run typecheck # tsc --noEmit pnpm run lint # eslint pnpm test # vitest (78 tests) pnpm run build # tsdown → lib/ pnpm run verify:manifest # dsh.bundle / exports / files contract pnpm run demo # end-to-end demo against examples/fixture Contributions welcome — see CONTRIBUTING.md and the Code of Conduct. Report bugs or propose features through the issue templates. ## FAQ Does it need an API key? No. Without an LLM adapter, generator: auto uses the deterministic template generator — structural smoke tests that prove imports resolve and exports exist. With an LLM composed, auto prefers behavioral tests. Which projects work with node --test? TypeScript targets need an ESM project ("type": "module" in package.json — the normal case) because Node strips types only from ESM files; JSX targets need vitest or jest (pin runner). Will it modify my source code? Never. It writes test files only, and skips targets that already have a test file. What if the fix loop can't make it green? It stops at maxIterations and reports failed with the last failures and warnings — no infinite retries. ## Changelog & versioning CHANGELOG.md — versions follow SemVer. ## License MIT © bujue600-arch and dsh-testgen contributors. This is a community project, not affiliated with DeepSeek.