简体中文 # Model Catalog — Model Catalog Auto-Discovery (dsh plugin) This plugin is designed for the dsh ecosystem (a plugin-based DeepSeek harness where everything is a plugin): once you configure an OpenAI-compatible API host in dsh (official API, relay gateway, local inference service, etc.), the plugin automatically pulls model information from that host's endpoints — model listings, pricing (input/output/cache prices per million tokens), inference parameters (context length, max output, capability flags: tool calling / structured output / vision / parallel tools, etc.) — normalizes it, and generates ready-to-use model configs, saving you the manual work. Core value: - Host-type auto-detection, no protocol details to configure; - End-to-end unit normalization (per-token USD / per-million USD / multipliers / per-call); - Traceable pricing (captured_at + source endpoint), structured dynamic pricing; - Optional lightweight live probing for missing capabilities; - Three outputs: full catalog, dsh config fragment, human-readable report. --- ## Features - Host type auto-detection: standard compatible (minimal fields) / rich metadata (/models with pricing and parameter lists) / multiplier-priced gateway (quota system) / capability-flag proxy (/model/info capability booleans) / Ollama / vLLM; fails with an error when all endpoint probes fail, and supports manual --kind. - Normalized model records: id / context window / max output / pricing family (input, output, cache_read, cache_write, internal_reasoning, in USD/1M) / capability flags / source provenance. - Pricing source chain: host endpoint → user override config → external price mirror → built-in default table → marked unknown (with warning). - Built-in default table: model facts and pricing fallback for the official API (DeepSeek); dynamic pricing (peak/off-peak billing from 2026-08-16) expressed as structured tiers. - Capability probe verification (switchable): sends minimal requests to live-test capabilities missing from metadata (tool calling / structured output / streaming), results cached per (baseUrl, model). - Cache and concurrency safety: classification/probe/mirror results cached with TTL; file locking + atomic writes prevent concurrent processes from corrupting each other; corrupted caches auto-recover. - Interactive config generation: the pick command lists models and prices; select one to generate a dsh config fragment. - Self-contained: manifest + entry factory + tool/event interfaces, directly loadable by the harness (see integration docs). ## How It Works baseUrl + apiKey │ ▼ ① Host detection (probe endpoints, result cached 1h) ├─ /models rich metadata → augmented ├─ /v1/models → ├─ /api/pricing → quota │ ├─ /version → vllm │ ├─ /model/info → flag │ └─ otherwise → bare ├─ /api/tags → ollama └─ all failed → unknown (error; override with --kind) │ ▼ ② Fetch (parse model listings/pricing/capabilities by type) │ ▼ ③ Normalize (unit conversion + source-chain completion + alias resolution) │ ▼ ④ Capability probing (optional; only fills gaps missing from metadata) │ ▼ ⑤ Output ├─ out/catalog.json full catalog (schema: model-catalog/v1) ├─ out/dsh-models.json dsh config fragment (schema: dsh/models/v1) └─ out/report.md human-readable report ## Installing in DSH bash dsh plugin --profile demo add github:JohnXu22786/model-catalog Remove with: bash dsh plugin --profile demo remove model-catalog ## Quick Start Requirements: Node.js ≥ 21 (no runtime dependencies; only TypeScript needed to build). bash npm install # install dev dependencies npm run build # compile to dist/ # Discover the DeepSeek official API model catalog export DEEPSEEK_API_KEY=sk-xxx node dist/src/main.js discover --base-url https://api.deepseek.com # Discover a local Ollama node dist/src/main.js discover --base-url http://127.0.0.1:11434 --probe always # Relay gateway (multiplier pricing) node dist/src/main.js discover --base-url https://gateway.example.com --api-key-env GATEWAY_KEY # Interactively pick models and generate a dsh config fragment (answers can be piped in one shot: # line 1 = baseUrl, line 2 = key env var name (blank = auto-detect), line 3 = model number) node dist/src/main.js pick --base-url https://api.deepseek.com After running, three artifacts land in the out/ directory. Hand dsh-models.json to the dsh harness to complete model configuration (field docs in integration docs). ## Host Types | Type | Detection | Pricing source | Notes | |---|---|---|---| | bare standard compatible | /v1/models with minimal fields only | none → fallback chain | The most basic OpenAI-compatible form; the DeepSeek official API is this type, completed by the built-in default table | | augmented rich metadata | /models includes context_length/pricing/supported_parameters | provided directly by the host (USD-per-token strings) | Implemented by a few gateways | | quota multiplier-priced gateway | /api/pricing (compatible with legacy map form) | multiplier × $2.0 × group multiplier | Common quota pricing system in relay gateways | | flag capability-flag proxy | /model/info (falls back to /v1/model/info on 404) | per-token USD numbers | Returns capability booleans and prices | | ollama | /api/tags | none → fallback chain | Local service; /api/show provides capabilities and context | | vllm | /v1/models + /version | none → fallback chain | Local inference service, no key required | ## Normalization and Unit Conversion All per-token prices in the catalog are normalized to USD per million tokens (USD/1M); per-call prices are marked separately: | Input form | Conversion | |---|---| | USD-per-token string (e.g. "0.00000056", "$0.00003") | ×1e6, rounded to 6 decimal places | | USD-per-token number (e.g. 1.5e-7) | ×1e6 | | Multiplier (quota system, model_ratio) | input = model_ratio × 2.0 × group_ratio; output = model_ratio × completion_ratio × 2.0 × group_ratio (completion_ratio/group_ratio default 1; group_ratio uses the default group when it is a map) | | Per-call (quota_type=1) | per-call USD = model_price × group_ratio, billing: "per-call" | augmented pricing field mapping: prompt → input, completion → output, input_cache_read → cacheRead, input_cache_write → cacheWrite, internal_reasoning → internalReasoning. ## Pricing Source Priority 1. Host endpoint (full trust: if the host provides some prices, no other sources are consulted) 2. User override config data/overrides.json (replacement semantics: override only the fields configured) 3. External price mirror --external-url (only fills fully-missing pricing; result cached 1h) 4. Built-in default table data/builtin-table.json (only fills gaps; includes DeepSeek official model facts) 5. None of the above → pricing: null, marked "unknown" in the report with a warning Capability fields, context and max output follow the same "host > override > mirror > built-in table" priority (override is replacement; the rest only fill gaps). ### Dynamic Pricing The DeepSeek official API has billed by peak/off-peak time-of-day since 2026-08-16 (peak: 01:00–04:00 and 06:00–10:00 UTC; half price at other times). In the catalog: - pricing.dynamic: true; - pricing.amounts holds the baseline tier (first off-peak tier) prices; - pricing.tiers carries all tiers (label, UTC window, per-tier prices); - the report (report.md) lists dynamic-priced models separately with a note that billing is per-tier; - the built-in default table records the official prices of the day; if they differ from the official docs, follow the docs and update data/builtin-table.json. ## Capability Probing For capabilities missing from metadata (tool calling / structured output / streaming), sends minimal probe requests (max_tokens tiny, message very short) to live-test host support: - Tool calling: sends tools + forced tool_choice; - Structured output: response_format: {type: "json_object"} (message contains the word "json" to dodge the classic JSON-mode rejection trap); - Streaming: stream: true; the response must actually be SSE (has data: events and a [DONE] terminator); a plain JSON body is judged unsupported. Interpretation: 2xx = supported; 400/404/405/422 = unsupported (error-body summary kept as evidence); 401/403 = abort all probing with a warning (avoid wasting quota); 5xx/network/timeout = stays unknown with a short-lived error cache. - Modes: auto (default; probes only with a key or a local host) / never / always; - Results cached per (baseUrl, model, capability) for 24h (errors 30m); - Capabilities that cannot be cheaply live-tested (vision, parallel tools) rely on metadata only; no probing. ## Cache and Concurrency Safety - Cache directory var/ (vault.json): host classification (1h), probe results (24h/30m), external mirror (1h); - All writes go through "temp file + atomic rename"; corrupted cache files are reset automatically without affecting operation; - Cross-process file lock (var/.lock): stale locks (>5 minutes) are taken over automatically; wait timeout (10s) raises an error; - cache --clear wipes everything in one command. ## Configuration Config file catalog.config.json (in the plugin root; overridable with --config FILE; all fields optional): json { "baseUrl": "https://api.deepseek.com", "apiKeyEnv": "DEEPSEEK_API_KEY", "probe": "auto", "externalUrl": "https://example.com/mirror.json", "outputDir": "out", "cacheDir": "var", "httpTimeoutMs": 10000 } | Field | Default | Description | |---|---|---| | baseUrl | none | Host address; also --base-url on the CLI | | apiKeyEnv | auto | Key environment variable name; auto-detects MODELCAT_API_KEY / DEEPSEEK_API_KEY / OPENAI_API_KEY | | kindHint | none | Force host type (bare/augmented/quota/flag/ollama/vllm) | | probe | auto | Capability probe mode | | catalogTtlSec | 900 | Catalog freshness (reused by the plugin list tool) | | probeTtlSec | 86400 | Probe result cache TTL | | detectTtlSec | 3600 | Host classification cache TTL | | externalUrl | none | External price mirror URL (structure in data/mirror.example.json) | | outputDir / cacheDir | out / var | Output and cache directories | | concurrency | 4 | Fetch concurrency (e.g. Ollama /api/show) | | httpTimeoutMs | 10000 | HTTP timeout | Keys are only passed via environment variables (--api-key is for transient CLI use only); no artifact file ever contains a plaintext key. ## Data Files | File | Purpose | |---|---| | data/builtin-table.json | Built-in default table (fallback): DeepSeek official model facts and time-of-day pricing | | data/overrides.example.json → copy to data/overrides.json | Manual overrides: per-field replacement | | data/aliases.example.json → copy to data/aliases.json | Alias mapping: old names/aliases → canonical ids | | data/mirror.example.json | External mirror format example (self-hostable isomorphic JSON) | ## Outputs | File | Content | |---|---| | out/catalog.json | Full catalog (schema model-catalog/v1): metadata, warnings, all normalized entries | | out/dsh-models.json | dsh config fragment (schema dsh/models/v1): directly consumable by the harness | | out/report.md | Human-readable: catalog tables, unknown-pricing list, dynamic pricing, warnings | ## CLI Overview model-catalog discover [args] discover and output the model catalog (default command) model-catalog pick [args] interactively pick models and generate a dsh config fragment model-catalog probe --model ID [args] run capability probing for a single model model-catalog cache --clear clear the cache model-catalog config show effective config --base-url URL --api-key-env NAME --api-key KEY --kind KIND --probe MODE --out DIR --cache DIR --external-url URL --config FILE --model ID --help ## dsh Integration The plugin is self-contained: manifest.json declares the entry and interfaces, and dist/src/plugin.js exports a createPlugin() factory registering 5 tools (catalog.discover/list/refresh/select/probe) and 2 events (catalog.updated/catalog.failed). It is also installable via the dsh bundle (package.json → dsh.bundle → cordis.patch.yml): the Cordis entry dist/src/dsh.js exports name/inject/apply and registers the same 5 tools on the harness. Harness loading, tool parameters and returns, event payloads, and how config fragments are consumed are all covered in docs/integration.md. ## Limitations and Notes - Capabilities like vision and parallel tools cannot be cheaply live-tested; they come from metadata only (unknown when missing); - If a multiplier gateway's group_ratio is a map without a default group, it counts as 1 — actual group multipliers can be corrected in the override config; - Probe requests consume a small number of tokens (at most 3 requests per model, 1–16 tokens each); keyless standard hosts are not probed in auto mode; - Built-in default table prices are fallbacks; for dynamic pricing, follow the official docs and the tiers. ## Development bash npm run build # compile TypeScript npm test # compile + run all tests (node:test, no external test deps) Test coverage: unit conversion, host detection, per-type fetching, normalization priority chain, probe interpretation and caching, cache store (TTL/locking/corruption recovery), output artifacts, and an end-to-end pipeline (local mock host). ## License MIT — see LICENSE. Copyright (c) 2026 JohnXu22786.