Api Key Pool

--- **English** | [中文](#中文文档) --- ## English ### ✨ Features | Feature | Description | |---|---| | 🔄 **Round-robin rotation** | Multiple keys per provider, rotated per request for load balancing | | 🛡️ **Automatic failover** | Key fails with `401 / 403 / 429` → automatically marked unhealthy, next key takes over | | ⏱️ **Cooldown & recovery** | Failed keys enter exponential-backoff cooldown (30s default), auto back in rotation when expired | | 🌐 **Web management panel** | DSH Web UI → Settings → Plugins → **API Key Pool**: add/remove keys, view health, reset cooldowns | | 🔒 **Key masking** | REST API only exposes masked keys (`sk-abc****1234`), never full secrets | | 💾 **Persistent config** | Keys added via Web UI persist to `pool-config.json`, survive restarts | ### 📦 Installation ```bash # Clone the repo git clone https://github.com/xiaozhe7772222/dsh-api-key-pool.git cd dsh-api-key-pool # Copy into your DSH profile's plugins directory mkdir -p ~/.dsh/profiles/web/plugins/dsh-api-key-pool cp -r lib package.json cordis.patch.yml ~/.dsh/profiles/web/plugins/dsh-api-key-pool/ ``` Then declare the bundle in `~/.dsh/profiles/web/package.json`: ```json { "dsh": { "profile": { "bundles": [ { "name": "dsh-api-key-pool", "platform": ["web"], "optional": false } ] } } } ``` Restart DSH: ```bash npx @deepseek-ai/dsh web ``` ### ⚙️ Configuration #### Option 1: Static config via `cordis.patch.yml` ```yaml - insert: - id: api-key-pool name: dsh-api-key-pool inject: [llm, webServer] config: pools: provider-a: apiKeyEnv: PROVIDER_A_API_KEY keys: - sk-your-first-key - sk-your-second-key - sk-your-third-key cooldownMs: 30000 provider-b: apiKeyEnv: PROVIDER_B_API_KEY keys: - sk-key2-1 - sk-key2-2 cooldownMs: 60000 defaultCooldownMs: 30000 ``` | Field | Type | Description | |---|---|---| | `pools..apiKeyEnv` | string | Env var name for this provider's API key (must match `apiKeyEnv` in `settings.yaml`) | | `pools..keys` | string[] | List of API keys to rotate | | `pools..cooldownMs` | number | Cooldown per key after failure (ms) | | `defaultCooldownMs` | number | Global default cooldown (ms), default `30000` | #### Option 2: Web management panel (runtime) After starting DSH, open **Settings → Plugins → API Key Pool**: - View each provider's key health (green = healthy, orange = cooling) - Paste a new key and click **Add** - Click `✕` to remove a key - Click **Reset cooldown** to clear cooling state for a provider - Click **Refresh** to reload status Keys added via the panel persist to `pool-config.json` automatically. ### 🧠 How it works (v0.3.0) ``` LLM request │ ▼ ┌──────────────────────────────────────────────────┐ │ agent/request waterfall (this plugin v0.3.0) │ │ v0.3.0 fix #1: event payload has NO provider │ │ → call next() first, read provider from the │ │ returned LlmCallConfig │ │ ┌──────────────────────────────────────────────┐ │ │ │ 1. pickKey(provider) — round-robin healthy key│ │ │ │ 2. inject process.env[apiKeyEnv] │ │ │ │ 3. credentials resolve env first → takes effect│ │ │ └──────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────┘ │ ▼ Provider API call │ ├── Success → markSuccess (reset failCount) │ └── Fail 429 (e.g. SenseNova 429001) / 401 / 403 / 5xx ... │ ▼ ┌──────────────────────────────────────────────────┐ │ agent/request-error waterfall (this plugin) │ │ v0.3.0 fix #2: no-error-left-behind │ │ ANY 4xx/5xx status, or message containing │ │ rate/limit/quota/exhaust/timeout/auth → retry │ │ v0.3.0 fix #3: return { kind: 'retry' } │ │ agent loop re-issues the request (not just │ │ mark-failed-and-quit) │ │ ┌──────────────────────────────────────────────┐ │ │ │ 1. markFailed(currentKey) → cooldown │ │ │ │ 2. return { kind: 'retry' } │ │ │ └──────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────┘ │ ▼ agent loop retries → agent/request again pickKey skips the cooling key → picks next key │ ├── key2 again 429 → cooldown → retry → key3 → OK ✅ │ └── all keys cooling → keeps retrying (wait for cooldown) ``` **Key design**: DSH resolves credentials in the order `process.env > .credentials.yaml > .env`. This plugin leverages that — it temporarily rewrites `process.env` in the `agent/request` waterfall for **hot key switching with zero config file changes**. Since v0.3.0, a failed key not only enters cooldown but also asks the agent loop to **retry with the next key immediately** (previously it only marked the key and the request died). ### 🔌 REST API The plugin registers routes on the DSH web server. #### `GET /dsh-api-key-pool/pools` Returns all pools with masked keys, health states, and env names: ```json { "pools": { "provider-a": { "apiKeyEnv": "PROVIDER_A_API_KEY", "keys": ["sk-your-first-key"], "maskedKeys": ["sk-you****-key"], "states": { "sk-you****-key": { "failCount": 0, "cooldownUntil": 0 } } } }, "defaultCooldownMs": 30000 } ``` #### `POST /dsh-api-key-pool/pools` | action | body | Description | |---|---|---| | `add` | `{ action, provider, key }` | Add a key | | `remove` | `{ action, provider, key }` | Remove a key | | `update` | `{ action, provider, keys: [] }` | Replace the full key list | | `reset` | `{ action, provider }` | Reset all cooldown states for a provider | Example: ```bash curl -X POST http://127.0.0.1:3080/dsh-api-key-pool/pools \ -H 'Content-Type: application/json' \ -d '{"action":"add","provider":"provider-a","key":"sk-new-key"}' ``` ### 🗂️ Project structure ``` dsh-api-key-pool/ ├── package.json # Package metadata + DSH bundle declaration ├── cordis.patch.yml # Static pool config (first-install example) ├── pool-config.json # Runtime persistence (written by Web UI, gitignored) ├── lib/ │ ├── index.js # Server: rotation logic + agent waterfalls + REST API │ └── client.js # Client: Web settings panel card ├── README.md ├── CHANGELOG.md └── LICENSE ``` ### 🏷️ Topics `deepseek-harness` · `dsh` · `api-key` · `api-key-rotation` · `key-pool` · `failover` · `load-balancing` · `round-robin` · `circuit-breaker` · `llm` · `openai` · `plugin` · `cordis` ### 📄 License MIT License — free to use, modify, distribute. **Never commit your real API keys to a public repo.** --- ## 中文文档 > 🚀 **解决在 DeepSeek Harness 框架上面使用中转站模型或者第三方模型限流问题** ### ✨ 功能特性 | 功能 | 说明 | |---|---| | 🔄 **自动轮换** | 同一供应商配置多个 Key，请求时依次轮换，均衡负载 | | 🛡️ **失败切换** | Key 返回 `401 / 403 / 429` 时自动标记故障，切换到下一个健康 Key | | ⏱️ **冷却恢复** | 故障 Key 按指数退避进入冷却期（默认 30s 起步），到期自动回到轮换池 | | 🌐 **Web 管理面板** | DSH Web UI → 设置 → 插件配置 → **API Key 池**：直接增删 Key、查看健康状态、重置冷却 | | 🔒 **Key 脱敏** | REST 接口对外只返回脱敏后的 Key（`sk-abc****1234`），不泄露完整密钥 | | 💾 **配置持久化** | Web 面板添加的 Key 持久化到 `pool-config.json`，重启后依然有效 | ### 📦 安装 ```bash # 克隆仓库 git clone https://github.com/xiaozhe7772222/dsh-api-key-pool.git cd dsh-api-key-pool # 拷贝到 DSH profile 的 plugins 目录 mkdir -p ~/.dsh/profiles/web/plugins/dsh-api-key-pool cp -r lib package.json cordis.patch.yml ~/.dsh/profiles/web/plugins/dsh-api-key-pool/ ``` 然后在 `~/.dsh/profiles/web/package.json` 的 `dsh.profile.bundles` 中声明： ```json { "dsh": { "profile": { "bundles": [ { "name": "dsh-api-key-pool", "platform": ["web"], "optional": false } ] } } } ``` 重启 DSH： ```bash npx @deepseek-ai/dsh web ``` ### ⚙️ 配置说明 #### 方式一：cordis.patch.yml（静态配置） ```yaml - insert: - id: api-key-pool name: dsh-api-key-pool inject: [llm, webServer] config: pools: provider-a: apiKeyEnv: PROVIDER_A_API_KEY keys: - sk-your-first-key - sk-your-second-key - sk-your-third-key cooldownMs: 30000 provider-b: apiKeyEnv: PROVIDER_B_API_KEY keys: - sk-key2-1 - sk-key2-2 cooldownMs: 60000 defaultCooldownMs: 30000 ``` | 字段 | 类型 | 说明 | |---|---|---| | `pools..apiKeyEnv` | string | 该供应商的 API Key 对应的环境变量名（需与 `settings.yaml` 中 `apiKeyEnv` 一致） | | `pools..keys` | string[] | 要轮换的多个 API Key | | `pools..cooldownMs` | number | 单个 Key 失败后的冷却时长（毫秒） | | `defaultCooldownMs` | number | 全局默认冷却时长（毫秒），默认 `30000` | #### 方式二：Web 管理面板（运行时） 启动 DSH 后，打开 **设置 → 插件配置 → API Key 池**： - 查看各供应商 Key 健康状态（绿点 = 健康，橙点 = 冷却中） - 输入框粘贴新 Key，点击「添加」 - 点击 `✕` 移除不想用的 Key - 点击「重置冷却」清空某个供应商的冷却状态 - 点击「刷新」重新加载状态 Web 面板添加的 Key 会自动持久化到 `pool-config.json`，重启后依然有效。 ### 🧠 工作原理（v0.3.0） ``` LLM 请求 │ ▼ ┌──────────────────────────────────────────────────┐ │ agent/request 瀑布（本插件 v0.3.0） │ │ v0.3.0 修复1：事件 payload 没有 provider 字段 │ │ → 先调 next()，从返回的 LlmCallConfig 里取 provider│ │ ┌──────────────────────────────────────────────┐ │ │ │ 1. pickKey(provider) 轮询选健康 Key │ │ │ │ 2. 注入 process.env[apiKeyEnv] │ │ │ │ 3. 凭据解析优先读 env → Key 立即生效 │ │ │ └──────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────┘ │ ▼ 供应商 API 调用 │ ├── 成功 → markSuccess（重置失败计数） │ └── 失败 429（如商汤 429001）/ 401 / 403 / 5xx ... │ ▼ ┌──────────────────────────────────────────────────┐ │ agent/request-error 瀑布（本插件拦截） │ │ v0.3.0 修复2：宁多勿漏 │ │ 任何 4xx/5xx 状态码，或消息含 rate/limit/quota/ │ │ exhaust/timeout/auth 等关键词 → 均判定可重试 │ │ v0.3.0 修复3：返回 { kind: 'retry' } │ │ agent loop 收到后重新发起请求（不再直接退出！） │ │ ┌──────────────────────────────────────────────┐ │ │ │ 1. markFailed(当前key) → 进入冷却 │ │ │ │ 2. return { kind: 'retry' } │ │ │ └──────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────┘ │ ▼ agent loop 重试 → 再次进入 agent/request pickKey 跳过冷却中的 key → 选中下一个 key │ ├── key2 也 429 → 标记冷却 → retry → key3 → 成功 ✅ │ └── 全部 key 冷却 → 继续重试（等冷却），日志预警 ``` **关键设计**：DSH 的凭据解析优先级是 `process.env > .credentials.yaml > .env`，本插件利用这一机制，在 `agent/request` 瀑布中临时改写 `process.env`，实现**零配置文件改动的 Key 热切换**。v0.3.0 起，失败的 Key 不仅进入冷却，还会让 agent loop **立刻换下一个 Key 重试**（此前只是标记了故障但请求直接失败，等于白轮换）。 ### 🔌 REST API 插件在 DSH webServer 上注册以下路由： #### `GET /dsh-api-key-pool/pools` 返回所有池的 Key 列表（脱敏）、健康状态、环境变量名： ```json { "pools": { "provider-a": { "apiKeyEnv": "PROVIDER_A_API_KEY", "keys": ["sk-your-first-key"], "maskedKeys": ["sk-you****-key"], "states": { "sk-you****-key": { "failCount": 0, "cooldownUntil": 0 } } } }, "defaultCooldownMs": 30000 } ``` #### `POST /dsh-api-key-pool/pools` | action | body | 说明 | |---|---|---| | `add` | `{ action, provider, key }` | 添加一个 Key | | `remove` | `{ action, provider, key }` | 移除一个 Key | | `update` | `{ action, provider, keys: [] }` | 整体替换 Key 列表 | | `reset` | `{ action, provider }` | 重置该供应商全部冷却状态 | 示例： ```bash curl -X POST http://127.0.0.1:3080/dsh-api-key-pool/pools \ -H 'Content-Type: application/json' \ -d '{"action":"add","provider":"provider-a","key":"sk-new-key"}' ``` ### 🗂️ 项目结构 ``` dsh-api-key-pool/ ├── package.json # 包元数据 + DSH bundle 声明 ├── cordis.patch.yml # 静态池配置（首次安装示例） ├── pool-config.json # 运行时持久化（Web 面板写入，已 gitignore） ├── lib/ │ ├── index.js # 服务端：轮换逻辑 + agent 瀑布 + REST API │ └── client.js # 客户端：Web 设置面板卡片 ├── README.md ├── CHANGELOG.md └── LICENSE ``` ### 🏷️ 标签 (Topics) `deepseek-harness` · `dsh` · `api-key` · `api-key-rotation` · `key-pool` · `failover` · `load-balancing` · `round-robin` · `circuit-breaker` · `llm` · `openai` · `plugin` · `cordis` ### 📄 许可证 MIT License — 自由使用、修改、分发。**请勿将你的真实 API Key 提交到公开仓库。**