Files
slate/app/lib/modules/llm/providers/deepseek.ts
Stijnus b7ef2247b8 feat: add Cerebras and Fireworks AI LLM providers (#2113)
* fix: improve local model provider robustness and UX

- Extract shared Docker URL rewriting and env conversion into BaseProvider
  to eliminate 4x duplicated code across Ollama and LMStudio
- Add error handling and 5s timeouts to all model-listing fetches so one
  unreachable provider doesn't block the entire model list
- Fix Ollama using createOllama() instead of mutating provider internals
- Fix LLMManager singleton ignoring env updates on subsequent requests
- Narrow cache key to only include provider-relevant env vars instead of
  the entire server environment
- Fix 'as any' casts in LMStudio and OpenAILike by using shared
  convertEnvToRecord helper
- Replace console.log/error with structured logger in OpenAILike
- Fix typo: filteredStaticModesl -> filteredStaticModels in manager
- Add connection status indicator (green/red dot) for local providers
  in the ModelSelector dropdown
- Show helpful "is X running?" message when local provider has no models

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Cerebras LLM provider

- Add Cerebras provider with 8 models (Llama, GPT OSS, Qwen, ZAI GLM)
- Integrate @ai-sdk/cerebras@0.2.16 for compatibility
- Add CEREBRAS_API_KEY to environment configuration
- Register provider in LLMManager registry

Models included:
- llama3.1-8b, llama-3.3-70b
- gpt-oss-120b (reasoning)
- qwen-3-32b, qwen-3-235b variants
- zai-glm-4.6, zai-glm-4.7 (reasoning)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add Fireworks AI LLM provider

- Add Fireworks provider with 6 popular models
- Integrate @ai-sdk/fireworks@0.2.16 for compatibility
- Add FIREWORKS_API_KEY to environment configuration
- Register provider in LLMManager registry

Models included:
- Llama 3.1 variants (405B, 70B, 8B Instruct)
- DeepSeek R1 (reasoning model)
- Qwen 2.5 72B Instruct
- FireFunction V2

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add coding-specific models to existing providers

Enhanced providers with state-of-the-art coding models:

**DeepSeek Provider:**
+ DeepSeek V3.2 (integrates thinking + tool-use)
+ DeepSeek V3.2-Speciale (high-compute variant, beats GPT-5)

**Fireworks Provider:**
+ Qwen3-Coder 480B (262K context, best for coding)
+ Qwen3-Coder 30B (fast coding specialist)

**Cerebras Provider:**
+ Qwen3-Coder 480B (2000 tokens/sec!)
- Removed deprecated models (qwen-3-32b, llama-3.3-70b)

Total new models: 4
Total coding models across all providers: 12+

Performance highlights:
- Qwen3-Coder: State-of-the-art coding performance
- DeepSeek V3.2: Integrates thinking directly into tool-use
- ZAI GLM 4.6: 73.8% SWE-bench score
- Ultra-fast inference: 2000 tok/s on Cerebras

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add dynamic model discovery to providers

Implemented getDynamicModels() for automatic model discovery:

**DeepSeek Provider:**
- Fetches models from https://api.deepseek.com/models
- Automatically discovers new models as DeepSeek adds them
- Filters out static models to avoid duplicates

**Cerebras Provider:**
- Fetches models from https://api.cerebras.ai/v1/models
- Auto-discovers new Cerebras models
- Keeps UI up-to-date with latest offerings

**Fireworks Provider:**
- Fetches from https://api.fireworks.ai/v1/accounts/fireworks/models
- Includes context_length from API response
- Discovers new Qwen-Coder and other models automatically

**Moonshot Provider:**
- Fetches from https://api.moonshot.ai/v1/models
- OpenAI-compatible endpoint
- Auto-discovers new Kimi models

Benefits:
-  No manual updates needed when providers add new models
-  Users always have access to latest models
-  Graceful fallback to static models if API fails
-  5-second timeout prevents hanging
-  Caching system built into BaseProvider

Technical details:
- Uses BaseProvider's built-in caching system
- Cache invalidates when API keys change
- Failed API calls fallback to static models
- All endpoints have 5-second timeout protection

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: add Z.AI provider with GLM models and JWT authentication

Merged changes from PR #2069 to add Z.AI provider:
- Added GLM-4.6 (200K), GLM-4.5 (128K), and GLM-4.5 Flash models
- Implemented secure JWT token generation with HMAC-SHA256 signing
- Added dynamic model discovery from Z.AI API
- Included proper error handling and token validation
- GLM-4.6 achieves 73.8% on SWE-bench coding benchmarks

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 15:26:46 +01:00

134 lines
3.5 KiB
TypeScript

import { BaseProvider } from '~/lib/modules/llm/base-provider';
import type { ModelInfo } from '~/lib/modules/llm/types';
import type { IProviderSetting } from '~/types/model';
import type { LanguageModelV1 } from 'ai';
import { createDeepSeek } from '@ai-sdk/deepseek';
export default class DeepseekProvider extends BaseProvider {
name = 'Deepseek';
getApiKeyLink = 'https://platform.deepseek.com/apiKeys';
config = {
apiTokenKey: 'DEEPSEEK_API_KEY',
};
staticModels: ModelInfo[] = [
{
name: 'deepseek-coder',
label: 'Deepseek-Coder',
provider: 'Deepseek',
maxTokenAllowed: 8000,
maxCompletionTokens: 8192,
},
{
name: 'deepseek-chat',
label: 'Deepseek-Chat',
provider: 'Deepseek',
maxTokenAllowed: 8000,
maxCompletionTokens: 8192,
},
{
name: 'deepseek-reasoner',
label: 'Deepseek-Reasoner',
provider: 'Deepseek',
maxTokenAllowed: 8000,
maxCompletionTokens: 8192,
},
{
name: 'deepseek-v3.2',
label: 'DeepSeek V3.2 (Coding + Tool Use)',
provider: 'Deepseek',
maxTokenAllowed: 64000,
maxCompletionTokens: 8192,
},
{
name: 'deepseek-v3.2-speciale',
label: 'DeepSeek V3.2 Speciale (High-Compute)',
provider: 'Deepseek',
maxTokenAllowed: 64000,
maxCompletionTokens: 8192,
},
];
async getDynamicModels(
apiKeys?: Record<string, string>,
settings?: IProviderSetting,
serverEnv?: Record<string, string>,
): Promise<ModelInfo[]> {
const { apiKey } = this.getProviderBaseUrlAndKey({
apiKeys,
providerSettings: settings,
serverEnv: serverEnv as any,
defaultBaseUrlKey: '',
defaultApiTokenKey: 'DEEPSEEK_API_KEY',
});
if (!apiKey) {
return [];
}
try {
const response = await fetch('https://api.deepseek.com/models', {
headers: {
Authorization: `Bearer ${apiKey}`,
},
signal: this.createTimeoutSignal(5000),
});
if (!response.ok) {
console.error(`DeepSeek API error: ${response.statusText}`);
return [];
}
const data = (await response.json()) as any;
const staticModelIds = this.staticModels.map((m) => m.name);
// Filter out models we already have in staticModels
const dynamicModels =
data.data
?.filter((model: any) => !staticModelIds.includes(model.id))
.map((m: any) => ({
name: m.id,
label: `${m.id} (Dynamic)`,
provider: this.name,
maxTokenAllowed: 64000, // Default, adjust per model if available
maxCompletionTokens: 8192,
})) || [];
return dynamicModels;
} catch (error) {
console.error(`Failed to fetch DeepSeek models:`, error);
return [];
}
}
getModelInstance(options: {
model: string;
serverEnv: Env;
apiKeys?: Record<string, string>;
providerSettings?: Record<string, IProviderSetting>;
}): LanguageModelV1 {
const { model, serverEnv, apiKeys, providerSettings } = options;
const { apiKey } = this.getProviderBaseUrlAndKey({
apiKeys,
providerSettings: providerSettings?.[this.name],
serverEnv: serverEnv as any,
defaultBaseUrlKey: '',
defaultApiTokenKey: 'DEEPSEEK_API_KEY',
});
if (!apiKey) {
throw new Error(`Missing API key for ${this.name} provider`);
}
const deepseek = createDeepSeek({
apiKey,
});
return deepseek(model, {
// simulateStreaming: true,
});
}
}