ai: bump hybridRouter connection timeout 8s → 30s
Manual fire of bb-company-coverage-scan tonight failed because the 8s connect timeout was tripping on llama3.1:70b cold-load when another model was resident in VRAM on AI_001. Bumped to 30s so the orchestrator can wait through a model swap. Also bumped background request timeout 120s → 180s for the same reason — story generation against the 70B can take >2 min on a saturated GPU. Coolify env OLLAMA_ENDPOINT was simultaneously flipped from AI_001 (10.10.0.66) to AI_002 (10.10.0.67) since AI_002 is less loaded — AI_001 is doing author enrichment and scrubber work.
This commit is contained in:
@@ -65,8 +65,11 @@ async function callOllama(
|
||||
endpoint: string,
|
||||
options: HybridOptions
|
||||
): Promise<string> {
|
||||
const requestTimeout = options.isBackground ? 120_000 : 60_000;
|
||||
const connectionTimeout = 8_000;
|
||||
const requestTimeout = options.isBackground ? 180_000 : 60_000;
|
||||
// 30s connection timeout (was 8s) — large models like llama3.1:70b
|
||||
// may need to swap into VRAM on AI_001/AI_002, which can exceed 10s
|
||||
// when another model is currently loaded.
|
||||
const connectionTimeout = 30_000;
|
||||
|
||||
// Build Ollama OpenAI-compatible payload
|
||||
const payload = {
|
||||
|
||||
Reference in New Issue
Block a user