fix: reduce max_completion_tokens for Groq API to 8192#1279
Open
yang1002378395-cmyk wants to merge 4 commits intokhoj-ai:masterfrom
Open
fix: reduce max_completion_tokens for Groq API to 8192#1279yang1002378395-cmyk wants to merge 4 commits intokhoj-ai:masterfrom
yang1002378395-cmyk wants to merge 4 commits intokhoj-ai:masterfrom
Conversation
added 4 commits
March 15, 2026 00:36
When ChatModel.friendly_name is None, __str__ returns None causing: TypeError: __str__ returned non-string (type NoneType) Fixed by falling back to name field when friendly_name is None.
…loading model When SearchModelConfig.ApiType.LOCAL is set with an embeddings_inference_endpoint, Khoj was still downloading the model from HuggingFace instead of using the API. Changes: - Only load SentenceTransformer locally when ApiType.LOCAL and no endpoint configured - Use OpenAI-compatible API for local endpoints (llama.cpp, vLLM, etc.) - Handle None API key for local servers that don't require authentication Fixes khoj-ai#1253
- Add NEXT_PUBLIC_CSP_IMG_DOMAINS env var for build-time configuration - Add KHOJ_CSP_IMG_DOMAINS env var for runtime hint - Add CSPHeadersMiddleware to pass domains via X-Khoj-CSP-Img-Domains header - Document new env var in docker-compose.yml Fixes khoj-ai#1249
Groq API has a maximum of 8192 tokens for max_completion_tokens, but the code was hardcoded to 16000, causing API errors. This fix: - Adds MAX_COMPLETION_TOKENS_GROQ constant (8192) - Uses Groq-specific limit when api_base_url is api.groq.com - Applied to both sync and async completion functions Fixes khoj-ai#1236
14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Groq API has a maximum of 8192 tokens for
max_completion_tokens, but the code was hardcoded to 16000, causing API errors.Fixes #1236
Solution
MAX_COMPLETION_TOKENS_GROQconstant (8192)api_base_urlisapi.groq.comcompletion_with_backoff) and async (chat_completion_with_backoff) functionsTesting
fields_to_excludeis handled for Groq inclean_response_schema)