Repository navigation
DeepseekException - Expecting value: line 1 column 1 (char 0) #1274
Description
Describe the bug
My issue is similar to #8182, and the logs content in the wren-ai-service container looks something like this:
litellm.exceptions.APIError: litellm.APIError: APIError: Deepseek Exception - Expecting value: line 1 column 1 (char 0)To Reproduce
The .env file I use has the following contents:
# version
# CHANGE THIS TO THE LATEST VERSION
WREN_PRODUCT_VERSION=0.15.3
WREN_ENGINE_VERSION=0.13.1
WREN_AI_SERVICE_VERSION=0.15.7
IBIS_SERVER_VERSION=0.13.1
WREN_UI_VERSION=0.20.1
WREN_BOOTSTRAP_VERSION=0.1.5
#...
DEEPSEEK_API_KEY=sk-***Expected behavior
What I found on LiteLLM related issues:
In #8266 has fixed the issue.
When (or if will) WrenAI update the LiteLLM version to v1.64.0?
from LiteLLM v1.60.4 Changelog:
Fix deepseek calling - refactor to use base_llm_http_handler by @krrishdholakia in BerriAI/litellm#8266
Thanks!!!
config.yaml FILE
# you should rename this file to config.yaml and put it in ~/.wrenai
# please pay attention to the comments starting with # and adjust the config accordingly
type: llm
provider: litellm_llm
timeout: 120
models:
# put DEEPSEEK_API_KEY=<your_api_key> in ~/.wrenai/.env
- api_base: https://api.deepseek.com/v1
kwargs:
n: 1
temperature: 0
response_format:
type: text
model: deepseek/deepseek-reasoner
- api_base: https://api.deepseek.com/v1
kwargs:
n: 1
temperature: 0
response_format:
type: text
model: deepseek/deepseek-chat
- api_base: https://api.deepseek.com/v1
kwargs:
n: 1
temperature: 0
response_format:
type: json_object
model: deepseek/deepseek-coder
---
type: embedder
provider: litellm_embedder
models:
# define OPENAI_API_KEY=<api_key> in ~/.wrenai/.env if you are using openai embedding model
# please refer to LiteLLM documentation for more details: https://docs.litellm.ai/docs/providers
- model: openai/bge-large # put your ollama embedder model name here, openai/<ollama_model_name>
api_base: http://host.docker.internal:11434/v1 # change this to your ollama host, api_base should be <ollama_url>/v1
timeout: 120
---
type: engine
provider: wren_ui
endpoint: http://wren-ui:3000
---
type: document_store
provider: qdrant
location: http://qdrant:6333
embedding_model_dim: 1024 # put your embedding model dimension here
timeout: 120
recreate_index: false
# recreate_collection: true
---
# please change the llm and embedder names to the ones you want to use
# the format of llm and embedder should be <provider>.<model_name> such as litellm_llm.gpt-4o-2024-08-06
# the pipes may be not the latest version, please refer to the latest version: https://raw.githubusercontent.com/canner/WrenAI/<WRENAI_VERSION_NUMBER>/docker/config.example.yaml
type: pipeline
pipes:
- name: db_schema_indexing
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: historical_question_indexing
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: table_description_indexing
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: db_schema_retrieval
llm: litellm_llm.deepseek/deepseek-coder
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: historical_question_retrieval
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: sql_generation
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: sql_correction
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: followup_sql_generation
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: sql_summary
llm: litellm_llm.deepseek/deepseek-coder
- name: sql_answer
llm: litellm_llm.deepseek/deepseek-chat
engine: wren_ui
- name: sql_breakdown
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: sql_expansion
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: sql_explanation
llm: litellm_llm.deepseek/deepseek-coder
- name: sql_regeneration
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: semantics_description
llm: litellm_llm.deepseek/deepseek-coder
- name: relationship_recommendation
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: question_recommendation
llm: litellm_llm.deepseek/deepseek-coder
- name: question_recommendation_db_schema_retrieval
llm: litellm_llm.deepseek/deepseek-coder
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: question_recommendation_sql_generation
llm: litellm_llm.deepseek/deepseek-coder
engine: wren_ui
- name: chart_generation
llm: litellm_llm.deepseek/deepseek-coder
- name: chart_adjustment
llm: litellm_llm.deepseek/deepseek-coder
- name: intent_classification
llm: litellm_llm.deepseek/deepseek-coder
embedder: litellm_embedder.openai/bge-large
document_store: qdrant
- name: data_assistance
llm: litellm_llm.deepseek/deepseek-chat
- name: sql_pairs_indexing
document_store: qdrant
embedder: litellm_embedder.openai/bge-large
- name: sql_pairs_deletion
document_store: qdrant
embedder: litellm_embedder.openai/bge-large
- name: sql_pairs_retrieval
document_store: qdrant
embedder: litellm_embedder.openai/bge-large
llm: litellm_llm.deepseek/deepseek-coder
- name: preprocess_sql_data
llm: litellm_llm.deepseek/deepseek-coder
- name: sql_executor
engine: wren_ui
- name: sql_question_generation
llm: litellm_llm.deepseek/deepseek-coder
- name: sql_generation_reasoning
llm: litellm_llm.deepseek/deepseek-reasoner
---
settings:
column_indexing_batch_size: 500
table_retrieval_size: 100
table_column_retrieval_size: 1000
allow_using_db_schemas_without_pruning: false # if you want to use db schemas without pruning, set this to true. It will be faster
query_cache_maxsize: 1000
query_cache_ttl: 3600
langfuse_host: https://cloud.langfuse.com
langfuse_enable: true
logging_level: DEBUG
development: true.env FILE
COMPOSE_PROJECT_NAME=wrenai
PLATFORM=linux/amd64
PROJECT_DIR=.
# service port
WREN_ENGINE_PORT=8080
WREN_ENGINE_SQL_PORT=7432
WREN_AI_SERVICE_PORT=5555
WREN_UI_PORT=3000
IBIS_SERVER_PORT=8000
WREN_UI_ENDPOINT=http://wren-ui:${WREN_UI_PORT}
# ai service settings
QDRANT_HOST=qdrant
SHOULD_FORCE_DEPLOY=1
# vendor keys
LLM_OPENAI_API_KEY=
EMBEDDER_OPENAI_API_KEY=
LLM_AZURE_OPENAI_API_KEY=
EMBEDDER_AZURE_OPENAI_API_KEY=
QDRANT_API_KEY=
# version
# CHANGE THIS TO THE LATEST VERSION
WREN_PRODUCT_VERSION=0.15.3
WREN_ENGINE_VERSION=0.13.1
WREN_AI_SERVICE_VERSION=0.15.7
IBIS_SERVER_VERSION=0.13.1
WREN_UI_VERSION=0.20.1
WREN_BOOTSTRAP_VERSION=0.1.5
# user id (uuid v4)
USER_UUID=
# for other services
POSTHOG_API_KEY=phc_nhF32aj4xHXOZb0oqr2cn4Oy9uiWzz6CCP4KZmRq9aE
POSTHOG_HOST=https://app.posthog.com
TELEMETRY_ENABLED=false
# this is for telemetry to know the model, i think ai-service might be able to provide a endpoint to get the information
GENERATION_MODEL=gpt-4o-mini
LANGFUSE_SECRET_KEY=
LANGFUSE_PUBLIC_KEY=
# the port exposes to the host
# OPTIONAL: change the port if you have a conflict
HOST_PORT=3000
AI_SERVICE_FORWARD_PORT=5555
# Wren UI
EXPERIMENTAL_ENGINE_RUST_VERSION=false
LLM_LM_STUDIO_API_KEY=random
OPENAI_API_KEY=sk-***
DEEPSEEK_API_KEY=sk-***wren-ai-service LOG
********************************************************************************
> generate [src.pipelines.generation.question_recommendation.generate()] encountered an error<
> Node inputs:
{'generator': '<function LitellmLLMProvider.get_generator.<locals...',
'prompt': "<Task finished name='Task-717' coro=<AsyncGraphAda..."}
********************************************************************************
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 771, in acompletion
headers, response = await self.make_openai_chat_completion_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 131, in async_wrapper
result = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 419, in make_openai_chat_completion_request
raise e
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 411, in make_openai_chat_completion_request
response = raw_response.parse()
^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_legacy_response.py", line 137, in parse
parsed = self._parse(to=to)
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_legacy_response.py", line 332, in _parse
data = response.json()
^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 766, in json
return jsonlib.loads(self.content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 463, in acompletion
response = await init_response
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 817, in acompletion
raise OpenAIError(
litellm.llms.openai.common_utils.OpenAIError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn
await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper
self._handle_exception(observation, e)
File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 517, in _handle_exception
raise e
File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper
result = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/src/pipelines/generation/question_recommendation.py", line 48, in generate
return await generator(prompt=prompt.get("prompt"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/src/providers/llm/litellm.py", line 71, in _run
completion: Union[ModelResponse] = await acompletion(
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1358, in wrapper_async
raise e
File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1217, in wrapper_async
result = await original_function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 482, in acompletion
raise exception_type(
^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2190, in exception_type
raise e
File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 450, in exception_type
raise APIError(
litellm.exceptions.APIError: litellm.APIError: APIError: DeepseekException - Expecting value: line 1 column 1 (char 0)
-------------------------------------------------------------------
Oh no an error! Need help with Hamilton?
Join our slack and ask for help! https://join.slack.com/t/hamilton-opensource/shared_invite/zt-2niepkra8-DGKGf_tTYhXuJWBTXtIs4g
-------------------------------------------------------------------
E0206 14:06:29.088 7 wren-ai-service:60] An error occurred during question recommendation generation: litellm.APIError: APIError: DeepseekException - Expecting value: line 1 column 1 (char 0)
Activity
[-]Deepseek Exception - Expecting value: line 1 column 1 (char 0)[/-][+]DeepseekException - Expecting value: line 1 column 1 (char 0)[/+]sys0226 commentedon Feb 6, 2025
The same problem:
Error log
cyyeh commentedon Feb 7, 2025
@guog @sdw777 we've released ai service to 0.15.9(which updated the litellm to the latest version), could you try again?
guog commentedon Feb 7, 2025
@cyyeh Thanks u reply
.env FILE:
wren-ai-serviceOutput LOG:sys0226 commentedon Feb 7, 2025
@cyyeh I have upgraded ai service to 0.15.9, but there is still an error.
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
generate [src.pipelines.generation.question_recommendation.generate()] encountered an error<
Node inputs:
{'generator': '<function LitellmLLMProvider.get_generator.<locals...',
'prompt': "<Task finished name='Task-1099' coro=<AsyncGraphAd..."}
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/chat/gpt_transformation.py", line 235, in transform_response
completion_response = raw_response.json()
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 766, in json
return jsonlib.loads(self.content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 466, in acompletion
response = await init_response
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 182, in async_completion
return provider_config.transform_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai/chat/gpt_transformation.py", line 238, in transform_response
raise OpenAIError(
litellm.llms.openai.common_utils.OpenAIError: Unable to get json response - Expecting value: line 1 column 1 (char 0), Original Response:
jinbu6666 commentedon Feb 7, 2025
me too
cyyeh commentedon Feb 7, 2025
hmm, seems even if litellm is upgraded, the problem is not solved
guog commentedon Feb 7, 2025
Yes.
I tried again with the following version and the problem is still the same.
However, compared to the previous version, this sentence is added to the log:
cyyeh commentedon Feb 7, 2025
would you like to try deepseek models hosted on other platform such as fireworks.ai?
guog commentedon Feb 8, 2025
Thanks reply, again!
Let me try.
guog commentedon Feb 8, 2025
I deployed deepseek-r1:14b locally using Ollama.
All is Well.
config.yamlFILE:cyyeh commentedon Feb 8, 2025
Great to hear that, then I will close this issue now
ufgjjj commentedon Feb 13, 2025
Internal server error
unexpected character: line 1 column 1 (char 0)
2025-02-13 16:36:41 wren-ai-service-1 | E0213 08:36:41.042 8 wren-ai-service:60] Error decoding JSON: unexpected character: line 1 column 1 (char 0)
2025-02-13 16:36:41 wren-ai-service-1 | E0213 08:36:41.042 8 wren-ai-service:60] An error occurred during question recommendation generation: 'list' object has no attribute 'get'
my version is 0.15.3
No matter what I send, the same error is returned.
cyyeh commentedon Feb 13, 2025
@ufgjjj could u share your config.yaml?
ampbuster commentedon Mar 20, 2025
The latest deepseek example config causes this error (all pipelines set to use deepseek-reasoner)
Revert to previous config to fix:
sql_answer and data_assistance = deepseek-chat
sql_generation_reasoning = deepseek-reasoner
everything else = deepseek-coder
fix(core): fix count statement for BigQuery (Canner#1274)