Timeout should be configurable

[nesretep-anp1]

In an evaluation env we do have only CPU based models running which leads certainly to long response times.

In many cases baibot failes due to timeout.

[spantaleev]

What kind of timeout error are you hitting and with which provider?

I've taken a quick look at the major providers we have (openai, openai-compatible and anthropic) and it seems like:

• we don't specify a timeout anywhere
• all these providers use the reqwest library under the hood with a default configuration (no timeouts)

So.. maybe you're hitting some other timeout somewhere else. More information is needed.

[nesretep-anp1]

That's interesting.

Having requests in the mentioned env last longer than 60 seconds baibot fails with ...

task ... panicked with message "called Result::unwrap() on an Err value: Custom { kind: InvalidData, error: "Failed to read JSON: expected value at line 1 column 1" }"

Where I could see (and reproduce with curl), that the request (within the n8n workaround I mentioned in the other issue) get completed normally by AnythingLLM and returns the appropriate data ... several seonds after having baibot already fail.

[spantaleev]

This error seems to indicate an invalid JSON response. Maybe it's your reverse-proxy which kills the request after 60 seconds and responds with some kind of HTML.

[nesretep-anp1]

One would think that; but the same error message occurs if response is empty

Like said, ...

1. If using curl on the proxy everything works find.
2. Via baibot I could see that n8n still waits for the response and then renders the response to baibot correctly ((full)logs)