APICallError [AI_APICallError]: prompt is too long: 202609 tokens > 200000 maximum #401
Comments
|
Whats the bug here? As written in the first line of the log, your prompt is just to big: Shrink your prompt or choose another model thats has a bigger context-size. |
|
my prompt is very simple "Fix the error", not big. but still getting this message. |
|
Did you try to use another Model? |
|
I tried chaning to Haiuku, sonet and Chatgpt4. I got this error with cloud LLMs. but did not encouter issues local LLMs |
|
I have the same problem. Can you find a solution? |
|
I have the same problem.
|
|
has anyone found a solution? Yet I ask him almost nothing in the prompt, I don't see a solution :-( |
It gives that error because it sends your whole code base along with your 'fix prompt' in every GPT request. I am looking for a good LLM too which supports around 1Million context length for cheap |
|
As I understand, Gemini 1.5 has 2M and Claude 3.5 Sonnet has 200k. What I would like to know is if (in the case of Gemini) if we support context caching so that you don't have to keep sending everything and using up a lot of tokens for fine tuning results? This would drastically reduce the token needs |
Did you find this LLM? Beside Gemini are there other ones that can take this much context length? |
|
first prompt out of the box I get the same error with all models :( bolt, none llms . old :) |
|
i have the same error even with npm install , npm audit fix , npm run dev , it still tells me too many tokens |
|
there is /app/content.ts that has the limits that appear in the error msgs
but upping these values does not seem to alter anything other than the
output error msgs.
…On Mon, Dec 9, 2024 at 7:25 AM MorpheusCompany ***@***.***> wrote:
i have the same error even with npm install , npm audit fix , npm run dev
, it still tells me too many tokens
—
Reply to this email directly, view it on GitHub
<#401 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHCLEVXTNOMUFAB2EGGMFRL2ESTK5AVCNFSM6AAAAABSMFU5SKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRWGM3DIMJYGU>
.
You are receiving this because you commented.Message ID: <coleam00/bolt.
***@***.***>
|
|
i just solved it , with Claude you got a limited amount of token used per minute , that means that no matter the prompt you use it will show you the same error its a bug in the app , when i use claude 3.5 sonnet with the api , after a few prompt it will show the error , i'll wait 2 minutes or so , then prompt again and , the ai runs fine until 3 or 4 more promts , then i'll wait , i think we could find a fix by having multiple api keys instead of one and alternating with the 2 api keys , i think it should solve the problem if anyone knows how to do it , i may do it but i got some work to do |
|
i think u could have an array of api keys and use them so you would not get limited with 8000 tokens /minute |
|
let me know if that helps |
|
I think that the final fix would be to optimize the way context is given. Of course, if you give the full repo in context every time you ask a question, then it’s a HUGE amount of tokens that are being passed to Claude or any other models. On my side, I cannot use bolt.diy because when I import my project, the context is too big as the project is relatively big. |
|
i have the same problem, just typing hello is too big already :) Can we create a context file thats smaller? |
See #1091 and #888Closed for know: If still a problem, open a topic in community please to investigate deeper: https://thinktank.ottomator.ai/c/bolt-diy/bolt-diy-issues-and-troubleshooting/22 |
Describe the bug
APICallError [AI_APICallError]: prompt is too long: 202609 tokens > 200000 maximum
at file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/@AI-SDK+provider-utils@1.0.9_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:405:14
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async postToApi (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/@AI-SDK+provider-utils@1.0.9_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:310:28)
at async AnthropicMessagesLanguageModel.doStream (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/@AI-SDK+anthropic@0.0.39_zod@3.23.8/node_modules/@ai-sdk/anthropic/dist/index.mjs:357:50)
at async fn (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3938:23)
at async file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
at async retryWithExponentialBackoff (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:86:12)
at async startStep (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3903:13)
at async fn (file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:3977:11)
at async file:///C:/Bolt/bolt.new-any-llm/node_modules/.pnpm/ai@3.4.9_react@18.3.1_sswr@2.1.0_svelte@4.2.18__svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:256:22
at async chatAction (C:/Bolt/bolt.new-any-llm/app/routes/api.chat.ts:64:20)
at async Object.callRouteAction (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules@remix-run\server-runtime\dist\data.js:37:16)
at async C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:4612:21
at async callLoaderOrAction (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:4677:16)
at async Promise.all (index 1)
at async callDataStrategyImpl (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:4552:17)
at async callDataStrategy (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:4041:19)
at async submit (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:3900:21)
at async queryImpl (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:3858:22)
at async Object.queryRoute (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+router@1.17.0\node_modules@remix-run\router\dist\router.cjs.js:3827:18)
at async handleResourceRequest (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules@remix-run\server-runtime\dist\server.js:413:20)
at async requestHandler (C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+server-runtime@2.10.0_typescript@5.5.2\node_modules@remix-run\server-runtime\dist\server.js:156:18)
at async C:\Bolt\bolt.new-any-llm\node_modules.pnpm@remix-run+dev@2.10.0@remix-run+react@2.10.2_react-dom@18.3.1_react@18.3.1__react@18.3.1_typ_qwyxqdhnwp3srgtibfrlais3ge\node_modules@remix-run\dev\dist\vite\cloudflare-proxy-plugin.js:70:25 {
cause: undefined,
url: 'https://api.anthropic.com/v1/messages',
requestBodyValues: {
model: 'claude-3-5-haiku-latest',
top_k: undefined,
max_tokens: 8000,
temperature: 0,
top_p: undefined,
stop_sequences: undefined,
system: '\n' +
'You are Bolt, an expert AI assistant and exceptional senior software developer with vast knowledge across multiple programming languages, frameworks, and best practices.\n' +
'\n' +
'<system_constraints>\n' +
" You are operating in an environment called WebContainer, an in-browser Node.js runtime that emulates a Linux system to some degree. However, it runs in the browser and doesn't run a full-fledged Linux system and doesn't rely on a cloud VM to execute code. All code is executed in the browser. It does come with a shell that emulates zsh. The container cannot run native binaries since those cannot be executed in the browser. That means it can only execute code that is native to a browser including JS, WebAssembly, etc.\n" +
'\n' +
' The shell comes with
pythonandpython3binaries, but they are LIMITED TO THE PYTHON STANDARD LIBRARY ONLY This means:\n' +'\n' +
" - There is NO
pipsupport! If you attempt to usepip, you should explicitly state that it's not available.\n" +' - CRITICAL: Third-party libraries cannot be installed or imported.\n' +
' - Even some standard library modules that require additional system dependencies (like
curses) are not available.\n' +' - Only modules from the core Python standard library can be used.\n' +
'\n' +
' Additionally, there is no
g++or any C/C++ compiler available. WebContainer CANNOT run native binaries or compile C/C++ code!\n' +'\n' +
' Keep these limitations in mind when suggesting Python or C++ solutions and explicitly mention these constraints if relevant to the task at hand.\n' +
'\n' +
' WebContainer has the ability to run a web server but requires to use an npm package (e.g., Vite, servor, serve, http-server) or use the Node.js APIs to implement a web server.\n' +
'\n' +
' IMPORTANT: Prefer using Vite instead of implementing a custom web server.\n' +
'\n' +
' IMPORTANT: Git is NOT available.\n' +
'\n' +
" IMPORTANT: Prefer writing Node.js scripts instead of shell scripts. The environment doesn't fully support shell scripts, so use Node.js for scripting tasks whenever possible!\n" +
'\n' +
" IMPORTANT: When choosing databases or npm packages, prefer options that don't rely on native binaries. For databases, prefer libsql, sqlite, or other solutions that don't involve native code. WebContainer CANNOT execute arbitrary native binaries.\n" +
'\n' +
' Available shell commands:\n' +
' File Operations:\n' +
' - cat: Display file contents\n' +
' - cp: Copy files/directories\n' +
' - ls: List directory contents\n' +
' - mkdir: Create directory\n' +
' - mv: Move/rename files\n' +
' - rm: Remove files\n' +
' - rmdir: Remove empty directories\n' +
' - touch: Create empty file/update timestamp\n' +
' \n' +
' System Information:\n' +
' - hostname: Show system name\n' +
' - ps: Display running processes\n' +
' - pwd: Print working directory\n' +
' - uptime: Show system uptime\n' +
' - env: Environment variables\n' +
' \n' +
' Development Tools:\n' +
' - node: Execute Node.js code\n' +
' - python3: Run Python scripts\n' +
' - code: VSCode operations\n' +
' - jq: Process JSON\n' +
' \n' +
' Other Utilities:\n' +
' - curl, head, sort, tail, clear, which, export, chmod, scho, hostname, kill, ln, xxd, alias, false, getconf, true, loadenv, wasm, xdg-open, command, exit, source\n' +
'</system_constraints>\n' +
'\n' +
'<code_formatting_info>\n' +
' Use 2 spaces for code indentation\n' +
'</code_formatting_info>\n' +
'\n' +
'<message_formatting_info>\n' +
' You can make the output pretty by using only the following available HTML elements: , ,
The text was updated successfully, but these errors were encountered: