Request too large for gpt-4o

[soungsid]

Describe the bug

Description

I am experiencing issues with bolt.diy when working on larger projects. As soon as my project starts to grow, bolt.diy becomes nearly unusable. Specifically:

1. For small projects (e.g., Astro or Qwik with fewer than 10 files), the tool works fine.
2. However, when the project size increases (e.g., Angular projects with more than 10 files), I encounter this error: AI_RetryError: Failed after 3 attempts. Last error: Request too large for gpt-4o in organization org-XXXX on tokens per min (TPM): Limit 30000, Requested 60847. The input or output tokens must be reduced in order to run successfully.

Thank you for your support! 😊

Link to the Bolt URL that caused the error

https://bolt.diy/

Steps to reproduce

1. Create a new Angular project or a large project with more than 20 files.
2. Attempt to load the project in bolt.diy and interact with it.
3. Observe that the error occurs almost immediately.

Expected behavior

Expected Behavior

• The tool should respect the MAX_TOKENS limit (8000 in this case) to avoid exceeding the token-per-minute limit for GPT-4o.
• Large projects should either process successfully or provide feedback to the user about token limits before attempting the operation.

Screen Recording / Screenshot

No response

Platform

Environment Details

• Library/Tool Version: Bolt.diy (version, if available)
• LLM Version: GPT-4o
• System: (e.g., Windows 11, Oracle linux)

Provider Used

open ai

Model Used

gpt-4o

Additional context

The error suggests that the request exceeds the token-per-minute (TPM) limit for GPT-4o.

---

Observed Issue

• Despite the presence of the constant export const MAX_TOKENS = 8000, it seems this limit is not being respected.
• This results in the tool attempting to process large requests that exceed both token-per-minute and max token limits.

---

Additional Information

• Suggested solutions:
  1. Ensure that the MAX_TOKENS constant is properly enforced.
  2. Provide a mechanism to gracefully handle larger requests by splitting them into smaller chunks.
  3. Improve feedback to the user about token or size limitations before processing large projects.

[leex279]

Hi @soungsid,

its a known problem and will be fixed with upcoming features like context optimization #1091.
Either you can checkout the PR and use Google Gemini 2.0 Flash which has 2M context size (and is free)

[alexja14]

#1088 did a mini fix to it if u need other models ftm till fix. cheers to leex