You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to make use of the extensive plugin offering to try various models available with ease and without unnecessary overhead. Latency is everything for UIs, and CLIs are generally great because they're fast and responsive, no memory and cpu hungry custom rendering to support text entry -- .5 s is noticable, 1.3 is checking my pulse, 10s is why am I using this tool. Could plugins be architecture to work in still more minimal ways until needed? Declare more of their offerings up front, defer initialization, or cache state, perhaps with modified time cache keys.
For subcommands that don't rely on loading accessory plugins, llm logs -t -n1, setting LLM_LOAD_PLUGINS='' speeds up access from 6.2s to 0.9s.
The text was updated successfully, but these errors were encountered:
I would like to make use of the extensive plugin offering to try various models available with ease and without unnecessary overhead. Latency is everything for UIs, and CLIs are generally great because they're fast and responsive, no memory and cpu hungry custom rendering to support text entry -- .5 s is noticable, 1.3 is checking my pulse, 10s is why am I using this tool. Could plugins be architecture to work in still more minimal ways until needed? Declare more of their offerings up front, defer initialization, or cache state, perhaps with modified time cache keys.
For subcommands that don't rely on loading accessory plugins,
llm logs -t -n1
, settingLLM_LOAD_PLUGINS=''
speeds up access from 6.2s to 0.9s.The text was updated successfully, but these errors were encountered: