support for Open-Assistant.io/gpt4all

[vasnt]

instead of using openai API, can it work with other open-source/local hosted models like llama etc.?

[unconv]

Currently not, because it uses the new ChatGPT API function calling. I will try to "backport" it to be able to use the older models without the function calling, and then you should be able to switch to using a different model. In theory.

[sarfraznawaz2005]

would be great if you add OPENAI_API_BASE to config file so we can use this with local modals. Thanks

[unconv]

@sarfraznawaz2005 Do any local models support function calling?

[sarfraznawaz2005]

Not sure what you mean but there are local models that support openai api format such as https://github.com/josStorer/RWKV-Runner or https://lmstudio.ai. They expose local url and many similar projects now support them by exposing option for base url in which case api key is not taken into account.

autopilot strictly checks for api key otherwise gives errror. If you allow option of base url and if it is local then you can modify code to skip checking for api key.

[unconv]

GPT-AutoPilot uses the OpenAI API's function calling feature which is not implemented in other LLM's than ChatGPT and Gemini as far as I know.

You can set the OPENAI_API_KEY environment variable to whatever and set OPENAI_BASE_URL to the URL of the local LLM to try it, but probably the endpoints do not support function calling.

I'm working on another project in TypeScript that does not use the function calling feature, and I might integrate it into this project as well, if I get it to work.

[sarfraznawaz2005]

yes I tried that way it does not work. Looking forward to other project :)