Control Vector UI (In progress)

[revolvedai]

Hello, I'm mindlessly smashing away at a gradio UI for chatting with LLMs using control vectors as well as for training control vectors. I believe this repo uses a locally downloaded model in full to train the control vector. Is it possible to do so using llama.cpp server and a gguf? Just trying to make something a bit lighter weight and easier to use with a server.

Based on this discussion ggerganov/llama.cpp#8724 it seems like this is not complete for Llama.cpp. Should I just wait until there is some progress? Or maybe stick with making control vectors using the methods proposed in the repeng repo?

I'm not skilled at all in python - mostly piecing this together through cursor, claude and raw determination.

Any guidance would be appreciated! My goal is simply to increase adoption of control vectors. They are so powerful and meet many needs people have in the space but hardly anyone knows about them!

Thank you