Add flake app to run openai proxy

[ParetoOptimalDev]

Works with:

nix run .#llama-server-openai-proxy

If merged, then you can run both the server and proxy via two commands:

nix run github:ggerganov/llama.cpp#llama-server
nix run github:ggerganov/llama.cpp#llama-server-openai-proxy

Note: This is likely not ideal or the best way to do this, but it works and can hopefully spark useful discussion to get this feature into the llama.cpp flake 😃

[SomeoneSerge]

This could've been a makeWrapper. I don't know if api_like_OAI.py takes any command line arguments, but makeWrapper would have passed those on

[SomeoneSerge]

Can't suggest anything concrete right now, but it might be a good thing to do some form of buildPythonApplication because its probably less affected by PYTHONPATH contamination.

[SomeoneSerge]

I was going to mention this in #4605, but we wouldn't have had to manually wrap these in apps if we instead exposed them (in the overlay and) in packages/legacyPackages, and if we ensured that meta.mainPrograms are set correctly (with writeShellScriptBin they are)

[SomeoneSerge]

You could now add this to scope.nix:

llama.cpp/.devops/nix/scope.nix

Lines 7 to 12 in 4e1d251

	lib.makeScope newScope (
	self: {
	inherit llamaVersion;
	llama-cpp = self.callPackage ./package.nix { };
	}
	)

This way the attribute would show up as llamaPackages.llama-server-openai-proxy in nixpkgs with the flake's overlays.default applied, as well as in the flake's legacyPackages.${system}.llamaPackages.

[ParetoOptimalDev]

I think this might not be needed since the default llama-server started on port 8080 seems to be openai compatible.

[ggerganov]

I think this might not be needed since the default llama-server started on port 8080 seems to be openai compatible.

Yes, I'm thinking we should probably deprecate examples/server/api_like_OAI.py and improve the built-in server support for the OpenAI API (#4216)