another server, based on stream

[colinator]

Looks like there are several attempts to make servers with this code - here is another! This approach doesn't actually make a server - it simply factorizes the 'stream' example into components. This would support many ways of creating servers and data encodings - json/http, protobuf/grpc, flexbuffers, message pack, etc.

Happy to make changes and/or fold in with another server-ization branch.

[ggerganov]

Interesting! Will look further, but first need to make some long-time pending updates (#1422) and after that will come back to this example

[litongjava]

I like it, but so far it's only done with modularization, it would be nice to write another websocker-server. I would like to do this work, do you have plans to add a web-socket-server?

Also I found out that recording audio with a sample rate of 48KHz and then converting it to 16KHz gives better recognition. I would also like to add support for

[colinator]

@litongjava Well, if this approach gets integrated, then yes please write a web-socket server. I actually want something different - a ROS-like pub-sub node. We can all have the server we want, with the encoding we want.

[codesoda]

It'd be great if the server didn't "have to" capture the audio. Then, this stream transcription approach can be incorporated into other apps with real-time audio/video data streams. The host app would convert to 16khz audio and keep throwing it to the stream server.

[colinator]

@codesoda Yes, me too. Hence the LocalSDLMicrophone class separation. I also want the final result to be encoded into some protocol other than json - hence the WhisperOutput/WhisperEncoder classes.

[litongjava]

I have completed the WebSocket service, as it was quite complex, so I created a separate project for it. The project can be found at https://github.com/litongjava/whisper-cpp-server.

After compiling, run the following command to start the server:

./cmake-build-debug/whisper_server_base_on_uwebsockets -m models/ggml-base.en.bin

Then, navigate to the web directory at https://github.com/litongjava/whisper-cpp-server/tree/main/web and open the index.html file.

However, my tests indicate that the results of the speech recognition are not very satisfactory