feat: record inference stats

[bcho]

This pull request added a simple struct InferenceStats for capturing the model evaluation stats. This struct can generate a similar output like llama.cpp . Closes #46

Example output:

feed_prompt_duration: 1615ms
prompt_tokens: 10
predict_duration: 6917ms
predict_tokens: 30
per_token_duration: 230.567ms

[setzer22]

Thanks a lot for sharing! 😄 It was always bugging me that I couldn't see the times per token with this.

At first I thought maybe we needed to integrate this into the low-level functions feed_prompt and predict_next_token, but on second thought I prefer this version because it's simple, and if users are building a lower level loop using the other functions, they can do their own time measurements too, so there's no point in overengineering things.

Just have a very minor nitpick about newlines 🤔 but overall LGTM

[setzer22]

Minor nit: Wouldn't it make more sense to print these in separate lines?

What do others think?

[bcho]

np, updated, PTAL

[setzer22]

Thanks again! We can merge once CI passes :)