feat: Adding ability to attach 'annotations' (custom metadata) to messages #879
Conversation
adding onEvent callback to ai-stream fixing streaming part typo removing onevent adding optional event param to AIStreamParser passing the extra arg in the transformer for event type
|
@nick-inkeep this looks great. To help me understand it better, could you provide an example of how to use it, or ideally add it to the Updates to the documentation and a test case for the full streaming (e.g. add to |
|
@lgrammel few things as discussed:
As an example, I'm not sure how to construe an example with OpenAI, thought it might be best to illustrate the use with the corresponding PR for the Inkeep provider. See:
Screen recording of the citations in use in that example. |
|
I see the annotations array repeats the same annotation 100s of times... I created an issue around that |
Proposed fix here. |
Scenario
Many developers would like to add domain or provider-specific information to a given message. For example, for Inkeep we stream information about the sources used to answer a question at the end of the event stream. This might be used by a client to, for example, show a "Sources" section at the end of a message in the UI:
Dependency for:
#861 (adding Inkeep as a provider)
Resolves:
#585
#425 (comment)
Implementation
appendMessageAnnotation(n)function todata(experimental_StreamData) objecteventtype to a CustomParser so streams can handle non-homogeneous server-side event. For example, so that the Inkeep stream provider can parse theonRecordsCitedevent type that's unique to it's SSE and has the information to append to a message.message.annotations[n]