send customized data in handleSubmit and set Message annotations

[rockywangxiaolei]

Custom Attachments Feature for Chat Endpoint

Objective

Attempting to implement a self-defined attachments feature by sending customized data in handleSubmit and appending it into messages at the backend chat endpoint.

Code

Frontend: Submitting Customized Data

handleSubmit(undefined, {
// experimental_attachments: attachments,
body: {
customData: {
other: 'data test',
}
}
});
Backend: Handling Data in POST Request

export async function POST(req: Request) {
const { conversationID, messages, customData }: {
conversationID: string;
messages: Array;
customData: JSONValue
} = await req.json();

// Initialize StreamData object
const data = new StreamData();

// Append initial data to StreamData
data.append({ test: 'initialized calls' });

// Attempt to annotate the message with customData
// Currently, this sets customData in message role='assistant' rather than role='user'
data.appendMessageAnnotation(customData);

// Convert messages to core format
const coreMessages = convertToCoreMessages(messages);

const result = await streamText({
model: customModel,
system: "you are a friendly assistant! Keep your responses concise and helpful.",
messages: coreMessages,
maxSteps: 5,
experimental_telemetry: {
isEnabled: true,
functionId: "stream-text",
},
onFinish() {
data.appendMessageAnnotation({
other: 'information',
});
data.append('call completed');
// Close the StreamData object
data.close();
},
});

return result.toDataStreamResponse({ data });
}
Question
It seems that using StreamData and data.appendMessageAnnotation always places customData in message(role='assistant'). Is there a way to set customData in message(role='user') instead?

Additional Inquiry
For experimental_attachments, could there be an option to specify whether attachments should be automatically processed by the AI provider? Since not all AI providers (e.g., Google) support file input, having this option could enable us to dynamically route requests to compatible providers.