streamUI with anthropic models #1980

amirm363 · 2024-06-17T06:51:51Z

amirm363
Jun 17, 2024

In the documentation it says that: "Anthropic language models can also be used in the streamText, generateObject, streamObject, and streamUI functions (see AI SDK Core and AI SDK RSC)."

until now I've worked only with OpenAI models with the streamUI function and everything worked flawlessly, I want to try anthropic models now, but it looks like the "model" property in the streamUI options can't use the anthropic provider (even though the docs says it does), this is the error: "Type 'AnthropicMessagesLanguageModel' is not assignable to type 'LanguageModelV1'."

const result = await streamUI({
system,
model: anthropic('claude-3-haiku-20240307'),
maxTokens: 1024,
messages: [...history.get()],
text: ({ content, done }) => {...},
tools: {...}
temperature: 0,
});
return {... };
}

what am I missing here?

lgrammel · 2024-06-17T07:05:40Z

lgrammel
Jun 17, 2024
Maintainer

Have you updated @ai-sdk/anthropic and ai to their latest versions? It could be a version mismatch (older version of ai-sdk/anthropic).

5 replies

amirm363 Jun 17, 2024
Author

Thats exactly what I've done, reran the project and now it works, the problem now that tool calls that worked perfectly fine with openAI sends an Error object to the client, which crushes the application.

Warning: Only plain objects can be passed to Client Components from Server Components. Error objects are not supported.
{error: Error}
^^^^^
Warning: Only plain objects can be passed to Client Components from Server Components. Error objects are not supported.
{name: ..., message: ..., cause: Error, stack: ..., toolName: ..., toolArgs: ...}
^^^^^
⨯ Error: Only plain objects, and a few built-ins, can be passed to Client Components from Server Components. Classes or null prototypes are not supported.
at stringify ()

is there anyway tracing it?

retired-Hurt Jun 18, 2024

I am having the same issue on all of the non openai providers like Google or anthropic or mistral. The vscode compiler gives an error saying below error. I've reinstalled the latest versions just to double check. I suspect this might be something at a framework level and not specific to a particular provider.

Type 'GoogleVertexLanguageModel' is not assignable to type 'LanguageModelV1'.
The types of 'doGenerate(...).then' are incompatible between these types.
Type '<TResult1 = { text?: string | undefined; toolCalls?: import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.10/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FunctionToolCall[] | undefined; ... 5 more ...; logprobs?: import("/Users/code/nextjs-...' is not assignable to type '<TResult1 = { text?: string | undefined; toolCalls?: import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.3/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FunctionToolCall[] | undefined; ... 5 more ...; logprobs?: import("/Users/code/nextjs-c...'.
Types of parameters 'onfulfilled' and 'onfulfilled' are incompatible.
Types of parameters 'value' and 'value' are incompatible.
Type '{ text?: string | undefined; toolCalls?: import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.10/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FunctionToolCall[] | undefined; ... 5 more ...; logprobs?: import("/Users/code/nextjs-chat-play/no...' is not assignable to type '{ text?: string | undefined; toolCalls?: import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.3/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FunctionToolCall[] | undefined; ... 5 more ...; logprobs?: import("/Users/code/nextjs-chat-play/nod...'.
Types of property 'finishReason' are incompatible.
Type 'import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.10/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FinishReason' is not assignable to type 'import("/Users/code/nextjs-chat-play/node_modules/.pnpm/@AI-SDK+provider@0.0.3/node_modules/@ai-sdk/provider/dist/index").LanguageModelV1FinishReason'.
Type '"unknown"' is not assignable to type 'LanguageModelV1FinishReason'.ts(2322)
index.d.ts(467, 5): The expected type comes from property 'model' which is declared here on type 'CallSettings & Prompt & { model: LanguageModelV1; tools?: {} | undefined; text?: RenderText | undefined; initial?: ReactNode; }'

lgrammel Jun 18, 2024
Maintainer

There is a version mismatch there. One of your dependencies is old and uses @ai-sdk/provider v0.0.3, another one is newer and uses v0.0.10

Are you sure you are on the latest versions? the latest version of e.g. the anthropic provider needs v0.0.10: https://github.com/vercel/ai/blob/main/packages/anthropic/package.json

lionstone Jun 26, 2024

i believe i am having the same issue - streamObject works fine with openai but not with anthropic

lgrammel Jun 26, 2024
Maintainer

@lionstone have you updated the anthropic provider and the ai sdk to the latest version? which error are you seeing?

lionstone · 2024-06-26T15:18:01Z

lionstone
Jun 26, 2024

I am using these packages: "@ai-sdk/anthropic": "^0.0.24", "@ai-sdk/openai": "^0.0.33", "ai": "^3.2.10", "zod": "^3.23.8".

Code initiating streaming looks like this:

;(async () => {
    const { partialObjectStream } = await streamObject({
      model: openai('gpt-4o'), // the openai calls work fine but replace with any anthropic model and it breaks
      schema: contentIdeaSchema,
      prompt: prompt,
      maxTokens: 2000,
      maxRetries: 3,
      temperature: 0.3
    })

    for await (const partialObject of partialObjectStream) {
      stream.update(partialObject)
    }

    stream.done()
  })()

It seems to fail silently for a while; eventually I get an error

The streamable UI has been slow to update. This may be a bug or a performance issue or you forgot to call `.done()`.
TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: undefined.
Error message: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "undefined",
    "path": [],
    "message": "Required"
  }
]
    at safeValidateTypes (webpack-internal:///(action-browser)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:224:14)
    at Object.transform (webpack-internal:///(action-browser)/./node_modules/.pnpm/ai@3.2.10_openai@4.52.1_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
    ... 46 lines matching cause stack trace ...
    at Object.transformAlgorithm (node:internal/webstreams/util:185:23)
    at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:526:37) {
  cause: ZodError: [
    {
      "code": "invalid_type",
      "expected": "object",
      "received": "undefined",
      "path": [],
      "message": "Required"
    }
  ]
      at get error (webpack-internal:///(action-browser)/./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:699:31)
      at safeValidateTypes (webpack-internal:///(action-browser)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:226:33)
      at Object.transform (webpack-internal:///(action-browser)/./node_modules/.pnpm/ai@3.2.10_openai@4.52.1_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
      at invokePromiseCallback (node:internal/webstreams/util:180:10)
      at Object.transformAlgorithm (node:internal/webstreams/util:185:23)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:526:37)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:572:10)
      at node:internal/webstreams/transformstream:377:16
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1127:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1242:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1116:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:1006:3)
      at [kChunk] (node:internal/webstreams/readablestream:1585:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2118:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2310:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:507:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:323:5)
      at Object.transform (webpack-internal:///(action-browser)/./node_modules/.pnpm/ai@3.2.10_openai@4.52.1_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.30_typescript@5.5.2__zod@3.23.8/node_modules/ai/dist/index.mjs:902:26)
      at invokePromiseCallback (node:internal/webstreams/util:180:10)
      at Object.transformAlgorithm (node:internal/webstreams/util:185:23)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:526:37)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:572:10)
      at node:internal/webstreams/transformstream:377:16
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1127:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1242:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1116:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:1006:3)
      at [kChunk] (node:internal/webstreams/readablestream:1585:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2118:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2310:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:507:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:323:5)
      at Object.transform (webpack-internal:///(action-browser)/./node_modules/.pnpm/@ai-sdk+anthropic@0.0.24_zod@3.23.8/node_modules/@ai-sdk/anthropic/dist/index.mjs:468:28)
      at invokePromiseCallback (node:internal/webstreams/util:180:10)
      at Object.transformAlgorithm (node:internal/webstreams/util:185:23)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:526:37)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:572:10)
      at node:internal/webstreams/transformstream:377:16
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1127:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1242:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1116:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:1006:3)
      at [kChunk] (node:internal/webstreams/readablestream:1585:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2118:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2310:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:507:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:323:5)
      at Object.transform (webpack-internal:///(action-browser)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:466:22)
      at invokePromiseCallback (node:internal/webstreams/util:180:10)
      at Object.transformAlgorithm (node:internal/webstreams/util:185:23) {
    issues: [ [Object] ],
    addIssue: [Function (anonymous)],
    addIssues: [Function (anonymous)],
    errors: [ [Object] ]
  },
  value: undefined
}```

0 replies

lionstone · 2024-06-26T17:14:52Z

lionstone
Jun 26, 2024

Interesting - it actually appears that when used with await streamUI() that things work fine with anthropic, but not when using await streamObject()

0 replies

lionstone · 2024-06-27T01:57:43Z

lionstone
Jun 27, 2024

any ideas for what might be the issue with streamObject() (only with Anthropic not OpenAI) when streamUI() working fine with Anthropic? Many thx!

0 replies

Kitenite · 2024-10-31T00:22:18Z

Kitenite
Oct 31, 2024

Can confirm this issue still exists. I opened an issue here with more details. Wonder if anyone found a workaround for this.

#3422

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

streamUI with anthropic models #1980

{{title}}

Replies: 5 comments 5 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

streamUI with anthropic models #1980

amirm363 Jun 17, 2024

Replies: 5 comments · 5 replies

lgrammel Jun 17, 2024 Maintainer

amirm363 Jun 17, 2024 Author

retired-Hurt Jun 18, 2024

lgrammel Jun 18, 2024 Maintainer

lionstone Jun 26, 2024

lgrammel Jun 26, 2024 Maintainer

lionstone Jun 26, 2024

lionstone Jun 26, 2024

lionstone Jun 27, 2024

Kitenite Oct 31, 2024

amirm363
Jun 17, 2024

Replies: 5 comments 5 replies

lgrammel
Jun 17, 2024
Maintainer

amirm363 Jun 17, 2024
Author

lgrammel Jun 18, 2024
Maintainer

lgrammel Jun 26, 2024
Maintainer

lionstone
Jun 26, 2024

lionstone
Jun 26, 2024

lionstone
Jun 27, 2024

Kitenite
Oct 31, 2024