-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat (core): add extractReasoningMiddleware (#4541)
- Loading branch information
Showing
11 changed files
with
616 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
'ai': patch | ||
--- | ||
|
||
feat (core): add extractReasoningMiddleware |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
content/docs/07-reference/01-ai-sdk-core/66-extract-reasoning-middleware.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
--- | ||
title: extractReasoningMiddleware | ||
description: Middleware that extracts XML-tagged reasoning sections from generated text | ||
--- | ||
|
||
# `extractReasoningMiddleware()` | ||
|
||
`extractReasoningMiddleware` is a middleware function that extracts XML-tagged reasoning sections from generated text and exposes them separately from the main text content. This is particularly useful when you want to separate an AI model's reasoning process from its final output. | ||
|
||
```ts | ||
import { extractReasoningMiddleware } from 'ai'; | ||
|
||
const middleware = extractReasoningMiddleware({ | ||
tagName: 'reasoning', | ||
separator: '\n', | ||
}); | ||
``` | ||
|
||
## Import | ||
|
||
<Snippet | ||
text={`import { extractReasoningMiddleware } from "ai"`} | ||
prompt={false} | ||
/> | ||
|
||
## API Signature | ||
|
||
### Parameters | ||
|
||
<PropertiesTable | ||
content={[ | ||
{ | ||
name: 'tagName', | ||
type: 'string', | ||
isOptional: false, | ||
description: | ||
'The name of the XML tag to extract reasoning from (without angle brackets)', | ||
}, | ||
{ | ||
name: 'separator', | ||
type: 'string', | ||
isOptional: true, | ||
description: | ||
'The separator to use between reasoning and text sections. Defaults to "\\n"', | ||
}, | ||
]} | ||
/> | ||
|
||
### Returns | ||
|
||
Returns a middleware object that: | ||
|
||
- Processes both streaming and non-streaming responses | ||
- Extracts content between specified XML tags as reasoning | ||
- Removes the XML tags and reasoning from the main text | ||
- Adds a `reasoning` property to the result containing the extracted content | ||
- Maintains proper separation between text sections using the specified separator | ||
|
||
### Type Parameters | ||
|
||
The middleware works with the `LanguageModelV1StreamPart` type for streaming responses. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
45 changes: 45 additions & 0 deletions
45
examples/ai-core/src/stream-text/groq-reasoning-fullstream.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
import { groq } from '@ai-sdk/groq'; | ||
import { | ||
experimental_wrapLanguageModel, | ||
extractReasoningMiddleware, | ||
streamText, | ||
} from 'ai'; | ||
import 'dotenv/config'; | ||
|
||
async function main() { | ||
const result = streamText({ | ||
model: experimental_wrapLanguageModel({ | ||
model: groq('deepseek-r1-distill-llama-70b'), | ||
middleware: extractReasoningMiddleware({ tagName: 'think' }), | ||
}), | ||
prompt: 'Invent a new holiday and describe its traditions.', | ||
}); | ||
|
||
let enteredReasoning = false; | ||
let enteredText = false; | ||
for await (const part of result.fullStream) { | ||
if (part.type === 'reasoning') { | ||
if (!enteredReasoning) { | ||
enteredReasoning = true; | ||
console.log('\nSTREAMING REASONING:\n'); | ||
} | ||
process.stdout.write(part.textDelta); | ||
} else if (part.type === 'text-delta') { | ||
if (!enteredText) { | ||
enteredText = true; | ||
console.log('\nSTREAMING TEXT:\n'); | ||
} | ||
process.stdout.write(part.textDelta); | ||
} | ||
} | ||
|
||
console.log(); | ||
console.log('\nFINAL REASONING:\n', await result.reasoning); | ||
console.log('\nFINAL TEXT:\n', await result.text); | ||
|
||
console.log(); | ||
console.log('Token usage:', await result.usage); | ||
console.log('Finish reason:', await result.finishReason); | ||
} | ||
|
||
main().catch(console.error); |
Oops, something went wrong.