-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add natural pattern for delineating between streaming and sync LLM output #29
Conversation
@@ -2,8 +2,8 @@ | |||
"name": "gensx", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where did the changes in here come from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to remove tsup
- more description on that in the PR description but in short I ran into an obtuse error that has several open bug reports. Looking through the repo, it appears to be abandoned.
@@ -1,9 +1,22 @@ | |||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe all of this is specified in tsconfig.json
. Why remove the extends and duplicate it?
The reason for the duplication is so that the playground code works when doing development, but does not end up in the published bundle.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was having trouble with the tsup and had to migrate over to plain typescript, and had trouble getting both tsconfig files to work with our publish target.
I think we're going to refactor the way that all of this works. examples need to be in a separate project/projects and we should probably just link in the gensx framework to that package.json -- then we don't need to worry about any of this complexity.
closing in favor of #36 |
LLM components need to support both (1) synchronous output -- the 99% use case -- and (2) streaming output -- the 1% output that is typically only used during the last step when output is being rendered to users via a UI.
We want to optimize for sync output since this is the common use case. To support this we've added:
gensx.StreamComponent
: a special kind of component that returns a standard formal with a plain text value and a stream to get the output<Stream>
component: wrap anygensx.StreamComponent
to coerce it to return a stream instead of awaiting the text output.This enables writing any component to either return a text value or a stream. Authors don't have to worry about supporting both formats. Consumers get the prompt value in the 99% use case or can wrap the component in
<Stream>
and get the expected behavior for the 1% use case.Removing
tsup
I ran into some cryptic errors:
So I removed tsup in favor of just using tsc (and removing cjs support - which seems fine). I found several related issues from September or later. Last commit in tsup was two months ago. Looks like the package isn't well maintained anymore.