[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openai[minor]: OpenAI - Ability to set stream_options and streamUsage when instantiating ChatOpenAI class. #5761

Merged
merged 21 commits into from
Jun 21, 2024

Conversation

deranga
Copy link
Contributor
@deranga deranga commented Jun 14, 2024

Fixes 965 - Where its not possible to get the usage back from OpenAI when streaming and using chains or agents.

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jun 14, 2024
Copy link
vercel bot commented Jun 14, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 21, 2024 9:40pm
langchainjs-docs ✅ Ready (Inspect) Visit Preview Jun 21, 2024 9:40pm

Copy link
Collaborator
@bracesproul bracesproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey, thanks for taking initiative on this! We're actually implementing a new streamUsage constructor arg & call option, but haven't gotten around to implementing it for OpenAI yet. If you wanted to implement it, here's a PR you can copy from. If not, feel free to close this PR.

Please ping me if you have any questions on it!

@bracesproul
Copy link
Collaborator

Actually, lets include both streamOptions: { includeUsage: boolean } (prefer camel case for our API) and streamUsage: boolean. OAI might add more fields to their stream_options API in the future, in which case we'd like a way to support those too!

@vercel vercel bot temporarily deployed to Preview – langchainjs-docs June 14, 2024 17:58 Inactive
@vercel vercel bot temporarily deployed to Preview – langchainjs-api-refs June 14, 2024 17:58 Inactive
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jun 14, 2024
@deranga
Copy link
Contributor Author
deranga commented Jun 14, 2024

@bracesproul, I think I've made the changes you are after.

@jacoblee93 jacoblee93 changed the title OpenAI - Ability to set stream_options and streamUsage when instantiating ChatOpenAI class. openai[minor]: OpenAI - Ability to set stream_options and streamUsage when instantiating ChatOpenAI class. Jun 20, 2024
@jacoblee93
Copy link
Collaborator

Getting a few 400s when running integration tests:

400 The 'stream_options' parameter is only allowed when 'stream' is enabled.

Seems like we may need to be a bit more selective on how we populate this?

@deranga
Copy link
Contributor Author
deranga commented Jun 20, 2024

Is it possible to run the integration tests outside of being kicked off by a maintainer?

To address the 400 for stream: false when requesting usage data I've updated so that either stream_options: {include_usage:true}/ streamUsage true will force the stream flag to be set to true, and if not fallback to this.streaming. To make it BC I've altered streamUsage to default to false.

It may make more sense to just throw the error back the user of the library, stating that you can't request stream usage info with the stream flag set to false?

Let me know if there's any other changes you'd like to see before its ready to merge

Thanks

@jacoblee93
Copy link
Collaborator

Changed to default to true, avoid passing as an invocation param if this.streaming is false.

@jacoblee93
Copy link
Collaborator

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature close PRs that need one or two touch-ups to be ready size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to get token count, callbacks work but just for ChatOpenAI but not for RetrievalQAChain
3 participants