[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add usage in stream response #1876

Merged
merged 2 commits into from
Jul 2, 2024
Merged

Add usage in stream response #1876

merged 2 commits into from
Jul 2, 2024

Conversation

fbzhong
Copy link
Contributor
@fbzhong fbzhong commented Jun 27, 2024

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily receiving feedbacks. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Return the usage info in stream response.

Modification

  1. add usage: Optional[UsageInfo] in ChatCompletionStreamResponse
  2. return usage in completion_stream_generator while res.finish_reason is not None

BC-breaking (Optional)

No.

Use cases (Optional)

No.

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@lvhan028 lvhan028 requested a review from AllentDan July 1, 2024 07:56
@lvhan028 lvhan028 self-requested a review July 1, 2024 07:57
@lvhan028
Copy link
Collaborator
lvhan028 commented Jul 1, 2024

Hi, @fbzhong
Could you fix the linting error? The following commands can help resolving it.

pip install pre-commit
cd lmdeploy
pre-commit install
pre-commit run --all-files

@fbzhong
Copy link
Contributor Author
fbzhong commented Jul 1, 2024

Hi @lvhan028
Linting errors are fixed, please check it. Thanks.

Copy link
Collaborator
@lvhan028 lvhan028 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lvhan028 lvhan028 merged commit 57af803 into InternLM:main Jul 2, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants