fix: treat usage_metadata.total_token_count of vertex ai as total
#1208
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
There was a problem hiding this comment.
LGTM
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile
There was a problem hiding this comment.
LGTM
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile
usage_metadata.total_token_count of vertex ai as totalusage_metadata.total_token_count of vertex ai as total
There was a problem hiding this comment.
LGTM
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile
|
Thanks for your contribution @gion-pop ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently,
langfuse.langchain.CallbackHandlercounts total token count of Vertex AI response as "Other usage".This PR fixes it.
ref. https://ai.google.dev/gemini-api/docs/tokens?lang=python#count-tokens
Important
Fixes
langfuse.langchain.CallbackHandlerto correctly mapusage_metadata.total_token_counttototalin_parse_usage_model().langfuse.langchain.CallbackHandlerto correctly treatusage_metadata.total_token_countastotalin_parse_usage_model().CallbackHandler.pyto include the mapping fortotal_token_counttototal.This description was created by
for db9ee41. You can customize this summary. It will automatically update as commits are pushed.
Greptile Summary
Disclaimer: Experimental PR review
Fixes token usage tracking for Google Vertex AI by correctly mapping
total_token_countin the Langfuse CallbackHandler, ensuring accurate usage metrics instead of incorrectly categorizing them as 'Other usage'.usage_metadata.total_token_countto the standardizedtotalfield inlangfuse/langchain/CallbackHandler.py💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!