New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve CRUD on scoring function llm-as-judge #1454

Open

yanxi0830 opened this issue Mar 6, 2025 · 1 comment

Assignees

Labels

Milestone

Contributor

yanxi0830 commented Mar 6, 2025 •

edited

Loading

🚀 Describe the new functionality needed

We need to refactor llm-as-judge to make it easy for user to perform CRUD operations.

unregister scoring functions
persisting judge prompts
migrating simpleqa judge prompts to other repo

💡 Why is this needed? What if we don't build it?

Prepare for llama-stack-evals repo

Other thoughts

No response

yanxi0830 added the enhancement label

yanxi0830 self-assigned this

yanxi0830 added this to the v0.1.7 milestone

Contributor Author

yanxi0830 commented Mar 12, 2025

pending on #1580

yanxi0830 modified the milestones: v0.1.7, v0.1.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

Alternative Proxy