Values from flags should override values in prompt yaml files. #49
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This PR ensures that CLI flag values take precedence over values defined in prompt YAML files.
- Reorders flag parsing so that flags override YAML defaults
- Adds an end-to-end test confirming flags override prompt.yaml parameters
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| cmd/run/run.go | Moves PopulateFromFlags below YAML-based assignments so flags override |
| cmd/run/run_test.go | Adds a test verifying CLI flags override model and parameter values |
Comments suppressed due to low confidence (1)
cmd/run/run_test.go:246
- [nitpick] The test uses a
*.prompt.yamlextension, but the CLI help refers to.prompt.yml—consider unifying the extension in docs and tests for consistency.
tmp, err := os.CreateTemp(t.TempDir(), "*.prompt.yaml")
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Very small follow up from #48:
Why?
Suppose I have a template in a prompt.yaml that defines model parameters (or even model name). If I want to iterate on the temperature, or even which model i want to use then it's easier to do so by repeatedly running the cmd and tweaking the flag value.
Basically, a cli flag has a higher order of "specificity" than the value in the yaml file. I'm more intentional, it is surprising that the cli flag value might be silently ignored.
Bonus observation
About 80% of this test was written via the following prompt: