feat: validate presets on template import #18844
Conversation
A few customers and users are already running into preset validation issues and it breaks their workspaces. I'd prefer to get preset validation in for classic templates if we can. |
@SasSwart I just fear a bug in preview could then break existing templates. We had some parsing bugs that prevented some templates from working. |
coderd/parameters_test.go
Outdated
| rejectMu sync.RWMutex | ||
| reject bool | ||
| } | ||
|
|
||
| // SetReject toggles whether GetGitSSHKey should return an error or passthrough to the underlying store. | ||
| func (d *dbRejectGitSSHKey) SetReject(reject bool) { | ||
| d.rejectMu.Lock() | ||
| defer d.rejectMu.Unlock() | ||
| d.reject = reject |
There was a problem hiding this comment.
nit: Is this a fix for a separate flake? If so, might be no harm to separate in its own PR.
WalkthroughThis change introduces backend validation and error handling for dynamic template version presets. It adds new functions for preset diagnostics, updates the control flow for template version creation to validate presets, and extends tests to cover valid and invalid preset scenarios. Dependency updates and minor test refactoring are also included. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant API
participant Preview
participant DB
Client->>API: POST /templateversions (with archive)
API->>API: Extract files from archive
API->>DB: Fetch workspace owner data
API->>Preview: Run preview on extracted files
Preview-->>API: Return preview output, diagnostics
API->>API: CheckPresets(preview output, diagnostics)
alt Preset validation error
API-->>Client: Return error response
else Preset validation ok
API->>API: dynamicTemplateVersionTags(preview output, diagnostics)
API-->>Client: Return tags or success response
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15–20 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Possibly related PRs
Suggested reviewers
Poem
Warning Review ran into problems🔥 Problemsresponse.data.map is not a function ✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
coderd/dynamicparameters/error.go (1)
31-31: Consider updating the error message as suggested in the previous review.The message "Unable to parse presets" should be "Unable to validate presets" to better reflect the function's purpose and align with the validation context.
go.mod (1)
486-486: Verify that the prerequisite PR has been merged.Based on the previous comment, this PR should not be merged until coder/preview#149 has been merged and the dependency updated. Please confirm that this prerequisite has been satisfied.
🧹 Nitpick comments (1)
coderd/templateversions_test.go (1)
645-755: Well-structured preset validation test.The test correctly follows the parallel testing pattern, uses unique identifiers, and covers both valid and invalid preset scenarios appropriately. The structure aligns well with other tests in the file.
[past_review_comments]
Regarding the past comment about checking if no provisioner job was created for the invalid case - this would be a good addition to ensure the validation fails before job creation rather than during job execution.Consider adding a check that no provisioner job is created for the invalid preset case:
} else { require.ErrorContains(t, err, tt.expectError) + // Verify no provisioner job was created for invalid preset + _, err := store.GetProvisionerJobByID(ctx, tv.Job.ID) + require.Error(t, err, "no job should be created for invalid preset") }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (6)
coderd/dynamicparameters/error.go(1 hunks)coderd/dynamicparameters/presets.go(1 hunks)coderd/parameters_test.go(3 hunks)coderd/templateversions.go(3 hunks)coderd/templateversions_test.go(1 hunks)go.mod(1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.go
📄 CodeRabbit Inference Engine (.cursorrules)
**/*.go: The codebase is rigorously linted with golangci-lint to maintain consistent code quality.
Coder emphasizes clear error handling, with specific patterns required: Concise error messages that avoid phrases like "failed to"; Wrapping errors with%wto maintain error chains; Using sentinel errors with the "err" prefix (e.g.,errNotFound).
**/*.go: NEVER use time.Sleep to mitigate timing issues. If an issue seems like it should use time.Sleep, read through https://github.com/coder/quartz and specifically the README to better understand how to handle timing issues.
Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.
Follow Uber Go Style Guide for Go code style.
Use dbauthz.AsSystemRestricted(ctx) for public endpoints needing system access and plain ctx for authenticated endpoints with user context.
Files:
coderd/dynamicparameters/error.gocoderd/dynamicparameters/presets.gocoderd/parameters_test.gocoderd/templateversions_test.gocoderd/templateversions.go
**/*.{go,sql,ts,tsx,js,jsx,md}
📄 CodeRabbit Inference Engine (CLAUDE.md)
Ensure files end with a newline to avoid missing newlines.
Files:
coderd/dynamicparameters/error.gocoderd/dynamicparameters/presets.gocoderd/parameters_test.gocoderd/templateversions_test.gocoderd/templateversions.go
**/*_test.go
📄 CodeRabbit Inference Engine (.cursorrules)
**/*_test.go: All tests must uset.Parallel()to run concurrently, which improves test suite performance and helps identify race conditions.
All tests should run in parallel usingt.Parallel()to ensure efficient testing and expose potential race conditions.Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Files:
coderd/parameters_test.gocoderd/templateversions_test.go
🧠 Learnings (4)
go.mod (2)
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*.go : Coder emphasizes clear error handling, with specific patterns required: Concise error messages that avoid phrases like "failed to"; Wrapping errors with %w to maintain error chains; Using sentinel errors with the "err" prefix (e.g., errNotFound).
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*.go : The codebase is rigorously linted with golangci-lint to maintain consistent code quality.
coderd/parameters_test.go (5)
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.777Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:10.063Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.155Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests must use t.Parallel() to run concurrently, which improves test suite performance and helps identify race conditions.
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests should run in parallel using t.Parallel() to ensure efficient testing and expose potential race conditions.
coderd/templateversions_test.go (7)
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests must use t.Parallel() to run concurrently, which improves test suite performance and helps identify race conditions.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.777Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:10.063Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.155Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderd/coderdtest/**/* : The coderdtest package in coderd/coderdtest/ provides utilities for creating test instances of the Coder server, setting up test users and workspaces, and mocking external components.
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests should run in parallel using t.Parallel() to ensure efficient testing and expose potential race conditions.
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderdenttest/**/* : Enterprise features have dedicated test utilities in the coderdenttest package.
coderd/templateversions.go (4)
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*.go : Coder emphasizes clear error handling, with specific patterns required: Concise error messages that avoid phrases like "failed to"; Wrapping errors with %w to maintain error chains; Using sentinel errors with the "err" prefix (e.g., errNotFound).
Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderd/coderd.go : The REST API is defined in coderd/coderd.go and uses Chi for HTTP routing.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.777Z
Learning: Applies to **/*.go : Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.
Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:10.063Z
Learning: Applies to **/*.go : Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.
🔇 Additional comments (9)
coderd/dynamicparameters/presets.go (1)
13-24: LGTM! Clean and well-structured preset validation logic.The function correctly aggregates preset-specific diagnostics and follows the established error handling patterns in the codebase. The logic is clear and handles the error aggregation appropriately.
coderd/parameters_test.go (4)
6-6: LGTM: Required import for mutex usage.The
syncimport is needed for the new mutex-based error injection control.
419-428: Excellent improvement to test error injection.The addition of mutex-protected error control significantly improves test flexibility by allowing dynamic toggling of error behavior instead of always failing. The thread-safety implementation is correct using RWMutex for efficient concurrent access.
[past_review_comments]
Regarding the past comment about this being a separate flake fix - this change does improve test determinism and could help with flakiness by providing more controlled error injection.
430-440: Clean implementation of conditional error injection.The method correctly uses RLock for reading the shared state and properly delegates to the underlying store when not rejecting. The mutex usage pattern is optimal - releasing the lock before the potentially slow database call.
197-197: Proper usage of the improved error injection mechanism.The test correctly creates the wrapper, passes it to setup, and enables error injection at the appropriate time. This allows initial setup to succeed while still testing error handling in the target scenario.
Also applies to: 205-205, 212-213
coderd/templateversions.go (4)
19-19: LGTM! Import addition is necessary.The
hcl/v2import is required for thehcl.Diagnosticstype used in the refactored function signature.
1586-1637: LGTM! Well-structured file processing and preset validation.The refactoring centralizes file extraction and preview logic effectively. Key strengths:
- Comprehensive error handling for unsupported file types and extraction failures
- Proper preset validation that addresses the PR objective of preventing broken workspace builds
- Clean separation of concerns by moving shared logic to the main handler
- Follows coding guidelines with concise error messages and proper error handling
The preset validation using
dynamicparameters.CheckPresetswill catch invalid presets early and provide clear error messages, which aligns perfectly with the PR objectives.
1642-1642: Function call updated correctly for refactored signature.The call to
dynamicTemplateVersionTagsnow passes the processed preview results instead of raw file data, which is consistent with the refactoring that moved file processing logic to the main handler.
1819-1819: Excellent refactoring that improves separation of concerns.The function signature change eliminates duplicate file processing logic and creates a cleaner separation where the main handler manages file extraction while this function focuses solely on tag validation from preview results. This addresses the concerns raised in past review comments about extracting this logic.
Typos and other errors often result in invalid presets in a template. Coder would import these broken templates and present them to users when they create workspaces. An unsuspecting user who chooses a broken preset would then experience a failed workspace build with no obvious error message.
This PR adds additional validation beyond what is possible in the Terraform provider schema. Coder will now present a more helpful error message to template authors when they upload a new template version:
The frontend warning is less helpful right now, but I'd like to address that in a follow-up since I need frontend help:
closes #17333
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Chores