Skip to content

Commit 90b9078

Browse files
SasSwartSasSwart
committed
Merge remote-tracking branch 'origin/main' into jjs/prebuilds-user-group-membership
2 parents 4b63826 + 78af5e0 commit 90b9078

File tree

1,033 files changed

+56246
-34201
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,033 files changed

+56246
-34201
lines changed

‎.claude/docs/DATABASE.md

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
# Database Development Patterns
2+
3+
## Database Work Overview
4+
5+
### Database Generation Process
6+
7+
1. Modify SQL files in `coderd/database/queries/`
8+
2. Run `make gen`
9+
3. If errors about audit table, update `enterprise/audit/table.go`
10+
4. Run `make gen` again
11+
5. Run `make lint` to catch any remaining issues
12+
13+
## Migration Guidelines
14+
15+
### Creating Migration Files
16+
17+
**Location**: `coderd/database/migrations/`
18+
**Format**: `{number}_{description}.{up|down}.sql`
19+
20+
- Number must be unique and sequential
21+
- Always include both up and down migrations
22+
23+
### Helper Scripts
24+
25+
| Script | Purpose |
26+
|--------|---------|
27+
| `./coderd/database/migrations/create_migration.sh "migration name"` | Creates new migration files |
28+
| `./coderd/database/migrations/fix_migration_numbers.sh` | Renumbers migrations to avoid conflicts |
29+
| `./coderd/database/migrations/create_fixture.sh "fixture name"` | Creates test fixtures for migrations |
30+
31+
### Database Query Organization
32+
33+
- **MUST DO**: Any changes to database - adding queries, modifying queries should be done in the `coderd/database/queries/*.sql` files
34+
- **MUST DO**: Queries are grouped in files relating to context - e.g. `prebuilds.sql`, `users.sql`, `oauth2.sql`
35+
- After making changes to any `coderd/database/queries/*.sql` files you must run `make gen` to generate respective ORM changes
36+
37+
## Handling Nullable Fields
38+
39+
Use `sql.NullString`, `sql.NullBool`, etc. for optional database fields:
40+
41+
```go
42+
CodeChallenge: sql.NullString{
43+
String: params.codeChallenge,
44+
Valid: params.codeChallenge != "",
45+
}
46+
```
47+
48+
Set `.Valid = true` when providing values.
49+
50+
## Audit Table Updates
51+
52+
If adding fields to auditable types:
53+
54+
1. Update `enterprise/audit/table.go`
55+
2. Add each new field with appropriate action:
56+
- `ActionTrack`: Field should be tracked in audit logs
57+
- `ActionIgnore`: Field should be ignored in audit logs
58+
- `ActionSecret`: Field contains sensitive data
59+
3. Run `make gen` to verify no audit errors
60+
61+
## Database Architecture
62+
63+
### Core Components
64+
65+
- **PostgreSQL 13+** recommended for production
66+
- **Migrations** managed with `migrate`
67+
- **Database authorization** through `dbauthz` package
68+
69+
### Authorization Patterns
70+
71+
```go
72+
// Public endpoints needing system access (OAuth2 registration)
73+
app, err := api.Database.GetOAuth2ProviderAppByClientID(dbauthz.AsSystemRestricted(ctx), clientID)
74+
75+
// Authenticated endpoints with user context
76+
app, err := api.Database.GetOAuth2ProviderAppByClientID(ctx, clientID)
77+
78+
// System operations in middleware
79+
roles, err := db.GetAuthorizationUserRoles(dbauthz.AsSystemRestricted(ctx), userID)
80+
```
81+
82+
## Common Database Issues
83+
84+
### Migration Issues
85+
86+
1. **Migration conflicts**: Use `fix_migration_numbers.sh` to renumber
87+
2. **Missing down migration**: Always create both up and down files
88+
3. **Schema inconsistencies**: Verify against existing schema
89+
90+
### Field Handling Issues
91+
92+
1. **Nullable field errors**: Use `sql.Null*` types consistently
93+
2. **Missing audit entries**: Update `enterprise/audit/table.go`
94+
95+
### Query Issues
96+
97+
1. **Query organization**: Group related queries in appropriate files
98+
2. **Generated code errors**: Run `make gen` after query changes
99+
3. **Performance issues**: Add appropriate indexes in migrations
100+
101+
## Database Testing
102+
103+
### Test Database Setup
104+
105+
```go
106+
func TestDatabaseFunction(t *testing.T) {
107+
db := dbtestutil.NewDB(t)
108+
109+
// Test with real database
110+
result, err := db.GetSomething(ctx, param)
111+
require.NoError(t, err)
112+
require.Equal(t, expected, result)
113+
}
114+
```
115+
116+
## Best Practices
117+
118+
### Schema Design
119+
120+
1. **Use appropriate data types**: VARCHAR for strings, TIMESTAMP for times
121+
2. **Add constraints**: NOT NULL, UNIQUE, FOREIGN KEY as appropriate
122+
3. **Create indexes**: For frequently queried columns
123+
4. **Consider performance**: Normalize appropriately but avoid over-normalization
124+
125+
### Query Writing
126+
127+
1. **Use parameterized queries**: Prevent SQL injection
128+
2. **Handle errors appropriately**: Check for specific error types
129+
3. **Use transactions**: For related operations that must succeed together
130+
4. **Optimize queries**: Use EXPLAIN to understand query performance
131+
132+
### Migration Writing
133+
134+
1. **Make migrations reversible**: Always include down migration
135+
2. **Test migrations**: On copy of production data if possible
136+
3. **Keep migrations small**: One logical change per migration
137+
4. **Document complex changes**: Add comments explaining rationale
138+
139+
## Advanced Patterns
140+
141+
### Complex Queries
142+
143+
```sql
144+
-- Example: Complex join with aggregation
145+
SELECT
146+
u.id,
147+
u.username,
148+
COUNT(w.id) as workspace_count
149+
FROM users u
150+
LEFT JOIN workspaces w ON u.id = w.owner_id
151+
WHERE u.created_at > $1
152+
GROUP BY u.id, u.username
153+
ORDER BY workspace_count DESC;
154+
```
155+
156+
### Conditional Queries
157+
158+
```sql
159+
-- Example: Dynamic filtering
160+
SELECT * FROM oauth2_provider_apps
161+
WHERE
162+
($1::text IS NULL OR name ILIKE '%' || $1 || '%')
163+
AND ($2::uuid IS NULL OR organization_id = $2)
164+
ORDER BY created_at DESC;
165+
```
166+
167+
### Audit Patterns
168+
169+
```go
170+
// Example: Auditable database operation
171+
func (q *sqlQuerier) UpdateUser(ctx context.Context, arg UpdateUserParams) (User, error) {
172+
// Implementation here
173+
174+
// Audit the change
175+
if auditor := audit.FromContext(ctx); auditor != nil {
176+
auditor.Record(audit.UserUpdate{
177+
UserID: arg.ID,
178+
Old: oldUser,
179+
New: newUser,
180+
})
181+
}
182+
183+
return newUser, nil
184+
}
185+
```
186+
187+
## Debugging Database Issues
188+
189+
### Common Debug Commands
190+
191+
```bash
192+
# Check database connection
193+
make test-postgres
194+
195+
# Run specific database tests
196+
go test ./coderd/database/... -run TestSpecificFunction
197+
198+
# Check query generation
199+
make gen
200+
201+
# Verify audit table
202+
make lint
203+
```
204+
205+
### Debug Techniques
206+
207+
1. **Enable query logging**: Set appropriate log levels
208+
2. **Use database tools**: pgAdmin, psql for direct inspection
209+
3. **Check constraints**: UNIQUE, FOREIGN KEY violations
210+
4. **Analyze performance**: Use EXPLAIN ANALYZE for slow queries
211+
212+
### Troubleshooting Checklist
213+
214+
- [ ] Migration files exist (both up and down)
215+
- [ ] `make gen` run after query changes
216+
- [ ] Audit table updated for new fields
217+
- [ ] In-memory database implementations updated
218+
- [ ] Nullable fields use `sql.Null*` types
219+
- [ ] Authorization context appropriate for endpoint type

‎.claude/docs/OAUTH2.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# OAuth2 Development Guide
2+
3+
## RFC Compliance Development
4+
5+
### Implementing Standard Protocols
6+
7+
When implementing standard protocols (OAuth2, OpenID Connect, etc.):
8+
9+
1. **Fetch and Analyze Official RFCs**:
10+
- Always read the actual RFC specifications before implementation
11+
- Use WebFetch tool to get current RFC content for compliance verification
12+
- Document RFC requirements in code comments
13+
14+
2. **Default Values Matter**:
15+
- Pay close attention to RFC-specified default values
16+
- Example: RFC 7591 specifies `client_secret_basic` as default, not `client_secret_post`
17+
- Ensure consistency between database migrations and application code
18+
19+
3. **Security Requirements**:
20+
- Follow RFC security considerations precisely
21+
- Example: RFC 7592 prohibits returning registration access tokens in GET responses
22+
- Implement proper error responses per protocol specifications
23+
24+
4. **Validation Compliance**:
25+
- Implement comprehensive validation per RFC requirements
26+
- Support protocol-specific features (e.g., custom schemes for native OAuth2 apps)
27+
- Test edge cases defined in specifications
28+
29+
## OAuth2 Provider Implementation
30+
31+
### OAuth2 Spec Compliance
32+
33+
1. **Follow RFC 6749 for token responses**
34+
- Use `expires_in` (seconds) not `expiry` (timestamp) in token responses
35+
- Return proper OAuth2 error format: `{"error": "code", "error_description": "details"}`
36+
37+
2. **Error Response Format**
38+
- Create OAuth2-compliant error responses for token endpoint
39+
- Use standard error codes: `invalid_client`, `invalid_grant`, `invalid_request`
40+
- Avoid generic error responses for OAuth2 endpoints
41+
42+
### PKCE Implementation
43+
44+
- Support both with and without PKCE for backward compatibility
45+
- Use S256 method for code challenge
46+
- Properly validate code_verifier against stored code_challenge
47+
48+
### UI Authorization Flow
49+
50+
- Use POST requests for consent, not GET with links
51+
- Avoid dependency on referer headers for security decisions
52+
- Support proper state parameter validation
53+
54+
### RFC 8707 Resource Indicators
55+
56+
- Store resource parameters in database for server-side validation (opaque tokens)
57+
- Validate resource consistency between authorization and token requests
58+
- Support audience validation in refresh token flows
59+
- Resource parameter is optional but must be consistent when provided
60+
61+
## OAuth2 Error Handling Pattern
62+
63+
```go
64+
// Define specific OAuth2 errors
65+
var (
66+
errInvalidPKCE = xerrors.New("invalid code_verifier")
67+
)
68+
69+
// Use OAuth2-compliant error responses
70+
type OAuth2Error struct {
71+
Error string `json:"error"`
72+
ErrorDescription string `json:"error_description,omitempty"`
73+
}
74+
75+
// Return proper OAuth2 errors
76+
if errors.Is(err, errInvalidPKCE) {
77+
writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "The PKCE code verifier is invalid")
78+
return
79+
}
80+
```
81+
82+
## Testing OAuth2 Features
83+
84+
### Test Scripts
85+
86+
Located in `./scripts/oauth2/`:
87+
88+
- `test-mcp-oauth2.sh` - Full automated test suite
89+
- `setup-test-app.sh` - Create test OAuth2 app
90+
- `cleanup-test-app.sh` - Remove test app
91+
- `generate-pkce.sh` - Generate PKCE parameters
92+
- `test-manual-flow.sh` - Manual browser testing
93+
94+
Always run the full test suite after OAuth2 changes:
95+
96+
```bash
97+
./scripts/oauth2/test-mcp-oauth2.sh
98+
```
99+
100+
### RFC Protocol Testing
101+
102+
1. **Compliance Test Coverage**:
103+
- Test all RFC-defined error codes and responses
104+
- Validate proper HTTP status codes for different scenarios
105+
- Test protocol-specific edge cases (URI formats, token formats, etc.)
106+
107+
2. **Security Boundary Testing**:
108+
- Test client isolation and privilege separation
109+
- Verify information disclosure protections
110+
- Test token security and proper invalidation
111+
112+
## Common OAuth2 Issues
113+
114+
1. **OAuth2 endpoints returning wrong error format** - Ensure OAuth2 endpoints return RFC 6749 compliant errors
115+
2. **Resource indicator validation failing** - Ensure database stores and retrieves resource parameters correctly
116+
3. **PKCE tests failing** - Verify both authorization code storage and token exchange handle PKCE fields
117+
4. **RFC compliance failures** - Verify against actual RFC specifications, not assumptions
118+
5. **Authorization context errors in public endpoints** - Use `dbauthz.AsSystemRestricted(ctx)` pattern
119+
6. **Default value mismatches** - Ensure database migrations match application code defaults
120+
7. **Bearer token authentication issues** - Check token extraction precedence and format validation
121+
8. **URI validation failures** - Support both standard schemes and custom schemes per protocol requirements
122+
123+
## Authorization Context Patterns
124+
125+
```go
126+
// Public endpoints needing system access (OAuth2 registration)
127+
app, err := api.Database.GetOAuth2ProviderAppByClientID(dbauthz.AsSystemRestricted(ctx), clientID)
128+
129+
// Authenticated endpoints with user context
130+
app, err := api.Database.GetOAuth2ProviderAppByClientID(ctx, clientID)
131+
132+
// System operations in middleware
133+
roles, err := db.GetAuthorizationUserRoles(dbauthz.AsSystemRestricted(ctx), userID)
134+
```
135+
136+
## OAuth2/Authentication Work Patterns
137+
138+
- Types go in `codersdk/oauth2.go` or similar
139+
- Handlers go in `coderd/oauth2.go` or `coderd/identityprovider/`
140+
- Database fields need migration + audit table updates
141+
- Always support backward compatibility
142+
143+
## Protocol Implementation Checklist
144+
145+
Before completing OAuth2 or authentication feature work:
146+
147+
- [ ] Verify RFC compliance by reading actual specifications
148+
- [ ] Implement proper error response formats per protocol
149+
- [ ] Add comprehensive validation for all protocol fields
150+
- [ ] Test security boundaries and token handling
151+
- [ ] Update RBAC permissions for new resources
152+
- [ ] Add audit logging support if applicable
153+
- [ ] Create database migrations with proper defaults
154+
- [ ] Update in-memory database implementations
155+
- [ ] Add comprehensive test coverage including edge cases
156+
- [ ] Verify linting compliance
157+
- [ ] Test both positive and negative scenarios
158+
- [ ] Document protocol-specific patterns and requirements

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy