Content-Length: 243957 | pFad | http://github.com/coder/coder/pull/19071

FF fix(e2e): improve stability of createWorkspace tests by blink-so[bot] · Pull Request #19071 · coder/coder · GitHub
Skip to content

fix(e2e): improve stability of createWorkspace tests #19071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

blink-so[bot]
Copy link
Contributor

@blink-so blink-so bot commented Jul 29, 2025

Problem

The createWorkspace.spec.ts e2e tests have been experiencing flakiness, causing intermittent failures in CI. After investigation, I identified several race conditions and timing issues in the test helpers.

Root Causes

  1. Race conditions in disableDynamicParameters function:

    • Used domcontentloaded instead of networkidle for page loading
    • Didn't wait for checkbox visibility before checking state
    • Fragile success message selector
    • No timeout buffers for UI state changes
  2. Timing issues in createWorkspace function:

    • Used domcontentloaded instead of networkidle
    • No wait for form readiness before filling parameters
  3. Insufficient waits in fillParameters function:

    • No explicit timeout on parameter field selectors

Solution

Enhanced disableDynamicParameters Function

  • ✅ Use networkidle for better page loading
  • ✅ Add explicit waits for form and checkbox visibility
  • ✅ Use more robust selectors for success messages ([role='alert'], .MuiAlert-root, text=Template updated successfully)
  • ✅ Add timeout buffers for UI state changes
  • ✅ Increase timeout to 15 seconds for success message

Improved createWorkspace Function

  • ✅ Use networkidle for more reliable page loading
  • ✅ Add form readiness check before filling parameters
  • ✅ Add 500ms timeout buffer for form element rendering

Enhanced fillParameters Function

  • ✅ Add explicit 10-second timeout to parameter field selectors

Testing

To verify these fixes work, run the tests multiple times:

# Run the specific test file 5 times to check for flakiness
pnpm run playwright:test createWorkspace.spec.ts --repeat-each=5

# Or run a specific test multiple times
pnpm run playwright:test createWorkspace.spec.ts -g "create workspace with default immutable parameters" --repeat-each=10

Expected Impact

These changes should significantly reduce test flakiness by:

  • Ensuring all UI elements are fully loaded before interaction
  • Adding proper waits for dynamic content
  • Using more reliable page loading strategies
  • Providing better error handling with explicit timeouts
  • Adding stability buffers for UI state changes

The tests should now pass consistently across multiple runs without the race conditions that were causing intermittent failures.

This commit addresses several potential sources of flakiness in the
createWorkspace e2e tests:

1. Enhanced disableDynamicParameters function:
   - Use networkidle instead of domcontentloaded for better page loading
   - Add explicit waits for form and checkbox visibility
   - Use more robust selectors for success messages
   - Add timeout buffers for UI state changes

2. Improved createWorkspace function:
   - Use networkidle for more reliable page loading
   - Add form readiness check before filling parameters
   - Add timeout buffer for form element rendering

3. Enhanced fillParameters function:
   - Add explicit timeout to parameter field selectors

These changes should significantly reduce test flakiness by ensuring
all UI elements are fully loaded and stable before interaction.

Co-authored-by: jaaydenh <1858163+jaaydenh@users.noreply.github.com>
@jaaydenh jaaydenh closed this Jul 29, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Jul 29, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/coder/coder/pull/19071

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy