Description
Description
I have been wondering about a behaviour present when running a PHPUnit functional test in a command line.
Current behaviour: when using the off the shelf test assertions, such as assertResponseIsSuccessful(), each constraint includes the entire response body via additionalFailureDescription() method.
This makes the PHPUnit output quite verbose, thanks to the beautiful error page generated by Symfony, which is great via the browser, but when it comes out as a plaintext, it's not so handy.
There is a good chance I'm doing something wrong - maybe I'm trying to solve a wrong problem, maybe there is a way to turn the error page off when accessing the page with the testing client. If so, please point me somewhere I can learn about it and read no more, and I apologise for wasting everyone's time.
If I'm right about the behaviour, then I'd like to propose changing it, because a single failed assertion generates so much clutter that it is difficult to navigate/read the test results.
I'm happy to create a PR for this, but before I jump into it, I'd like to collect some feedback.
My proposal:
- Let the developer say whether or not the constraint is verbose. Sometimes we might want to how the response exactly looks like, but most of the time the exception message might be just enough to give us a hint.
- Add a boolean argument to constraints living in
Symfony\Component\HttpFoundation\Test\Constraint, say$verbose = true - Support for this argument be added to all methods in
\Symfony\Bundle\FrameworkBundle\Test\BrowserKitAssertionsTrait - Optionally, the
BrowserKitAssertionsTraitcould define a default $verbose value, which could be overridden by individual test classes using the trait.
All this can be done without affecting any existing functionality, providing the tests would be verbose by default.
What do you think?
Thanks for reading.
Example: Failed test in the current state, with the complete response body attached:
PHPUnit 9.5.9 by Sebastian Bergmann and contributors.
Testing ControllerSmokeTest
F 1 / 1 (100%)
Time: 00:03.920, Memory: 68.50 MB
There was 1 failure:
1) App\Tests\ControllerSmokeTest::testNoSmoke with data set "test" (Closure Object (...), array('test'))
Failed asserting that the Response is successful.
HTTP/1.1 404 Not Found
Cache-Control: max-age=0, must-revalidate, private
Content-Type: text/html; charset=UTF-8
Date: Thu, 09 Sep 2021 01:08:53 GMT
Expires: Thu, 09 Sep 2021 01:08:53 GMT
Referrer-Policy: no-referrer, strict-origin-when-cross-origin
Set-Cookie: MOCKSESSID=375c8950a650cc3698c4603766660b93b590317cf18bc3b2ba3eed6b911b4709; path=/; secure; httponly; samesite=lax
X-Content-Type-Options: nosniff
X-Debug-Exception: No%20route%20found%20for%20%22GET%20http%3A%2F%2Flocalhost%2Ftest
X-Debug-Exception-File: %2Fapp%2Fvendor%2Fsymfony%2Fhttp-kernel%2FEventListener%2FRouterListener.php:136
X-Frame-Options: DENY
X-Robots-Tag: noindex
X-Xss-Protection: 1; mode=block
<!-- No route found for "GET http://localhost/test" (404 Not Found) -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="robots" content="noindex,nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<title>No route found for "GET http://localhost/test" (404 Not Found)</title>
<style>/* This file is based on WebProfilerBundle/Resources/views/Profiler/profiler.css.twig.
If you make any change in this file, verify the same change is needed in the other file. */
:root {
--font-sans-serif: Helvetica, Arial, sans-serif;
--page-background: #f9f9f9;
--color-text: #222;
/* when updating any of these colors, do the same in toolbar.css.twig */
--color-success: #4f805d;
--color-warning: #a46a1f;
--color-error: #b0413e;
--color-muted: #999;
--tab-background: #fff;
--tab-color: #444;
[... thousands of lines of HTML and CSS - you get the point ...]
app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php:137
app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php:33
app/tests/BaseTestCase.php:125
app/tests/BaseTestCase.php:66
app/tests/ControllerSmokeTest.php:31
Caused by
ErrorException: No route found for "GET http://localhost/path" in app/vendor/symfony/http-kernel/EventListener/RouterListener.php:136
Stack trace:
#0 app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php(33): Symfony\Bundle\FrameworkBundle\Test\WebTestCase::assertThatForResponse(Object(Symfony\Component\HttpFoundation\Test\Constraint\ResponseIsSuccessful), '')
#1 app/tests/BaseTestCase.php(125): Symfony\Bundle\FrameworkBundle\Test\WebTestCase::assertResponseIsSuccessful()
[... stack trace not important...]
#13 app/bin/phpunit(11): PHPUnit\TextUI\Command::main()
#14 {main}
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
Example: without the response body, the error is still perfectly legible
PHPUnit 9.5.9 by Sebastian Bergmann and contributors.
Testing ControllerSmokeTest
F 1 / 1 (100%)
Time: 00:03.920, Memory: 68.50 MB
There was 1 failure:
1) App\Tests\ControllerSmokeTest::testNoSmoke with data set "test" (Closure Object (...), array('test'))
Failed asserting that the Response is successful.
app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php:137
app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php:33
app/tests/BaseTestCase.php:125
app/tests/BaseTestCase.php:66
app/tests/ControllerSmokeTest.php:31
Caused by
ErrorException: No route found for "GET http://localhost/path" in app/vendor/symfony/http-kernel/EventListener/RouterListener.php:136
Stack trace:
#0 app/vendor/symfony/framework-bundle/Test/BrowserKitAssertionsTrait.php(33): Symfony\Bundle\FrameworkBundle\Test\WebTestCase::assertThatForResponse(Object(Symfony\Component\HttpFoundation\Test\Constraint\ResponseIsSuccessful), '')
#1 app/tests/BaseTestCase.php(125): Symfony\Bundle\FrameworkBundle\Test\WebTestCase::assertResponseIsSuccessful()
[... stack trace not important...]
#13 app/bin/phpunit(11): PHPUnit\TextUI\Command::main()
#14 {main}
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.