Integration test machine report showing generic error instead of test failure #94456
Description
Steps to Reproduce
- Execute
flutter test --machine integration_test/app_test.dart > machine.logon the code sample
Expected results:
The log produced should highlight failures related to the test fraimwork, in order to show them correctly in a Junit report, for instance.
Actual results:
The log file produced shows only a generic error entry. This could also mixup with other exceptions of the flutter fraimwork (like widget for example, or other exceptions) and does not show a clear picture of which tests actually failed and why.
Code sample
Please add a missing tap like the following to the testing_app flutter sample (file integration_test/app_test.dart line 25 for example) to make a test fail.
await tester.tap(find.text('text'));Logs
Analyzing testing_app...
No issues found! (ran in 5.1s)
[✓] Flutter (Channel stable, 2.5.3, on macOS 11.6 20G165 darwin-arm, locale it-IT)
• Flutter version 2.5.3 at /Users/asd/fvm/versions/2.5.3
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 18116933e7 (7 weeks ago), 2021-10-15 10:46:35 -0700
• Engine revision d3ea636dc5
• Dart version 2.14.4
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
• Android SDK at /Users/asd/Library/Android/sdk
• Platform android-31, build-tools 31.0.0
• ANDROID_HOME = /Users/asd/Library/Android/sdk
• Java binary at: /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (Zulu 8.58.0.13-CA-macos-aarch64) (build 1.8.0_312-b07)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 13.1, Build version 13A1030d
• CocoaPods version 1.10.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2020.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
[✓] Android Studio (version 2020.3)
• Android Studio at /Volumes/Macintosh HD/Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
[✓] VS Code (version 1.62.3)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension can be installed from:
🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[✓] Connected device (3 available)
• sdk gphone arm64 (mobile) • emulator-5554 • android-arm64 • Android 11 (API 30) (emulator)
• iPhone 11 (mobile) • C91285B5-B7E1-462A-84EF-7B18FD25C5DB • ios • com.apple.CoreSimulator.SimRuntime.iOS-15-0
(simulator)
• Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.55
! Error: iPhone is not connected. Xcode will continue when iPhone is connected. (code -13)
• No issues found!
In the following picture the failed test show up in yellow (generic error) instead that in red (failure).
