In #153445 we investigated that Android 35 (the upcoming release) is too unstable on emulators to be used on pre or post-submit infrastructure (see our [CI changes here (https://github.com//issues/153445)). In an informal conversation between @zanderso @reidbaker @johnmccutchan and @matanlurey we talked about our inability to determine root causes.

@johnmccutchan wrote:

I guess I'm wondering why we can't have finer grained signals of test failures?

Like if an emulator is crashing I'd like that to be flagged as something distinct than a flakey test. Because it wasn't the test that flaked actually, it was our environment.

@zanderso wrote:

Yeah, I was thinking like flutter drive would have different error exit codes for a test failure vs. failures for other reasons like the emulator/device disappearing.

So, this issue is tracking doing this.

I admit I am not sure how easy this will be, in particular to do across every platform we support.

If we limit the scope (i.e. Android devices), or file sub-issues for every platform, it might be viable to do some of these.