Question

I have a task of breaking down webkit layout test results by passed/failed/skiped/crashed tests
I’m trying to comprehend what kind of “categories” to expect on the output.
I understood the NEW tests and the FAILED tests are shown in the results.txt but what about the rest?

These are the current categories I discovered so far

stderr
incorrect layout
succeeded
timed out

what else should I consider?

Answer 1

You can see the failure types in test_failures.py. They are

• Test crashed
• Test timed out
• Expected output (whether image, audio, or text) is missing
• Expected text output did not match actual output
• Expected image hash (checksum) output did not match actual output
• Expected image data (actual pixels) output did not match actual output
• Expected image and text output did not match actual output
• Expected audio output did not match actual output