(I am not sure if this should be on https://softwareengineering.stackexchange.com/ or not, please comment if you think so)
I am about to create a benchmark for WebGL implementations for my bachelor thesis. I am unsure what kind of tests I should create and if I should only measure the frames per seconds of if I could get some other useful data to benchmark…
Currently I just thought about tests like that:
- 1 single colored object
- 1 multi colored object
- 1 textured object
- 1 textured object with blending
- 1 textured object with lightning
- 1 textured object with multiple lighting points
- 1 scene rendered to texture and use it on another object
- 1 model animation
- all tests with more objects: 50, 500, 5000
- changing shaders/programs per rendering (once, twice, multiple times)
This would result in 40 different tests, but I am not sure if these are significant tests for the performance.
A friend suggested to test on complex shader but as the shader are running on the graphic hardware there shouldn’t be a difference to a java desktop application, right?
Anyway my feeling is, that the performance of the JavaScript is the main bottleneck in WebGL.
Update
I finally did my test. After a discussion about the way the benchmark should work, I created the following: http://martin.cyor.eu/benchmark
What you choose to benchmark should depend on what you intend to do with the benchmark scores (or who the intended audience for the scores are and what they intend to do with them); presumably they are supposed to influence some decision – you should start with that decision and work backwards from that to figure out what the benchmark should measure.
I’d say your proposed list falls at the “synthetic” end of the benchmarking spectrum; they seem like the sort of thing which would be more useful to a WebGL/browser implementer with a profiler than to someone trying to choose between WebGL implementations/platforms to decide which is likely to run apps best (c.f the history of OpenGL benchmarking where all sorts of nonsense used to be claimed for synthetic polygon-per-second numbers until SPECviewperf came along and imposed some realistic real-world test cases).