Google has updated its Octane speed test for measuring JavaScript performance, expanding its scope and rounding out the benchmark with computing tasks from rival browser makers Mozilla and Microsoft.
With the expanded scope, Google tries to measure not just execution performance but also delays that can trip up Web pages and Web apps, Google said in its Octane 2.0 announcement Wednesday. Specifically, it added new tests to gauge sluggishness from the initial compilation of JavaScript programs into machine-executable code and from the memory-scrubbing operation called garbage collection.
More politically interesting is Octane 2.0's addition of Microsoft's Typescript compiler and the zlib compression software from Mozilla's Emscripten project. The latter is particularly interesting since it's affiliated with Mozilla's asm.js project to dramatically improve JavaScript performance by optimizing browsers for a particular subset of the language.
Benchmarks appeal to geeky types, to be sure, but they matter for everyone.
Any browser maker worth its salt these days cares passionately about performance. The faster it can process HTML content, CSS formatting and effects, and JavaScript programs, the faster users get to see Web pages and the more sophisticated tools like Office 365 become.
There are so many JavaScript tests out there that it's tough to pick just one. Other options include Mozilla's Kraken, Purdue University's JSBench, and WebKit's SunSpider, which many browser cognoscenti consider outdated but which Microsoft continues to use to promote IE performance. (In a Microsoft IE anime video to promote IE with an anime heroine, Kraken and Octane make an appearance on posters plastered around a seedy part of a metaphorical Internet.)
Perhaps by including rivals' computing chores in Octane, Google will have an easier time swaying them to respect Octane -- and to refrain from any criticism that Google cherry-picked tests that make Chrome look good.
How exactly should a browser maker measure that performance? Finding a benchmark is tough, since benchmarks measure different aspects of performance, and what might matter in playing Hover on the Web might not matter for recalculating spreadsheet values.
For that reason, JavaScript benchmarks are typically composites of several smaller tests.
And, in Google's view, Octane has the most appropriate selection of tests.
"We believe that most other benchmarks don't stress adequately the performance bottlenecks that are worth optimizing, in order to improve everybody's experience on the Web," Google said in its Octane FAQ. "The Web has evolved and they are often not representative, not comprehensive enough or in some cases too 'game-able'" -- in other words, susceptible to cheats that give good scores despite not having good performance.