KEMBAR78
Using jsPerf correctly | PDF
Using jsPerf correctly
@mathias – #jsconfeu 2011
<sup>
@mathias
jsPerf
jsPerf is 426 days old.
24,164 test cases → ~56.72 per day
83,536 tests → ~3.46 per test case
1,012 comments → ~2.38 per day
jsPerf is 426 days old.
24,164 test cases → ~56.72 per day
83,536 tests → ~3.46 per test case
1,012 comments → ~2.38 per day
“68% of all jsPerf test cases
        is broken.”
             — Jean-Jacques Rousseau
“It would be unwise to take
all jsPerf results for granted.”
                 — William Shakespeare
Benchmarking pitfalls




    “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
• Inaccurate millisecond timers




     “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
• Inaccurate millisecond timers
• Browser bugs




     “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
• Inaccurate millisecond timers
• Browser bugs
• No statistically significant results




     “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
•   Inaccurate millisecond timers
•   Browser bugs
•   No statistically significant results
•   Browser plugins and addons




       “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
•   Inaccurate millisecond timers
•   Browser bugs
•   No statistically significant results
•   Browser plugins and addons
•   No cross-browser testing


       “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
•   Inaccurate millisecond timers
•   Browser bugs
•   No statistically significant results
•   Browser plugins and addons
•   No cross-browser testing
•   Incorrect tests
       “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
•   Inaccurate millisecond timers
•   Browser bugs
•   No statistically significant results
•   Browser plugins and addons
•   No cross-browser testing
•   Incorrect tests
       “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Benchmarking pitfalls
•   Inaccurate millisecond timers
•   Browser bugs
•   No statistically significant results
•   Browser plugins and addons
•   No cross-browser testing
•   Incorrect tests
       “Bulletproof JavaScript benchmarks” — http://mths.be/asv
Unfair
comparisons
Function declarations
go in the preparation
code or in the setup.
Only the bare minimum
goes inside each test
body.
http://jsperf.com/string-reverse
http://jsperf.com/fibonacci-numbers
Make sure the methods
you’re comparing
actually do the same
thing.
If not, mention this in
the description of the
test case.
Make sure you’re testing
what you intended to
test.
http://jsperf.com/jquery-id-vs-
    native-getelementbyid
When re-using variables
across tests, make sure
you reset them as
needed.
http://jsperf.com/a-b-vs-a-a-b/2
Don’t introduce
randomness in your tests.
http://jsperf.com/zeropad
Don’t test asynchronous
stuff synchronously.
http://jsperf.com/settimeout-vs-setinterval/4
Asynchronous tests — a good example:

  http://jsperf.com/smallest-timeout
There’s a difference
between preparation, setup
and teardown.
Example: scope lookups

 http://jsperf.com/truth
Example: DOM cleanup

http://jsperf.com/basic-dom-manipulation
jsPerf uses Benchmark.js,
which allows for advanced
usage.
       http://benchmarkjs.com/docs
http://jsperf.com/removechild
This messes with your results, too:
This messes with your results, too:
• Unexpected browser features
This messes with your results, too:
• Unexpected browser features
• Dead code removal
This messes with your results, too:
• Unexpected browser features
• Dead code removal
•…
Questions?
 @mathias ~ @jsprf

Using jsPerf correctly