I was browsing the open p5.js issues when I stubbled across this one. There was an open request to add performance benchmarking to the p5.js development tools, that would run benchmarks in multiple real browsers. The goal was to know when performance optimizations were really helping or adding unnecessary complexity. That looked like a fun problem to work on. Since performance testing is one thing I really love doing, I decided to take it on. I learned A lot in the process. This post explains the main parts of the of the system I put together, using existing plugins. It is copied from the article on the p5.js wiki .
We have a grunt task that runs performance benchmarks in multiple real browsers on the developers local machine. It will automatically detect which browsers are installed from the following list (Chrome, Firefox, Safari, Edge, IE) and run the benchmarks in all installed browsers and report the results.
Our benchmarking system consists of 3 main components:
npm install
grunt
grunt karma:<target_benchmark>
grunt karma
grunt karma:random-dev
Outputs:1
2
3
4
5
6Chrome 62.0.3202 (Linux 0.0.0)
p5 random() vs Math.random(): Math.random() at 95811115 ops/sec (1.26x faster than p5 random())
Firefox 56.0.0 (Fedora 0.0.0)
p5 random() vs Math.random(): Math.random() at 2367566507 ops/sec (1.14x faster than p5 random())
Done, without errors.
grunt-karma.js
Here is an example benchmark that compares p5 random() to Math.random() with Friendly Error System disabled. It has two suites one for instanced mode and one using the global window random().
1 | p5.disableFriendlyErrors = true; |
Then you would need to add your new benchmark to grunt-karma.js
1 | ... |
Now you can run your new benchmark with:grunt karma:random-fe-off-dev
karma-benchmark can load remote files. So it’s easy to include the p5.js prod build and compare it to the dev build. This is important when you’re working on improving performance to compare your changes to what is in production. To do this simply make two targets in grunt-karma.js
one for prod and one for dev.
1 | 'random-prod': { |
You can see that random-prod
actually loads the latest build from CDN. Then to compare you can run both targets using:grunt karma:random-prod karma:random-dev
I chose to put the grunt-karma tasks in it’s own file grunt-karma.js
instead of the main Gruntfile.js
, because as we add more benchmarks overtime the file could grow quite long, and I wanted to keep the main Gruntfile clean.