Max RozenMax Rozen
TwitterArticlesNewsletter

How to choose a median result when running Google Lighthouse multiple times

According to the docs, running Google Lighthouse five times against the same URL will reduce the variability of your results by around half if you pick the median result.

The problem is, how do you pick the right result to use as your median, or representative result?

A naïve approach would be to take the median of the performance score, however since the performance score itself is a weighted average of other scores, and is more prone to outliers, it isn’t as useful as you’d think.

Lighthouse actually exports a function to help you calculate which result to use: computeMedianRun.

How to use computeMedianRun in Node

const spawnSync = require('child_process').spawnSync;
const lighthouseCli = require.resolve('lighthouse/lighthouse-cli');
const {
  computeMedianRun,
} = require('lighthouse/lighthouse-core/lib/median-run.js');

const results = [];
for (let i = 0; i < 5; i++) {
  console.log(`Running Lighthouse attempt #${i + 1}...`);
  const { status = -1, stdout } = spawnSync('node', [
    lighthouseCli,
    'https://example.com',
    '--output=json',
  ]);
  if (status !== 0) {
    console.log('Lighthouse failed, skipping run...');
    continue;
  }
  results.push(JSON.parse(stdout));
}

const median = computeMedianRun(results);
console.log(
  'Median performance score was',
  median.categories.performance.score * 100
);

Under the hood, computeMedianRun finds the run that’s closest to the median First Contentful Paint (FCP), AND the closest to the median Time to Interactive (TTI).

FCP and TTI are used because they’re the earliest and latest moments in a page’s lifecycle.

Do you struggle to keep up with best practices in React?

I send a single email weekly with an article like this one to help improve the quality of your React apps. Lots of developers like them, and I'd love to hear what you think as well. You can always unsubscribe.

    Join 158 React developers who signed up last month!

    rssTwitterGitHub

    © 2020 Max Rozen. All rights reserved.