Written by Harry Roberts on CSS Wizardry.
One of the key tools that performance engineers have at their disposal is the Performance Budget: it helps us—or, more importantly, our clients—ensure that any performance-focused work is monitored and maintained after we’ve gone.
By establishing an acceptable threshold, be that based on RUM data, bundle analysis, image weight, milestone timings, or any other suitable metric, we can be sure that new or unrelated bodies of work do not have a detrimental impact on the performance of our site.
The difficulty, however, lies in actually defining those thresholds. This post
is for anyone who has struggled with the question,
But what should our
budgets actually be?!
When faced with the task of setting a brand new budget, it can feel daunting—almost paralysing—trying to find a value that is both attainable but effective. Is it too ambitious? Is it not ambitious enough? Is it going to be really difficult to hit? Or do we risk making it so easy to achieve that it’s almost pointless? How do we know? How long will it take to find out?
Here’s the thing: most organisations aren’t ready for challenges, they’re in need of safety nets. Performance budgets should not be things to work toward, they should be things that stop us slipping past a certain point. They shouldn’t be aspirational, they should be preventative.
Time and again I hear clients discussing their performance budgets in terms of
we hope to be interactive in 2.75s. While it’s absolutely vital that
these goals exist and are actively worked toward, this is not part of your
budgeting. Your budgeting is actually far, far simpler:
Our budget for [metric] is never-worse-than-it-is-right-now.
Whatever monitoring you use (I adore SpeedCurve and Treo), my suggestion for setting a budget for any trackable metric is to take the worst data point in the past two weeks and use that as your limit. Load times were 7.2s? You’re not allowed to get any slower than that. JS size was 478KB? You can’t introduce any more without refactoring what’s already there. 68 third party requests? You can’t add any more.
Then, every two weeks, you revisit your monitoring and one of three things can happen:
By constantly revisiting and redefining budgets in two-weekly snapshots, we’re able to make slow, steady, and incremental improvements. But the key thing to remember is that we can’t ever increase a performance budget; we can’t change the test just to make it passable. We can maintain or decrease one, but we can’t increase it, because that would completely undermine them.
The following SpeedCurve charts respectively show the Visually Complete, Start Render, and Fully Loaded budgets for CSS Wizardry over a 3G Galaxy S4 from the UK.
I’ve set a Visually Complete budget of 3s, but I’m doing far better than that every single day across the last fortnight. The slowest Start Render in that period was 2.4s, so I need to update my budget accordingly and maintain the new maximum for the next two weeks before revisiting once more.
With an allotted budget of 2s for Start Render, I find that on most days I am actually quite nicely under budget, which is great news. That said, on four occasions I am just nudging the line. This means that I need to be mindful over the next two weeks, but do not need to update anything. This budget is probably quite sensibly set as it’s clearly attainable, but is on the verge of needing some proactive input to keep within the bounds.
Uh oh. I’m completely overshooting my Fully Loaded time every single day, and sometimes by up to four seconds. I can’t reset the budget (unless it’s decided that for some reason 6s was completely incorrect in the first place) but I do have some work to do. Clearly I have decided that load times are important enough to be monitored, so I need to roll up my sleeves and get to work fixing things.
Setting performance budgets doesn’t need to be complex or confusing, you simply need some existing data, some monitoring, and to remember that budgets and targets are two different things. Don’t make life difficult by treating performance budgets as aspirations.
Hi there, I’m Harry. I am an award-winning Consultant Web Performance Engineer, designer, developer, writer, and speaker from the UK. I write, Tweet, speak, and share code about measuring and improving site-speed. You should hire me.
I am available for hire to consult, advise, and develop with passionate product teams across the globe.
I specialise in large, product-based projects where performance, scalability, and maintainability are paramount.