Case study: Does Raising PSI Score Improve Page Speed?

With Shopify’s integration of the Google Lighthouse tool (also used by Google PageSpeed Insights) for their speed score report a lot of store owners are looking to improve their scores but don’t fully understand how it works and what the score means.

In this case study I’m going to dive into Google PageSpeed Insights (aka Google PSI) and share what happened when I solved Google PSI warnings on a Shopify site.

Hypothesis

For this test, my hypothesis is that improving the Google PageSpeed Insights score will improve the actual page speed on real devices.

Side note: The Shopify speed score is directly related to Google PSI score, since they both use the same engine (Lighthouse). For a breakdown on the Shopify speed score tool + how it works, see Shopify Speed Tool Analysis + How to Make Your Site Faster.

When I started the project, I captured screenshots of the mobile benchmarks using Google PageSpeed Insights (the client wasn’t concerned with Desktop since the mobile bounce rate was higher and our first guess was that this may be related to page performance).

Google PSI Score Before

Since Google PageSpeed Insights is an emulated testing tool (meaning that the results are based on an estimation) I also grabbed a screenshot from WebPageTest (aka WPT) since this tool captures metrics using real devices and networks.

WebPageTest Score Before

Score Summary Before

Document Complete = the page content is loaded and the DOM (Document Object Model) is ready, but there may still be scripts retrieving content.

Fully Loaded = 2 seconds have passed without any network activity, indicating that all the pages resources are fully loaded.

In my experience testing and working on hundreds of Shopify sites, I’ve never seen a store with a perfect score. In fact, most sites are in the 15 – 40 range.

Google Recommendations

I dug in deep and followed Google’s official recommendations to make a positive impact to the score – with very surprising results. Here’s what I did to create a higher score:

• Combined CSS and JS
• Deferred parsing of JS
• Applied critical path CSS
• Added height and width attributes to images
• Used font-display: swap for fonts

Google offers a lot of recommendations on improving each of these areas, most require some high level programming knowledge, here’s a list of their explanations:

Why didn’t we compress or lazy load images for this project? For this particular client we had already implemented all of our recommended changes (image compression, lazy loading, using progressive jpg format, preloading, etc.) and were looking for ways to further improve the site speed by focusing on the PSI score and improving core web vitals.

Google’s updated 2020 guidelines for Core Web Vitals suggest that CLS, FID and LCP are three important factors for improving platform-independent performance.

This particular theme was comprised of around 35 different open-source libraries, containing over 50 individual javascripts and stylesheets. Because of how the browser interprets and renders the HTML document, styles and scripts typically load upfront which means they can block other assets on the page from loading quickly.

All scripts and styles couldn’t be combined (this recommendation by Google is actually out of date, combining files can actually lead to slower performance).

Since integration of a lot of open source libraries and customizations is common on Shopify, max optimization for Google PSI calls for all of that to be re-written in a way that omits all unused code. The file size is typically pretty small here, however a single library can take up to 20 hours to analyze and rewrite – this leads to a more expensive project than commissioning a new, custom-built theme that is engineered to avoid these issues.

I also applied a technique to lazy load and preload certain vital scripts and stylesheets. This ensures that page-to-page browsing doesn’t require new scripts and styles to be loaded on every page and instead keeps those ready for the next page, eliminating requests, reducing page size, and helping those other pages load faster.

Critical path CSS analysis was tricky – since there were such a large volume of open source libraries that were added indiscriminately by the original theme developer, there were a high volume of conflicting styles. This is one drawback to using pre-built libraries however I have not yet seen a Shopify theme that doesn’t use any of these – they serve a purpose to save time developing (and in turn saving cost to the store owner purchasing the theme).

Most pages can’t use the critical CSS technique when a lot of third-party CSS is added because of conflicting styles and the complexity of dissecting the code in production stylesheets to only load exactly where they’re needed..

Results

The results were a bit shocking at first – I had doubled the score but I also noticed that when testing on real devices (including devices in my own testing lab) the page was noticeably more sluggish while the browser timing events on real devices was drastically different than what the Lighthouse tool displayed in its results.

Google PSI Score After

WebPageTest Score After

Score Summary After

Although in testing the new version of the theme scored higher using Google PageSpeed Insights, the actual load speed was slower.

The client also had a number of complaints about issues that he felt detracted from the customer experience.

I had to agree with him because these updates gave users the appearance of a slow loading website:

• Flashing of fonts
• Flickering and shifting page elements that change style (despite the CLS improvement)
• Various deferred javascripts that change and apply styles client-side populate too late after the page load – this means that styles and elements that display on your web page which are controlled by javascripts aren’t ready when the page loads and these took longer to appear visually on the page

In fact, after PSI score improvement, collections URLs slowed down by almost 2 full seconds for a fully loaded page!

I also noted that even though test data showed PageSpeed Insights thinks it’s a “faster” page (one the pitfalls of emulation-based testing), both users and real devices using real networks noted a much slower site.

Project update: we actually ended up rolling these changes back and sticking with a faster site with a lower PSI score.

But wait – even if we kept the Google PSI recommended changes, all that work for only 18 points? Why couldn’t it go higher?

The answer is simple – using store facing 3rd party apps slows your site down and prevents you from getting a high score.

This doesn’t mean you should uninstall your apps. As an online store I’m assuming your primary goal is have the most sales, not the highest score in an analysis tool.

Having a low PSI score is not as bad as it sounds – even Amazon.com, the most successful ecommerce website in the world, only scores a 31.

How to optimize a Shopify site

Let’s take a deeper look into what this means (and why you shouldn’t worry about a low PSI score).

If you’re going to optimize a website for Shopify, you first need to understand how these sites work – and how these are different from other websites. As a developer, I basically break these up into three main components.

1. Backend / server side

This basically controls everything from how many product variants you can add down to CDN configuration (a Content Delivery Network is a way to distribute your website faster around the world which Shopify configures out of the box for all of its store owners) and URL patterns for the website.

Spoiler alert: you can’t change anything here.

This is the part that the internal Shopify development team builds and maintains. Thankfully you don’t need to since Shopify’s internal team does a great job at improving things like latency and internal response timing 👍

2. Store facing apps

Usually you’ll want to extend functionality or create extra features. You can do this with custom coding or downloading an app from the Shopify app store.

Example store facing app functionality: product reviews, email marketing, wishlists, product bundles, and anything that interacts with the user on the website.

Not all apps affect your load speed, only store facing apps.

Apps that run in the background and do not have any code or content loading on your website do not affect your load speed.

Example backend apps that don’t affect load speed: order processing and automation apps like Order Automator, apps that sync data (for example to other marketing channels), or apps that generate code directly into your theme but don’t rely on 3rd party scripts (for example Tip Cup places a tip widget onto your site by generating theme code on the server side.)

3. Frontend / theme website code

Themes are the last component of optimization-driven development for Shopify websites. Your theme contains the code-based templates that control the individual page layouts and the core functionality of your site as a whole.

This is also the main component of optimization since this the part you actually have control over. This is where I was able to make those changes I discussed earlier.

Wait, you mean I can only control one of these things – what about apps?

Ok, let’s talk more about apps for a second. Third-party content is typically injected into a Shopify site in one of two places:

• In the {{ content_for_header }} block (explained in theme.liquid docs)
• or injected directly into your code on a theme-level

If code is stored directly in your theme files, you might have some control over this like limiting the pages where it loads.

But third-party apps will often inject their code in the {{ content_for_header }} block.

Ay, there’s the rub. On one hand, you’re getting the functionality you want but on the other hand it’s at the sole discretion of app developers to optimize their own server response timing, how their content is served, whether or not it’s compressed, and whether or not it’s identified as a render-blocking asset.

This means that you have no control over this content. In fact, more than half of the Shopify sites I personally analyzed in 2020 relied on a third-party app developer for 60% or more of their total website content, so you’re totally not alone.

Optimization roadmap

So, what next? If you can’t control most of my content on Shopify then what do you do?

You could build a completely custom website and then use the Shopify API to communicate your store data with your front end website, but that’s a major project and probably won’t pay off in ROI unless you’re doing at least 8 figures annual revenue.

For most stores, it doesn’t make sense to migrate away from such a robust platform or to do away with every app you have (that would be a conversion killer).

For the majority of Shopify stores, here’s what I recommend:

Summary

A lot of the advice that the Lighthouse tool (and by extension Google PSI and Shopify speed score) provides is good, however it isn’t always accurate as far as timings go.

Relying strictly on emulation-based advice can actually hurt your page speed. Sometimes these types of tools can locate opportunities for improvement, however this tool is geared for general sites and not specific to ecommerce sites that use engaging features to help win more sales.

There are a lot of tools out there but for real-time recommendations that you actually have direct control over that are proven to have a positive impact on page speed when optimizing Shopify sites, check out the Shopify Analyzer.

And to combo performance analysis with testing using live performance metrics, I recommend WebPageTest. You can test using different browsers, locations, devices and more. It also seems to be the most consistent tool for measuring load speed.

Those are the 2 best tools for optimizing Shopify sites. 🛠