Speedmine: a new solution for better page speed insights

When Google introduced Core Web Vitals in 2020, it solidified something our SEO experts already knew: page speed is a crucial determinant in how a website ranks in search results. But not only does page speed impact ranking, a slow load speed also negatively affects user experience and can lead to higher bounce rates and lower conversions.

That makes monitoring your site’s page speed performance an essential task – and while there are a variety of tools available to help, we’ve noticed some big gaps in the insights they provide. There just isn’t a tool that suits every business – so we made one.

Read on to find out about our new Speedmine tool, but first, let’s take a closer look at why page speed matters so much.

What is page speed?

Simply put, page speed is how quickly content loads on your website – but it’s also a valuable indicator of how a user is going to feel when they visit your website. Waiting three seconds for a page to load feels a lot longer if the screen stays blank than it does if the skeleton of the page appears.

Faster page speeds also increase conversion rates – and slower speeds dramatically reduce them. On mobile, a one second delay in page load can reduce conversions by up to 20%.

Chance of bounce rate increases by 32% when page load speed goes from one to three seconds, and by 90% when load speed goes from one to five seconds. When a page takes more than 10 seconds to load, the probability of a bounce goes up by a whopping 123%. (Source: Google/SOASTA Research, 2017)

That makes page speed an important factor for SEO. Better loading times mean improvements in rankings, user experience, and conversion rates – even a 0.1 second improvement on mobile speed has been shown to improve retail conversions by up to 8.4%.

That makes page speed an important factor for SEO. Better loading times mean improvements in rankings, user experience, and conversion rates – even a 0.1 second improvement on mobile speed has been shown to improve retail conversions by up to 8.4%.

The three factors of core web vitals

With the Core Web Vitals update, Google solidified the three factors they use to evaluate the user’s experience – all of which relate to page speed. While it was first rolled out for mobile, it’s now been introduced for desktop as well.

What’s available

There are currently a number of tools available for collecting page load performance data for your website, each with their pros and cons.

Google offers a number of useful tools:

• Chrome User Experience (CrUX) Report is a public dataset pulled from real-world data – but runs on user-generated information, so only contains data on websites with a large enough volume of traffic. It also averages data across domain (rather than page) and month (rather than a bespoke timeframe), so it can take a while to evaluate page optimisation efforts.
• PageSpeed Insights (PSI) lets you see both real-world usage from CrUX alongside your lab data (how potential users may use your site), but needs to be run individually for each URL.
• Google Search Console enables you to see your Core Web Vitals – but it pulls data from the CrUX report, so also only works for sites with enough traffic.

Other tools – like Lighthouse, Request Metric’s Core Web Vitals Monitoring and clickio’s Core Web Vitals Monitoring – are great for seeing your page speed results on a page-by-page basis. However, they’re limited in their ability to connect to other data sources. 

Screaming Frog’s PageSpeed Insights plugin collects page speed data in bulk using the PageSpeed Insights API. It’s efficient and gives you data by URL, but you’ll need to manually run the crawls and keep your computer on while they run – and the data isn’t presented very clearly, making it difficult to share.

The problem

You might have noticed that many of these tools have serious limitations – we certainly did. Most of them rely on user-generated data, so don’t work for websites with lower volume. They can also be tedious to use, as most require running data by hand.

On top of that, many don’t provide lighthouse results to identify valuable optimisation opportunities, and you may not get the detailed data you need to make improvements to your website.

For companies with multiple websites with thousands of pages, some of which are niche enough to have fairly small amounts of traffic, collecting relevant data can be an enormous task. Likewise, smaller companies may struggle to get enough data to provide meaningful metrics.

Our solution: Speedmine

We wanted to be able to provide our clients with a better way to assess their page speed. The challenge was to create something that would work as well on smaller pages as it does on large websites, that could store and collate data into meaningful metrics, and could do it all automatically.

Enter Speedmine, an Everybody tool that runs in the background to collect page speed performance data on any website.

How it works

Speedmine can run on any website, on any number of pages. It collects both lab and field data on mobile and desktop for thousands of pages every day. 

Lab data shows performance under consistent conditions, which makes it possible to make a clinical like-for-like comparison between pages.

Field data shows what users actually experience across a variety of conditions (such as different networks, devices, and browsers).

The results are securely stored in a data warehouse which can be connected with first party data from other sources, such as Google Analytics, Search Console, or SEMrush.

The tool updates in real time as page speed data is collected, enabling us to provide big-picture and targeted overviews to our clients in a way that works for them. Sometimes that means an easy-to-read dashboard that shows the overall performance of a site and identifies top opportunities. For other clients, it may be a detailed report that gives technical users and developers the data they need to prioritise optimisations.

The benefits are simple:

It’s easy to identify pages that need optimisation
Our clients can better prioritise what to send to developers
Users can run their own reports
Users can monitor a site’s performance before and after optimisations, making it easy to evaluate success

Speedmine reduces what used to take hours to minutes while giving users the ability to connect to other data sources and upstream services. For example, if you have a website crawler you can automatically run the tool on all valid pages.

The most valuable part of the Speedmine dashboard is the opportunities section. Here, users can quickly see the top five issues affecting their site’s page speed. This makes it easy to brief third parties on important optimisation opportunities and compare existing data. 

“With global visibility on top issues and opportunities, clients can prioritise developer resources so time and budget is spent working on issues that have the most impact.”

“Speedmine can also be connected with first party data to give even deeper insights, such as how many website rankings changed as a result of speed optimisations, or which page templates should be prioritised.”

Speedmine in action

Speedmine has been a great tool to bring to our clients. When a client was migrating to a new hosting server, they were adamant about ensuring the technical performance of their new site. We were able to collect page speed data before and after the migration to compare results on a one-to-one basis.

For many of our pharmaceutical clients, it’s given them the ability to collate data into meaningful results that speak to their specific business objectives and use cases. When we’re talking about thousands of pages across therapeutic areas, medical specialty, or resources for healthcare providers and patients, isolating optimisations to improve usability makes a significant difference.

The future of page speed

Site performance is an ongoing effort that requires continuous upkeep and evaluation. Competition is fierce, and users expect a seamless experience no matter how big (or small) your website. Mobile use is also set to continue to grow – 60% of all search volume comes from mobile devices, and that number rises every year.

As Google continues to improve the way they rank websites, the importance of page speed will only increase. Google has even hinted that in the future, search results may include user experience badges for websites that meet their standards of a good web experience.

That means that users may soon be able to see if you have a slow site from their search results page – putting them off before they even visit. If you don’t want drops in rankings and traffic, it’s time to look at your page speed results.