Gone are the days when you could easily hack SEO by loading a page with keywords and creating artificial backlinks.
Today, Google is consistently rolling out changes to its algorithms to reward quality.
Unlike the past, if you have low-quality pages on your website, it can negatively impact your overall ranking.
What’s a low-quality page?
It’s one that isn’t used or visited, is full of duplicate content from other pages, has thin content or very low engagement in the eyes of Google.
Here’s the thing:
It’s entirely possible that you have dozens, hundreds, or thousands of low-quality pages on your site — in the eyes of Google — and you might not even realize it.
We call this problem index bloat.
It happens when Google has indexed a lot of URLs for your website that it views as low-quality.
In this article, we’ll show you:
- An example of index bloat
- Common causes
- The exact steps you can take to see if you have a problem
Note: We can help you spot and fix issues on your website that are harming your overall ranking. Contact us here.
Index Bloat: A Real-life Example
We recently started working with an eCommerce client and discovered something fascinating (and troubling) as we did our standard checks to evaluate their site.
After talking to them, we expected the site to have somewhere around 10,000 pages.
When we looked in Google Webmaster Tools, we saw — to our surprise — that Google had indexed 38,000 pages for the website. Find this chart here: Web Tools > Search Console > Google Index > Index Status.
That was way too high for the size of the site.
We also saw that the number had risen dramatically.
In July of 2017, the site had only 16,000 pages indexed in Google Analytics.
What happened?
How a Hidden Technical Glitch Caused Massive Index Bloat
It took a while to figure out what had gone wrong with our client’s site.
Eventually, we found a problem in their software that was creating thousands of unnecessary product pages.
At a high-level, any time the website sold out of their inventory for a brand (which happened often), the site’s pagination system created hundreds of new pages.
Put another way, the site had a technical glitch that was creating index bloat.
The company had no idea their site had this problem, which is common with a site that has a technical glitch.
For eCommerce sites that automatically generate new pages for products, brands, or categories, things like this can easily happen.
It’s one common cause of index bloat, but not the only one.
Other common causes include:
- Pages with too little original content
- Old blog posts, news releases, or case studies that get little to no traffic
- Search pages that get accidentally indexed by Google
Don’t think you’re safe just because your list of indexed pages looks like this:
Even if the overall number of pages on your site isn’t going up, you might still be carrying unnecessary pages from months or years ago — pages that could be slowly chipping away at your relevancy scores as Google makes changes to its algorithm.
The good news is: it’s relatively easy to identify and remove pages that are causing index bloat on your site.
We also have a free tool you can use that will help.
How to Identify and Remove Poor Performing Pages
Here’s the step-by-step process we use with our clients to identify and remove poor performing pages:
(1) Estimate the number of pages you should have
Estimate to the number of products you carry, the number of categories, blog posts, and support pages, and add them together. Your total indexed pages should be something close to that number.
(2) Use the Cruft Finder Tool to find poor-performing pages
The Cruft Finder tool is a free tool we created to identify poor-performing pages. It’s designed to help eCommerce site managers find and remove pages that are harming your SEO ranking.
The tool sends a Google query about your domain and — using a recipe of site quality parameters — returns page content we suspect might be harming your index ranking.
Mark any page that:
- Is identified by the Cruft Finder tool
- Gets very little traffic (as seen in Google Analytics)
These are pages you should consider removing from your site.
(3) Decide what to keep and what to remove
For years, you’ve been told that adding fresh content on your site increases traffic and improves SEO. You should be blogging at least once a week, right?
Well, maybe.
If a blog post has been on your website for years, has no backlinks pointing to it, and no one ever visits it, that old content could be hurting your rankings.
Recently, we deleted 90% of one client’s blog posts. Why? Because they weren’t generating backlinks or traffic.
If no one is visiting a URL, and it doesn’t add value to your site, it doesn’t need to be there.
(4) Revise and revamp necessary pages with little traffic
If a URL has valuable content you want people to see — but it’s not getting any traffic — it’s time to restructure.
Could you consolidate pages? Could you promote the content better through internal linking? Could you change your navigation to push traffic to the page?
Also, make sure that all your static pages have robust, unique content. When Google sees thousands of pages on your site with sparse or similar content, it can lower your relevancy score.
(5) Make sure your search results pages aren’t being indexed
Not all pages on your site should be indexed. The main example of this is search results pages.
You almost never want search pages to be indexed because there are better pages to funnel traffic that have better quality content. These are not meant to be entry pages.
This is a common issue.
For example, here’s what we found using the Cruft Finder tool for one major retail site: over 5,000 search pages indexed by Google.
If you find this issue on your own site, follow Google’s instructions to get rid of search result pages.
The Results and Impact on Traffic and Revenue
What kind of impact can index bloat have on your results?
And what kind of positive effect have we seen after correcting it?
Here’s a graph of indexed pages from a recent client that was letting their search result pages get indexed — the same way we explained above. We helped them implement a technical fix so those pages wouldn’t be indexed anymore.
In the Google Analytics graph, the the blue dot is where the fix was implemented. The number of indexed pages continued to rise for a bit, then dropped significantly.
Year over year, here’s what happened to the site’s organic traffic and revenue:
3 Months Before the Technical Fix
- 6% decrease in organic traffic
- 5% increase in organic revenue
3 Months After the Technical Fix
- 22% increase in organic traffic
- 7% increase in organic revenue
Before vs. After
- 28% total difference in organic traffic
- 2% total increase in organic revenue
This process takes time.
For this client, it took three full months before the number of indexed pages returned to the mid 13,000s, where it should have been all along.
Note: Interested in a personalized strategy to reduce index bloat and raise your SEO ranking? We can help. Contact us here.
Hi Chris, I have been exploring the idea of pruning useless pages and reducing index bloat.
I had a question. Once you have identified such pages, do you simply add a noindex tag to them? Are these pages deleted? Redirected?
BTW, really nice article- I’ll be using the Cruft Finder tool 😉
Hey Gurbir… Good question. There are lots of ways to “prune” a page once you’ve identified it isn’t a good current target for your SEO efforts. Our older article on Moz about pruning eCommerce sites is a good place to start. It also contains a link to again a bit older but still relevant overview of our content audit process. We plan to update both these articles soon.
Its extremely good and very helpful for me.Thanks for sharing this great post.
Chris, Thanks for the article. I had a question about low-value pages and whether they should be kept. Some argue that the larger a website, the better. With user engagement now a ranking factor, it seems logical that poor performing pages be removed.
Hi John,
As with many things SEO, I’d say it depends on a number of factors. Also ‘removing’ a page can mean a couple different things. For example, eCommerce sites wouldn’t want to 404 product pages if the product was still in stock. Low performing product pages are typically candidates for a noindex tag. There could also be the case of near-duplicate product pages if a product were available in multiple sizes and/or colors, and the CMS didn’t have the proper functionality to combine multiple URLs into one. In this case we might use canonical tags if there weren’t sufficient search volume to justify different pages for every size/color.
If the poor performing page is strategic content (blog post or similar), we’ve had success by using the “Remove, Improve or Consolidate” strategy. If there are multiple pieces of content on the same or similar topic, they can sometimes be consolidated into a single, more authoritative post. We also often improve posts, which include things like improving keyword targeting and expanding the length of the content (don’t forget to update the DateModified meta tag). Finally, we have rolled out a strategy to 404 pages that just aren’t performing (little to no traffic over a period of time) – for one client we 404’d 90% of their blog posts that received little traffic and saw a boost in rankings, traffic and revenue to their foundational content (category & product pages).
Hope this helps!
Hi there, did you update the article mentioned in this reply comment?
Thanks
I am not sure whether to use a redirect or not, but I am going to consolidate some pages on my website.
Hi Janice,
It’s usually a good idea to 301-redirect URLs that have been consolidated to the URL to which you consolidated – especially if those pages were indexed and/or received any traffic. Don’t forget to also update your internal links so they don’t point to redirected URLs!