Regardless of your website’s size, the keys to a successful SEO campaign are (on-page), links (off-page), and technical factors.
Your content acts like the body of a race car, and it’s referring backlinks like race fuel. The technical factors act like the nuts and bolts allowing everything else to perform at its best.
If you miss either of these then your website will struggle to rank much like a race car struggling to get down the track.
In this case study, you’ll learn the exact steps that my team at The Search Initiative took to increase our client’s organic traffic by 96%. You will get a crash course in the three pillars of SEO (on-page, off-page, and technical factors) through the lens of this case study.


In this article, you’ll learn how to:
Manage your crawl budget by fixing index bloat and ensuring your sitemap is implemented correctly to make it easier for Google to crawl and index the right pages.
Implement hreflang tags correctly to avoid potential duplicate content issues.
Target long-tail keywords with supplementary blog content to improve keyword visibility and build topical relevance.
Identify underperforming important pages that are lacking in backlinks to help build their authority and boost their rankings.
Before getting into the details of the strategy, here’s some important information about the website’s goals and the main challenges that were faced.
The Challenge
The main objective for this campaign was to increase the amount of quality organic traffic on the site to grow the number of leads.
The client is a US-based SaaS (Software as a Service) B2B company that builds and offers cloud software with web pages targeting a range of countries including English speaking countries such as the U.S.A., as well as Japan, China, Korea, and France.

With this in mind, one of the main challenges was index bloat. There were over 30k crawlable URLs on the English version of the website alone – quite excessive for a SaaS website. Fixing these crawl budget issues and uploading the XML sitemap which was missing when the client joined TSI was a priority. More on that, below…
The client’s hreflang (a way to tell Google about the language and target location of your content) setup had not been implemented correctly.
Although the core landing pages of the site were relatively well optimized, there was a lack of supporting content to drive traffic towards them. This is because the client’s blog was not active, with just a handful of articles published.

This was tackled by researching and writing informational blog articles to target long-tail keywords. This helped build the client’s topical relevance within the niche as well as provide internal linking opportunities towards the main pages on the site.
The final step was to build authority with a link-building strategy that focused on building page authority on the website’s most important pages: the homepage and service pages.
Follow the steps below and find out how you can also overcome these challenges for your own websites.
Crawl Budget Management
What Is Crawl Budget & Why Is It Important?
Google only has a limited amount of time and resources that it can allocate to crawling and indexing the World Wide Web. Therefore, Google sets a limit on how much time it spends crawling a given website – this is known as the crawl budget.

The crawl budget is determined by two elements:
Crawl capacity limit – this is the maximum number of connections that Google can use to crawl your website at the same time. It’s there to prevent Googlebot from overwhelming your website’s server with too many requests.
Crawl demand – Google calculates how much time it needs to crawl your website based on several factors like its “size, update frequency, page quality, and relevance, compared to other sites”.
If you have a large website with hundreds of thousands of pages, you’re going to want to make sure that only the most important pages are being crawled i.e. that you aren’t wasting your crawl budget on unimportant URLs.
Crawl budget management is about making sure that you’re stopping Google from crawling irrelevant pages that cause index bloat.
How To Fix Index Bloat
Index bloat occurs when Googlebot crawls too many pages of poor quality. These pages may offer little to no value to the user, be duplicated, be thin in content, or may no longer exist.

Too many unimportant and low-quality pages being crawled wastes precious crawl budget as Google spends time crawling those URLs instead of the important ones.
Our client’s English site had over 30k legacy event pages indexed – these were pages that included very little content about industry events within the client’s niche i.e. a flier for the event along with essential information such as dates and times.
Let’s look at some of the most common culprits that cause index bloat and how you can find them using a site search:
HTTP pages – websites with SSL certificates that still have HTTP pages indexed cause unnecessary duplicate content. Use the following site searches on Google:
site:yourdomain.com inurl:https:// site:yourdomain.com -inurl:https://

Pagination – likewise, paginated pages (where your content is divided across several pages) create needless duplicate content. To find them, use the following site searches on Google:
site:yourdomain.com inurl:/page/ site:yourdomain.come inurl:p=

/tag/ pages – Tag pages are like category pages where you can group similar pages together. They’re commonly used to group similar blog posts i.e. example.com/tag/sports/.
Use the following site search to find /tag/ pages indexed on your site:
site:yourdomain.com inurl:/tag/

/author/ pages – Author pages are similar to tag pages, except they are groups of pages written by the same author i.e. example.com/author/matt-diggity/
Use the following site search on Google to identify unnecessary /author/ pages:
site:yourdomain.com inurl:/author/

and non-www. pages – another common culprit is having www pages crawled and indexed when you serve non www pages (and vice versa). To find these, use:
site:yourdomain.com inurl:www. site:yourdomain.com -inurl:www.

You may also come across these kinds of pages:
Trailing slash – this is a problem if all of your URLs end with a trailing slash “/” but you have URLs without a trailing slash still indexed. For example:
example.com/with-trailing-slash/ example.com/without-trailing-slash
Duplicate pages – if you have multiple pages containing the same content i.e. domain.com, domain.com/index.html, domain.com/homepage/ etc.
Test/Dev pages – any pages from your staging site or development site should not be indexed. For example, dev.example.com or dev.example.com/category/sports/.
Miscellaneous pages – i.e. checkout pages, thank you pages etc.
There are several ways to tell Google which pages you want crawled, and which ones you don’t:
Robots.txt – Your robots.txt file is where you can specify any pages or resources that you do not want Googlebot to spend time crawling. Note that this file does not prevent Google from indexing the page – for this, you need to implement a noindex tag (next point).
Here’s the robots.txt file for my site:

You can generally find your robots.txt by accessing: yourdomain.com/robots.txt
The basic format for blocking Google from crawling your page(s) is:
User-agent: [user-agent name] Disallow: [URL string not to be crawled]
User-agent – the name of the robot/crawler that should follow the rule, you can replace this with an asterisk (*) to set a catch-all rule for all robots.
Disallow – this is the URL string that should not be crawled.
Here’s an example:
User-agent: * Disallow: /author/
The above rule prevents all robots from accessing any URL that contains /author/.
Find out more about the best practices for your robots.txt file here.
Noindex tag – if you want to prevent Google from indexing a page, add a “noindex” meta tag within the <head> section of the page you don’t want indexed. It looks like this:
<meta name="robots" content="noindex">
If you have a WordPress website, you can do this easily via a plugin like Yoast SEO.

On any page, scroll to the Advanced tab on the plugin and where it says “Allow search engines to show this Post in search results?” select No.
URL Removal Tool – The Removals tool on Google Search Console is another way to (temporarily) remove pages from Google’s index. Google recommends using this method for urgent cases where you need to quickly remove a page from Google Search.

Find out more about how to manage your crawl budget here.
XML Sitemaps
While your robots.txt file is used to prevent search engine bots from accessing certain pages, there’s another important file that you need to guide Google in the right direction regarding which pages you do want it to find and index.
That’s the XML sitemap – which our client happened to have missing from their website.
What Is An XML Sitemap & Why Is It Important?
The XML sitemap is a “map” of URLs using Extensible Markup Language.
Its purpose is to provide information about the content on your website i.e. the pages, videos, and other files, along with the respective relationships between them.

XML sitemaps are important because they allow you to specify your most important pages directly to Google.
Here’s an example of what a sitemap looks like: https://diggitymarketing.com/sitemap.xml

Providing this information makes it easier for crawlers like Google to improve crawl efficiency as well as understand the structure of your web pages. Think of it as a table of contents for your website.
By doing this, you’re increasing your chances of your web pages getting indexed more quickly.
Here’s an example of a basic XML sitemap:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="https://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://domain.com/</loc>
</url>
<url>
<loc>https://domain.com.com/blog/</loc>
</url>
</urlset>More often than not, your XML sitemap will likely look like this auto-generated one from Yoast: https://lakewoodrestorationpro.com/page-sitemap.xml

Why? Because it’s much easier to use a plugin/tool to generate your sitemap than to hardcode it yourself manually.
How To Create An XML Sitemap
There are many ways to create an XML sitemap for your website depending on your CMS.
Shopify, Wix and Squarespace – all automatically create your sitemap for you (which you can access at com/sitemap.xml) – but unfortunately, all of these are quite restrictive as you can’t make edits to them.
WordPress – to create a sitemap for WordPress.org websites, I recommend using a plugin like Yoast – it’s free and really easy to use.
Log into your WordPress dashboard and go to Plugins > Add New

Search for “Yoast SEO” and press Install Now

Then Activate

Go to Yoast SEO > General > Features

Ensure the “XML sitemaps” option is set to “On”

Your sitemap should now be automatically generated and available at either com/sitemap.xml or yourdomain.com/sitemap_index.xml
No CMS / Any Website (Screaming Frog) – for any other website, you can use a site crawler like Screaming Frog to generate your XML sitemap. If your site has less than 500 pages, then you can use the free version – otherwise, you’ll need to upgrade to the paid version.
Once you’ve installed and opened ScreamingFrog’s SEO Spider, Make sure Spider mode is selected.

Enter your domain and click Start.

The tool will crawl your pages and display a progress bar like this. Again, there’s a limit to 500 URLs, so if your site has more pages, you’ll need to purchase a license.

Once your site has been crawled, go to Sitemaps > XML Sitemap in the top menu bar.

Make sure 2xx response code is selected. If your site contains PDFs only include them if they are important and relevant.

Go to the Images tab and select Include Images. The third box (Include only relevant Images with up to 10 inlinks) will automatically be checked. Click Export.

Name your file “sitemap” it will be saved in .xml format.
There are also XML Sitemap generators like XML-Sitemaps.com where all you need to do is:
Enter your domain and click START.

The tool will crawl your pages and display a progress bar like this. Again, there’s a limit to 500 URLs, so if your site has more pages, you’ll need to upgrade to the paid version.

Once it’s complete, click VIEW SITEMAP DETAILS.

You can then download your generated sitemap by clicking DOWNLOAD YOUR XML SITEMAP FILE.

You can also click VIEW FULL XML SITEMAP to preview the generated file in a new tab: https://www.xml-sitemaps.com/download/diggitymarketing.com-29b52add/sitemap.xml?view=1
One thing to note about the methods detailed above is that they may contain URLs or pages that you do not want to be included. For example, crawlers like Screaming Frog may include paginated pages or /tag/, /author/ pages – which as you learned above, cause index bloat. So, it’s always good practice to review the generated files and make sure that only the right pages are there.
There’s also the option to code your XML sitemap manually, this is fine for small websites with very few pages but perhaps not the most efficient for massive sites.
