Search engine optimization (SEO) requires constant testing to be effective. If you don’t understand what changes drive growth (or why visitors respond the way they do), you’ll waste your time guessing at solutions while your competitors leave you in the dust.
In this article, as someone with significant SEO expertise, I will provide a comprehensive understanding of SEO testing, the challenges of acquiring actionable data from Google Analytics, and how to conduct SEO tests that effectively improve your crucial performance indicators.
Upon concluding this article, you will not only recognize the transformative potential of SEO testing but also benefit from three of my recent test results that defy traditional SEO principles.
To begin, let’s briefly define SEO testing and examine its different manifestations.
Table Of Contents
- What is SEO Testing?
- What Impact Does an SEO Testing Have on SEO?
- What are the Challenges of Performing Accurate SEO Tests?
- What’s the Solution?
- Best Practices for SEO Testing
- Recent Findings in SEO
- What Types Of SEO Tests Are There?
- Success Through SEO Testing Is Within Your Reach
What is SEO Testing?
SEO testing refers to the experiments you perform to measure how search engines and users react to your site.
There are many tools you can use to get data about each web page (such as Surfer, Ahrefs, and other free tools), but no Google Analytics tool or SEO checker can tell you what effect changes will have.
The purpose of SEO testing is to manipulate different search traffic factors to determine if the changes result in an increase or decrease in organic traffic or your position in search results.
There are many ways you can run SEO tests, including:
- Creating different sets of content
- Adjusting keywords, meta descriptions
- Trading out different links
Like your high school science classes, SEO testing relies chiefly on utilizing experimental and control groups’ concepts.
The experimental groups are the cases (pages or entire sites) that experience your test variable (such as trying a new link strategy).
The cases in the control groups are untouched. Their rankings and organic traffic will give you the reference point to determine whether or not the experiment worked (or not).
Instructions on how to create great SEO tests will be covered a little later…
But first, it may help you precisely understand what kind of benefits you can enjoy if you do the work to develop reliable SEO tests.
What Impact Does an SEO Testing Have on SEO?
The impact that testing has on your SEO strategy is immeasurable—no pun intended. It provides all of the following concrete benefits:
Testing Gives You the Power to Be First
Whenever you run an SEO test, you put yourself in a limited category of people on the industry’s cutting edge.
Some of the most effective SEO tactics ever discovered have been neutered by Google search console shortly after they reached the wider community.
If you’re a tester, you get to be in the rare company of people who use the latest techniques while still being powerful. That enables you to operate in the most profitable niches.
Testing Helps You Develop a Ranking System That Works
Repeated testing gives you the power to separate what you can and can’t know about SEO, based on experience.
This allows you to skip time-wasting techniques and move directly to those with the most significant, most reliable SEO performance impact.
That makes you a far more efficient SEO. This ability is one of the critical differences between amateur SEOs, and the kind who can successfully dominate even the most competitive niches.
Testing Can Make You Algorithm-Proof
If you test SEO changes often enough, you can develop a second sight for the trends that are guiding search engine updates.
You’ll spot trends before they’re officially announced, and adjust your practices to avoid penalties.
What are the Challenges of Performing Accurate SEO Tests?
The biggest challenge with SEO testing is collecting untainted data. Your results become tainted when they become affected by outside factors or wrong assumptions.
These obstacles make it impossible for you to say which results are a product of your experiments.
Challenge #1: You Must Make Sure Your Variables Are Stable
To understand what’s happening on each web page in the experiment, you need to isolate what changes affect your data. Stabilizing your variables means working to prevent changes from happening that are outside of your test.
For example, if you are trying to measure the effect of content upgrades, you need to prevent your link profile from changing during the testing phase.
If a lousy link gets disavowed or a useful link gets added, your site’s changes could easily mask any result of the content upgrades.
Challenge #2: You Need to Make Test Cases as Similar as Possible
The first thing you need to do is make each experimental and control group nearly identical. This will allow you to start measuring the changes once you start altering the test site.
When you come back to check website changes, you’ll have a better idea of how they happened.
Here are just a few examples of why you’ll have trouble with that:
- No two web pages can have the same page URL, yet the URL is itself a ranking factor.
- Those links still won’t be the same age if you manage to get the same links for both sites. It’s also highly unlikely that you’ll be able to gather them at an identical velocity.
- If you create two identical sites for your SEO split testing, one is now filled with duplicate content. Duplicate content is a factor that Google can detect and may choose to penalize. In some cases, google search console may refuse to index the new material at all.
Challenge #3: You Need to Expect the “Random Ranking Factor”
The random ranking factor is a phenomenon coined by SEO Terry Kyle (founder of WPX hosting). It is best illustrated with an example…
Imagine that you launch five identical sites or landing pages on the same day. They are in the same niche, use corresponding keywords, and employ the same design style.
You will likely notice the following:
- Three sites tend toward average performance
- One site tends toward exceptional performance
- One site tends toward inexplicable sluggishness
This is not a precise rule, but many SEOs have recorded the effect over the years.
For reasons that are nearly impossible to measure efficiently, some sites simply behave as if they are blessed while others act as if they are cursed.
What’s the Solution?
The solution is to simply increase the number of test cases.
One control page is never going to be enough, and neither is one SEO test page.
Adapt to this challenge by creating larger groups for each test group and control group. Have 10 URLs in the control group, and 10 in the experimental group…or better yet, 50.
Then, you’re going to measure the results by taking the average of the changes in both groups.
Maximizing the number of test cases helps to resolve our three biggest challenges in the following ways:
For example, if we have 50 test cases, then we know that a single random backlink hitting one of them isn’t going to throw off our test.
For example, it helps you control all the minor differences caused by URLs, age, and other factors that play a role in SEO.
We now have enough information to identify sites that behave differently. This will help prevent us from either getting overconfident or giving up on a great technique just because it failed randomly.
Let’s look at all this through an example.
Pretend you have a blog in the wellness niche, and you want to test the effect of double-counting keywords in the title tag.
If you have 30 posts with organic traffic, a simple experiment might involve creating two groups:
- 15 posts preserved as a control group (titles unchanged)
- 15 posts to test your idea of an ideal title
If you’re curious about how a test like this might play out, don’t miss the ‘clickbait title’ test that will be revealed near the end.
For now, let’s focus on what roles these groups play:
- Group 1 will help you follow any changes that happen outside your experiment. If your control group experiences sudden SEO shifts, then you know the same effect in your other groups isn’t a result of your SEO tests.
- Group 2 will test your hypothesis. Once you’ve subtracted any SEO shifts from the control group and random group—you’ll have an idea of the effects of your real changes.
Naturally, testing like this is going to depend on the funding you have. If you can afford it, testing ten sites instead of 5 (or 20 instead of 10) will allow you to create a more accurate average.
However, there will be more room for error with those numbers than most experienced testers will tolerate.
The more web pages you can create for a test, the more effectively you can cancel out noise. Sometimes, the most effective way to cancel out noise is to remove important factors altogether.
Best Practices for SEO Testing
Follow these rules to draw some better data from each SEO test:
Set Aside Enough Time
Beyond your money budget, you need to put a significant time budget in place. Once again, the more time that you can give your experiment, the more sure of the accuracy of any data that you collect. Some results (especially offsite related) may take months to yield a difference.
Exercise Attention to Detail
If you can’t measure it, you can’t manage it. The more information you’re tracking, the more awareness you’ll have of how sites are different, so you must be tracking even when you’re not testing.
I can illustrate this with a real-life example. At one point, I had a great site in the Brazilian testosterone niche. One week, we suddenly jumped from page two to page one.
I was only able to figure out why, because of my tracking. It turns out, some links I snagged from a local citation package were worth a lot more than I imagined it would be.
Thanks to my research, I caught the factor. Thanks to that insight, I now had a new strategy to use for many other sites.
Recent Findings in SEO
Nothing illustrates the impact of SEO testing, like having access to information most of the industry doesn’t yet possess.
The following experiments were performed as personal research. What follows are the theories, tests, and findings from research built using the rules above.
Test #1: Measuring the Effect of Inserting NLP-Friendly Sentences Into the Content
This test involved a look into whether implementing Natural Language Processing (NLP)-friendly language would change the way that Google responded to a site. NLP refers to the ability of software to interpret and manipulate language. This technology is improving but still limited.
Theory: If we can make it easier for NLP to process our content, Google will be more likely to reward us with snippets and better rankings.
We aimed to optimize our content for NLP by doing the following:
- Echoing back the question
- Giving the answer
- Mentioning the correct unit
For example, here’s a typical way of answering a question to another human, compared to the NLP-friendly style.
Question: What is the best temperature to brew beer?
Standard answer: “68 to 72 degrees.”
NLP-friendly answer: The best temperature to brew beer is between 68 to 72 degrees Fahrenheit (20 to 22 degrees C).
|# of featured snippets
Implications: NLP-friendly language measurably improves rankings and attracts snippets. The number of snippets we controlled doubled after our work.
Test #2: Is It True That Clickbait Titles Stopped Working in 2024?
Theory: People have started to avoid clickbait titles, and may prefer more straightforward descriptions of what they’re getting.
We understand clickbait meta titles as titles that tease exciting results, but may not explain the article’s content:
Clickbait title: 7 Strategies that will increase your organic search traffic by 200%
Straightforward title: How to increase organic traffic to your blog
Implications: The new titles had far fewer clicks. People are still more responsive to clickbait titles than they are too straightforward descriptions of the content.
This test examined whether irrelevant internal links were noticed or acted upon by Google. The SEO site that inspired this experiment was a beard oil shop interlinked freely with pages that were slightly, but not directly, related (such as beard shampoo).
Theory: Irrelevant internal links hurt the potential to rank
We understand “irrelevant” internal links as those that link to pages with unrelated topics. We were pretty strict in defining links as irrelevant.
For example, for the topic of Beard oil, we purged links from pages that covered:
- Beard shampoo
- Beard wax
- Beard balm
These topics are related to one another, but they don’t serve the same need, so we considered them irrelevant for testing purposes.
The test involved creating internal links to the irrelevant pages from the test pages. 80 keywords were tracked to determine changes
|% of keywords that went down
Oddly enough, the pages that were given links from irrelevant pages experienced short-term ranking and organic traffic improvement. Only 2% of all keywords went down.
Implications: Internal links from irrelevant pages are not that harmful, and may even result in short-term improvements.
This video outline everything you need to know about interlinking, watch this video.
What Types Of SEO Tests Are There?
There are several types of SEO tests that can be performed to optimize a website and improve its ranking on the google search console.
1. SEO Serial Testing
Serial testing involves modifying all of a website’s pages simultaneously and observing the results. This is not recommended for three reasons:
- If the update has a negative effect on SEO, it affects all of your pages, not just a select handful.
- It is often more time-consuming to implement site-wide changes and considerably more time-consuming to undo them.
- It does not account for seasonality or uncontrollable circumstances.
2. SEO Time-based Testing
In time-based testing, you modify a single page and observe its performance over time. It is unwise to imply causality with a sample size of one; hence, we do not advocate this form of SEO testing approach.
3. SEO A/b Testing (Split Testing)
A/B testing (also known as split testing) entails modifying certain web pages but not others. Then, you compare the performance of the modified pages to that of the unmodified ones. The group of unaltered pages is referred to as the control group, while the group of altered pages is referred to as the variation group.
The SEO split testing form of SEO testing is recommended because:
- If the adjustment adversely affects SEO, only a tiny fraction of pages will be affected.
- It is less time-consuming than serial testing since just a subset of pages must be modified.
- Seasonality and elements outside of your control are accounted for since they affect both sets of pages.
Success Through SEO Testing Is Within Your Reach
SEO testing is one of the most effective ways to improve your sites and your skill as an SEO. You can learn the most effective techniques by understanding the role of testing, meeting the challenges head-on, and setting up your SEO tests the right way.
As the SEO tests above showed you, the conventional wisdom about what works for SEO is often flawed, or only applicable in some cases. SEO checker or Google Analytics tools will only tell you where your website is now. Testing empowers you to see where it could go.
By taking the time and expense of testing seriously, you can discover a more productive path for your sites, and plan for the changes google search console
make. Armed with the knowledge that only you possess, you can carve out a niche for yourself in the SEO world.
If you want to learn more about tested SEO tactics and the strategies that work, you can join The Affiliate Lab. You’ll get access to all my private SEO tests as soon as I have results, and the chance to discuss your own with the most exclusive affiliate SEO Facebook group around.
Got Questions or Comments?
Join the discussion here on Facebook.