A/B Testing Doesn't Need to Track People

A/B testing is a coin toss with a scoreboard. Half your visitors see one version of a page, half see another, and you count which one produces more signups, sales or clicks.

The software around it has made that sound far more complicated, and far more invasive, than it needs to be. You don't need to follow people around the web to learn whether a shorter form works better.

Here are five website changes I'd test, what to measure for each, and when your site doesn't have enough traffic to bother.

Do you really need all those form fields?

Contact forms have a habit of growing. Someone wants a phone number. Sales wants company size. Marketing adds a budget field. After a year, asking a question feels like applying for a mortgage.

Test the current form against a shorter version that keeps only the fields you need. Measure completed enquiries, the goal you already track, not clicks on the submit button.

This is a good first experiment because the question is easy to understand, the change is reversible, and the result settles an argument almost every company has had. My bet: the short form wins on volume, and sales learns to ask for company size on the call.

On most sites the optional fields can be hidden with a few lines of CSS in the theme, no rebuild needed. Preview the short version on your own device, start the test, and read the result in the dashboard you already use.

Should annual pricing appear first?

Your pricing page has a monthly/yearly toggle. It starts on monthly, because that's the lower number. Try starting it on annual, with the discount visible, and measure trials started.

This is the pricing test I'd run before any other. It changes what people anchor on, not what they pay, so the effect is often large, and large effects settle quickly. The risk is small too. If annual-first loses, you flip the toggle back and nobody noticed.

What I wouldn't test here is the price itself. A price test on a normal-sized site takes months, and the customers who notice two prices don't forget it.

Does your landing page keep the ad's promise?

You pay for a click. The ad says "Free migration in 48 hours". The page says "The analytics platform for modern teams". The visitor came for the first thing and landed on the second, and some of them leave.

Test the current headline against one that repeats the ad's promise, word for word. Measure the lead form, and run the test only on the page the campaign lands on. Every visitor here cost you money, so this is the test that pays back fastest.

One warning. A pushier headline can win the form and lose the reader. So Clickport watches engagement on every test as a second number, and says so when the winner reads worse:

It never stops a test for you. It puts both numbers on the table and leaves the call to you, which is where I think that call belongs.

Does a proper product box earn more clicks?

If your site earns from product recommendations or affiliate links, you've had this argument. Does a box with a picture, a price and a button beat a plain text link inside the paragraph? Test the two against each other and measure clicks out to the product.

I like this one for content sites because the thing you measure happens all the time. A newsletter signup might convert 2 readers in 100. A product box might get clicked by 20. Which means the box test ends roughly ten times sooner. The more often the goal fires, the sooner you know, and that's why I'd measure the click and not the eventual sale.

Both versions can live in the page, with a style rule showing one or the other. The docs have the three lines.

Is the whole redesign better?

Sometimes the change is the whole page: new layout, new copy, new order of sections. Don't try to squeeze that into a style tweak. Build the new page as a copy at its own URL, and let the test send half your visitors there. Measure whatever the page is for: signups, purchases, demo requests.

This is the test most likely to surprise you, in both directions, and I've been on the wrong side of it. Redesigns feel like improvements to the people who made them. Visitors don't always agree, and a split-URL test is the only fair way to find out before you replace the original.

If you test two URLs, keep the original as the real page. Google's guidance is short: use a temporary redirect, mark the copy as a copy with a canonical tag pointing at the original, and never show search engines a different page than visitors see. The split-URL section of the docs has the one line to add.

How Clickport keeps the test honest

Clickport gives each visitor one version at random and remembers that choice on the same device. Every signup, sale or click is then counted against the version they saw. The metric is a goal you already track, so there's nothing new to set up and no second script on your site.

Once enough visitors have passed through, and the test has run for at least a week, Clickport tells you whether one version probably won or just got lucky. Until then, it says so:

The short form is ahead by 3.5 points and the tool still won't call it. That's deliberate. Checking a test every morning and stopping the first time it looks good is the most common testing mistake there is. Evan Miller showed years ago that this habit pushes the false-winner rate from 5% to about 26%. That means one in four of those "wins" would be noise. So there's no verdict before 7 days and 1,000 visitors per variant, however good day three looks.

When the verdict comes, it comes in words: "short won at 97% chance to win", or "control kept". A tie is a real result too. If the two versions sit level after thousands of visitors, the change doesn't matter for this goal. Keep whichever you like and test something bigger.

When testing is a waste of time

Small improvements need a lot of visitors to prove. This is the part most tools don't tell you before you start. At a 4% conversion baseline:

A page with 40 visitors a day can't reliably detect a small improvement. In practice you could still be waiting in 2028. Either make a bigger change, measure something that happens more often, or trust your judgment and move on.

What I wouldn't test at all: button colours, one word in a headline on a page nobody visits, anything where you already know the answer. Test the pages that get traffic, and changes big enough to matter.

What about privacy?

Clickport sets no cookie and sends no persistent visitor identifier. The version choice stays on the visitor's device, and only the version name travels with the pageview, like any other property. Consent requirements for browser storage differ by country, so check the rules that apply to your site. The cookie banner guide covers the main ones.

One limit, stated plainly. The choice is remembered per device, so the same person on a phone and a laptop can see different versions. For a marketing site that's fine. For logged-in product experiments you want a different kind of tool.

Start with the argument you keep having

Every company has one page it argues about. That's your first test. Pick the page with traffic, change one thing big enough to notice, and let the scoreboard settle it.

Experiments is included in every Clickport plan, up to 20 per site. You can try it free for 30 days. If you'd rather write the split yourself, the 20-line JavaScript version still works.

Frequently asked questions

Do you need cookie consent for an A/B test?

A tool that sets a cookie or sends a visitor identifier to a vendor needs consent in most of Europe. A tool that stores only the version choice on the device stores far less, but rules on browser storage differ by country. Check the rules for your site before you rely on any tool's promise, ours included.

What replaced Google Optimize?

Nothing from Google. Google Optimize shut down on September 30, 2023, and Google pointed users at paid partner tools. Two routes remain: a standalone testing tool with its own script and price, or an analytics tool with experiments built in.

How much traffic do you need for an A/B test?

Enough visitors on the tested page to detect the lift you expect. At a 4% baseline, a +50% lift needs about 1,900 visitors per variant, +20% needs about 10,300, and +10% needs about 39,500. That means a page under 100 visitors a day can only test big changes.

How long should you run an A/B test?

At least 7 days, so the test sees every weekday once, and until each variant has the visitors the lift needs. Don't stop on the first good-looking day. Clickport's minimum run is 7 days and 1,000 visitors per variant.