Testing your landing pages every week: the right A/B testing cadence
Partner · Strategy, advertising and measurement
Testing a landing page every week has become possible on the production side: with AI, a page variant can be prepared within the day rather than over several weeks. The limit has moved to traffic. To reach a conclusion, an A/B test needs a number of visitors calculated in advance and at least two full weeks. On a page that gets a few thousand visits a month, the realistic pace is closer to one test a month, on significant changes. The week remains the team's unit of work, not the length of a test.
- With AI, the time it takes to produce a variant no longer sets the pace of testing: traffic and statistical rigour set it.
- A test is sized before it launches. At a 3% conversion rate, detecting a 30% lift takes about 6,500 visitors per version.
- A test runs for at least two full weeks, even when traffic is plentiful.
- The more often you test, the greater the risk of keeping a false winner: the stop date and the metric are set in advance.
- A win is measured in sales and qualified leads, not in clicks.
On this page
A/B testing cadence
A/B testing cadence is the number of tests a business launches and carries through to a reliable conclusion over a given period, such as a month. It depends on the time it takes to produce a variant, the traffic of the page being tested and decision discipline (a written hypothesis, a calculated sample size, a fixed stop date). AI eases the first constraint: the other two set the pace.
Production no longer limits the pace
For years, a page variant required a designer, a developer and a spot in the queue. One test per quarter was often the maximum, for lack of production time more than lack of traffic. Our pillar article on what AI changes in website development describes how that delay has shrunk: a written instruction can produce a new version of a page within the day.
Testing tool vendors see the same shift. In a 2026 report covering 173,000 experiments run by more than 1,200 companies, Optimizely says its customers who use its AI increased their experiment velocity by 70.2%. This is vendor data, which shows an association, not a cause.
The same report points to the constraint that remains. Without the vendor's statistical accelerator, only 22.3% of experiments reached a conclusive result. Producing more variants is pointless if the page lacks the traffic to tell them apart. Traffic sets the pace, not production.
On the tools side, Google Optimize, the free testing tool that came with Google Analytics, has not been available since September 30, 2023. Google chose to invest in integrations with third-party tools such as AB Tasty, Optimizely and VWO.
How much traffic it takes to reach a conclusion
Even if the two versions of a test were identical, their conversion rates would never be exactly equal: chance always creates a gap. The sample size is the number of visitors needed for a real gap to stand out from that one. Our article on A/B testing explains the principle. Here, we turn it into a calendar.
Three figures go into the calculation:
- Your starting conversion rate. The lower it is, the more visitors you need.
- The gap you want to be able to detect. A 50% lift shows up quickly. A 10% lift takes about twenty times more traffic.
- The level of rigour. The usual calculation targets 95% confidence (a one-in-twenty risk of seeing an effect that does not exist) and 80% power (four chances in five of seeing a real effect).
Any sample size calculator does this calculation. Here are its results for a few common cases, rounded.
| Starting rate | Lift to detect | Visitors per version | Conversions per version |
|---|---|---|---|
| 2% | 50%, to 3% | About 3,800 | About 75 |
| 2% | 20%, to 2.4% | About 21,100 | About 420 |
| 3% | 30%, to 3.9% | About 6,500 | About 190 |
| 3% | 20%, to 3.6% | About 13,900 | About 420 |
| 5% | 20%, to 6% | About 8,200 | About 410 |
| 10% | 10%, to 11% | About 14,800 | About 1,480 |
The number of conversions matters more than the number of visitors: to detect a 20% lift, you need about 400 conversions per version, whatever the starting rate. A two-version test needs double the figure in the table, a three-version test triple.
A realistic cadence based on page traffic
The traffic that counts is that of the page being tested, not of the whole site. The table assumes a page that converts at 3% and a two-version test. These are orders of magnitude, to redo with your own figures.
| Monthly visits to the page | What a test can detect and how long it takes | Realistic pace |
|---|---|---|
| About 1,000 | A 50% lift would take about five months | No A/B test: deliberate fixes, measured before and after |
| About 5,000 | A 50% lift in about a month. A 30% lift in more than two and a half months. | One test a month, on significant changes |
| About 25,000 | A 30% lift in just over two weeks. A 20% lift in about five weeks. | One test every two to five weeks on that page |
| 60,000 and more | A 20% lift in two weeks | One variant a week, on different journeys |
One page a week is therefore within reach of a high-traffic site, or of a business that tests several pages in parallel: the page behind its Google Ads, a service page, the product page of its best seller. Elsewhere, one test a month carried through to the end is worth more than four tests stopped too early. Every week, a hypothesis gets prepared, a variant gets produced, a test is launched or read. The test itself lasts as long as the calculation requires.
Do not lower the bar to go faster. Moving from 95% to 80% confidence shortens the test. In return, a version that adds nothing can then be declared the winner one time in five.
What to test first
When producing a variant becomes easy, the temptation is to test everything. Yet each test uses up part of a limited pool of traffic. The order is therefore set by the expected effect on sales, not by how easy the variant is to produce.
Changes big enough to be detected
On a page with a few thousand visits a month, only a 30% to 50% gap shows up within one to three months. Test a new value proposition, a shorter form or another way of presenting the price, not a shade of colour.
A hypothesis written before the variant
For example: “If we show the starting price at the top of the page, quote requests should go up, because visitors who call ask that question first.” According to Optimizely, well-prepared experiments produced a winner 58% of the time, compared with 35% for poorly prepared ones.
Clarify the value before anything else
In the same report, hypotheses that aim to clarify the value of the offer appear in more than one experiment in four and win 41% of the time. It is a good starting point for a small business: what you offer, for whom and why you rather than someone else.
One test at a time on the same journey
Two simultaneous tests on pages that follow each other contaminate each other. Parallel tests should therefore target different journeys.
How to avoid false winners
A false winner is a version declared better when it is not. At 95% confidence, a version that adds nothing is declared the winner one time in twenty. If none of the fifty versions tested in a year were better, two or three would still be declared winners.
Set the stop date before launch
Stopping at the first favourable gap is the best-known trap. In 2015, Optimizely claimed that its sequential method, designed so results could be checked at any time, cut the risk of wrongly declaring a winning or losing version from 30% to 5%. Without a tool of that kind, you read the result on the planned date, not before.
Cover full weeks
Visitor behaviour varies by day, in B2B as in B2C. Two full weeks is a minimum, even if the planned number of visitors is reached sooner.
Check how visitors are split
If one version gets 55% of visitors in a test planned as fifty-fifty, something is wrong: a redirect, the cache or bots. The result is not read until the cause is found.
Confirm big changes
The gap measured during a test is often smaller once the version is in place. Before a major decision (a redesign, a price change), a second test confirms the first.
Measure the sale, not the click
A variant that gets more clicks can produce fewer sales if it attracts less qualified visitors. According to Optimizely, more than 90% of experiments target five metrics: button clicks, pageviews, checkout, registration and add-to-cart. The decision should be made on the sale or the qualified lead.
Measurement: GA4, key events and sales
Google Analytics 4 calls the actions that matter most to the business “key events”: a quote request, a call, a purchase. Google keeps the word “conversion” for the actions used to measure advertising campaigns. For a test, the key event is checked before launch: a form counted twice would skew both versions at once.
- Each visitor carries the name of their version. The testing tool or the page sends the version seen to GA4, so key events can be compared version by version.
- The lead is followed through to the sale. In B2B, a completed form is not a sale. Link each request to its outcome in your CRM: a version that produces fewer leads but more sales is the winner. Our article on conversion tracking shows how to make that link.
- Consent affects both versions. In Quebec, Law 25 governs analytics cookies. Some visitors refuse them, which reduces the data visible in GA4. If the split is random, both versions lose the same share of visitors: the comparison remains valid, but it takes longer to reach a conclusion. Our article on analytics consent under Law 25 details that gap.
Organizing a week of testing
The weekly rhythm rests on a short routine that is always the same.
Read the tests that have reached their end date
Only tests whose stop date has passed get read. The others stay closed.
Choose the next hypothesis
The next one is the one that promises the biggest effect on sales for the available traffic. The sample size calculation tells you whether it can reach a conclusion.
Produce the variant
With an AI-managed website, the variant is written as an instruction and goes through the checks of a regular page. This is the step that used to take weeks.
Approve, then launch
A person reviews the variant (copy, price, legal notices, French version) before it goes live. The stop date is recorded at launch.
Log the result
Each finished test goes into a log: the hypothesis, the versions, the figures, the decision. A losing test keeps you from repeating the same mistake six months later.
What an AI-managed website changes about the pace
On an AI-managed website, a page variant is an instruction written in your own words to Claude Code (Anthropic's programming tool) or to ChatGPT Codex (OpenAI's programming agent). The variant is prepared on a working copy, goes through automated checks (links, tags, structured data), then waits for approval. As of October 20, 2026, 28 automated checks blocked any faulty publication on falia.co.
This way of working changes three things for testing:
- The variant is a real page. It can live at its own address, with no script changing the page in the visitor's browser. You still need a testing tool or a server rule to split visitors. Google sets its conditions: the variant points to the original with a canonical tag, the redirect to the variant is temporary (302) and the test ends as soon as the data is sufficient.
- Removal is as fast as creation. A losing variant is removed with one instruction, along with its redirect, as Google asks at the end of a test.
- The winner applies everywhere. An argument that convinces better can move, through the same instruction, to the other pages of the service and then to the French version.
Traffic, however, does not change: an AI-managed website produces variants faster, it does not add visitors. The hypothesis and the reading of results remain human. AI executes, a senior signs off. On WordPress, plugins that open up to AI agents allow part of this pace. Our guide to AI in WordPress 7 presents them.
Five questions before speeding up your tests
- How many visits and conversions does the page to be tested get each month?
- What lift can we hope to detect in a month with that volume?
- Which metric will we decide on: the lead, the qualified lead or the sale?
- Who writes the hypotheses and who approves the variants before they go live?
- Where do we log the results, winners and losers alike?
AI has made variants fast. Your traffic decides how many you can draw conclusions from.
Three months of fixes and tests on your priority pages, with a report with figures on day 90: that is the sprint of the goal Improve your site's conversion.
Prefer to move month by month? Our monthly CRO engagement covers one variant or one change a month on whatever weighs the most in your conversions, measured over 30 to 60 days depending on your volume, from $1,295 a month.
Frequently asked questions about A/B testing cadence
Can you really test one page a week?
Yes, if the page gets tens of thousands of visits a month or if you test several pages in parallel. Each test still runs for at least two full weeks. Otherwise, one test a month on a significant change is more realistic.
How many visitors do I need to conclude a test?
It all depends on your conversion rate and the gap to detect. At 3%, detecting a 30% lift takes about 6,500 visitors per version, or about 13,000 for a two-version test. To detect a 20% lift, count on about 400 conversions per version.
My site has little traffic: can I still test?
Not with a classic A/B test, which could not reach a conclusion. Apply the obvious fixes, measure before and after, then watch a few users work through the page. These methods give clues, not proof.
What tool replaces Google Optimize?
Since September 30, 2023, Google has pointed to third-party tools integrated with Google Analytics, such as AB Tasty, Optimizely and VWO. On an AI-managed website, a variant can also live at its own address, with the split handled on the server.
Does an A/B test hurt my SEO?
Not if you follow Google's rules: show Googlebot what visitors see, link variants to the original with a canonical tag, use a temporary redirect (302), then remove the test as soon as the data is sufficient.
Can AI pick the winning version for me?
It can produce the variants, run the sample size calculation and summarize the figures. The decision stays with the business: only it knows whether one more lead is worth one less sale, or whether an argument truly matches what you offer.
- Optimizely, Lessons learned from 173K experiments, 2026 report (vendor data), accessed in 2026.
- Optimizely, Leonid Pekelis, Statistics for the internet age: the story behind Optimizely's new Stats Engine, January 20, 2015.
- Google Analytics Help, [Sunset September 2023] Google Optimize, accessed in 2026.
- Google Analytics Help, Conversions vs. key events in Google Analytics, accessed in 2026.
- Google Search Central, Minimize A/B testing impact in Google Search, updated December 10, 2025, accessed in 2026.
Gabriel almost always takes your first call and carries out your audit. He builds the strategy starting from your growth goal: where to put your budget, which market to test and how to connect each lead to a real sale in your CRM. He mainly leads engagements for three goals: Optimize the profitability of your digital campaigns, Develop a new market, and Generate demand and growth. With Geneviève, he also works on organic search (SEO), AI visibility (GEO) and conversion rate optimization (CRO). The sales a Google Ads or Meta Ads campaign brings in depend on the page that receives the click. He writes mainly about marketing strategy, paid advertising and measurement.
About Falia →
Strategy
SEO
Paid advertising
Conversion
AI visibility
Web design