A/B testing shows two versions of a page, ad, or email to comparable groups of people and measures which one produces more of a chosen action. It replaces opinion with evidence. In Nepal, two things make it harder than global guides suggest: most business websites have modest traffic, and the free tool many guides recommended, Google Optimize, shut down in September 2023.

This guide covers when a website A/B test is realistic, how to size one, which tools to use now, and what to do when traffic is too low.

Step 1: Check whether you have enough traffic

The number of visitors you need depends on three things:

• Baseline conversion rate: how often visitors currently take the action.
• Minimum detectable effect: the smallest improvement worth detecting.
• Confidence and power: usually 95% confidence and 80% power.

Worked example (hypothetical numbers): a landing page converts at 3%. You want to detect a 20% relative improvement, from 3.0% to 3.6%. A standard sample size calculation gives roughly 14,000 visitors per version, so about 28,000 in total.

Baseline conversion	Improvement to detect	Approx. visitors per version
3%	+20% (to 3.6%)	~14,000
3%	+50% (to 4.5%)	~2,500
10%	+20% (to 12%)	~3,800

Figures are approximate, from the standard two-proportion formula. Run your own numbers in the free A/B test sample size and significance calculator; see also statistical significance.

The pattern matters more than the exact figures: bigger changes and higher-frequency actions need far fewer visitors. If a page gets 2,000 visitors a month, a test hoping to detect a 10% lift will never finish.

Step 2: Write a hypothesis before you build anything

A useful hypothesis names the change, the reason, and the measure:

“Showing the NPR price and delivery areas above the fold will increase WhatsApp clicks from mobile visitors, because session recordings show people scrolling to the bottom looking for price.”

Base hypotheses on evidence: heatmaps and session recordings, customer questions on WhatsApp, search terms from Google Ads, and drop-off points in GA4. Tests chosen by taste rarely win.

Step 3: Choose what to test

With limited traffic, test changes big enough to matter:

• Offer and pricing presentation: showing price versus “contact for price”, free delivery thresholds, payment options.
• Primary action: WhatsApp button versus form versus call.
• Page structure: short page versus long page with FAQs and proof.
• Headline and first screen: what the page promises and to whom.
• Form length: fewer fields, or a two-step form.

Avoid testing button colours or single words on low-traffic pages. The effect is too small to detect.

Step 4: Pick a tool

Since Optimize closed, options fall into three groups:

Type	Examples	Good for
Website testing platforms	VWO, Optimizely, AB Tasty, Convert	Visual editors, targeting, GA4 integration
Open-source / developer tools	GrowthBook, PostHog experiments	Teams with a developer; flexible and lower cost
Built into other platforms	Google Ads experiments, Meta A/B tests, email tool subject-line tests, Shopify and WordPress plugins	Testing where volume already is

Check current pricing and free tiers on each vendor’s site, as they change often. Whatever you use, make sure the test sends version data to GA4, so you can analyse results by device and source.

Microsoft Clarity is free and useful alongside any of these for heatmaps and session recordings, though it does not run A/B tests itself.

Step 5: Run the test properly

• Split traffic randomly and evenly between versions.
• Run for full weeks, at least one and ideally two business cycles, so weekday and weekend behaviour are both included.
• Avoid festival periods such as Dashain and Tihar unless you are testing a festival offer; behaviour changes sharply.
• Do not stop early because one version looks ahead after a few days. Early leads often disappear.
• Change nothing else on the page, in the ads, or in pricing during the test.
• Check mobile and desktop separately. Most Nepali traffic is mobile; a desktop-only win can hide a mobile loss.

Step 6: Read the result honestly

At the end of the planned period:

• If the difference is significant and meaningful, ship the winner and record what you learned.
• If it is not significant, the honest conclusion is “no detectable difference”. Keep the simpler or cheaper version and test something bolder.
• Check that the winning version also improved the business outcome, such as paid orders or qualified leads, not only clicks.

Keep a simple test log: hypothesis, dates, traffic, result, and decision. It stops teams from re-running the same ideas.

What to do when traffic is too low

Most small Nepali sites fall here. Options that still improve conversions:

1. Test in ads first. Google Ads experiments and Meta A/B tests compare headlines, offers, and creative with more volume than your site gets. Carry winning messages to the landing page. See landing pages in Nepal.
2. Test in email and SMS. Subject lines and offers can be tested on your customer list.
3. Measure a more frequent action. WhatsApp clicks or add-to-cart happen more often than purchases, so tests finish faster. Confirm later that they lead to sales.
4. Run before-and-after comparisons with care. Compare equal periods, avoid festivals and campaign changes, and treat results as directional, not proof.
5. Do qualitative research. Watch 20 session recordings, read a month of WhatsApp questions, or ask five customers to use the page while you watch. Fix obvious problems without testing them.

The CRO checklist lists common fixes that rarely need a test.

Checklist

• Enough conversions per variant to detect the change you care about; check with the A/B test calculator before building.
• A written hypothesis: what you change, why, and which metric should move.
• One primary metric, chosen before launch, such as form submissions or WhatsApp clicks.
• A tool that splits traffic reliably and records conversions in GA4.
• A fixed run time of at least one or two full weeks, so weekday and weekend behaviour are both included.
• No peeking and stopping early because one version looks ahead.
• Results written down, including tests that showed no difference.

Hypothetical example: A Kathmandu clinic’s booking page gets a few hundred visits a month. The calculator shows a split test would take many months to detect a realistic improvement, so the team instead fixes obvious problems (a slow page, a long form, a hidden phone number) and compares bookings before and after, noting other changes that happened at the same time.

Common mistakes

• Starting a test without checking the required sample size.
• Stopping when one version first looks ahead.
• Testing tiny changes on low-traffic pages.
• Running tests across a festival or a price change.
• Declaring a winner on clicks when sales did not change.

Key takeaways

• Google Optimize is gone; use a third-party tool, open-source option, or built-in platform tests.
• Calculate sample size first. Small sites need big changes or frequent actions.
• Write evidence-based hypotheses and test meaningful changes.
• Run full weeks, avoid festivals, and do not stop early.
• When traffic is low, test in ads and email, and use qualitative research.

How this fits the stack

Testing is part of conversion optimization Nepal, with the approach on the CRO expert page. For earlier context, see A/B testing challenges in Nepal.

Related dictionary terms

• A/B testing
• Statistical significance
• Conversion rate
• CRO

Next step

Before your next test, write the hypothesis and run the sample size numbers. If the test cannot finish in four to six weeks, change the plan. For help setting up a testing programme, scope work through hire packages or contact.