What should a growth experiment template include?

At minimum: a written hypothesis, one primary metric, a baseline, the smallest lift worth acting on, the required sample size, a fixed end date, and a decision rule that says what you will do for a win, a loss and an inconclusive result. Fill it out before launch, not after.

How many experiments should a small marketing team run at once?

One to two per month is enough for a team of two to four people. Fewer tests with enough traffic to reach a clear answer beat a dozen underpowered ones that all end in 'inconclusive'.

How do I calculate sample size for a marketing test without a statistician?

Use the rule of thumb: visitors per variant is roughly 16 times p times (1 minus p), divided by the square of the absolute lift you want to detect, where p is your baseline conversion rate. It assumes about 80% power and a 5% false-positive rate, which is fine for most marketing decisions.

What if my traffic is too low to A/B test?

Test bigger changes, so the lift you need to detect is large, or move the test to a higher-volume step such as email or ads. If neither works, skip the A/B test and use customer interviews or a before-and-after comparison, and label it as weaker evidence.

Growth Experiment Template for Small Marketing Teams

A growth experiment template is a one-page document you fill out before launching any test. It forces four decisions up front: the hypothesis, the single metric that judges it, the sample size you need, and the rule that tells you what to do with each possible result. Small teams that use it run fewer tests, but nearly every test ends in a decision instead of a shrug.

Why small teams need a stricter template than big ones

A large growth team can run 40 tests a quarter and accept that most will fail or wash out. They have the traffic and the analysts to absorb the waste. You probably have two to four people, one landing page that gets a few thousand visits a month, and a founder who wants to know what is working by Friday.

At that scale, the expensive failure is not a losing test. It is the test that ends with "conversion went up a bit, maybe?" That result costs you three weeks of design and dev time and teaches you nothing. You cannot ship it, you cannot kill it, and everyone remembers it differently.

The fix is boring: decide everything that can be decided before the data arrives. The rest of this article gives you the template, a worked example with real numbers, and the mistakes that most often break it.

Template

Copy this into a doc, a Notion page or a spreadsheet tab. Keep it to one page. If a field takes a paragraph, your idea is not sharp enough yet.

EXPERIMENT: [short name]
OWNER: [one person]   START: [date]   END: [fixed date]

1. PROBLEM / OBSERVATION
   What did we see, and where? (data point, not opinion)

2. HYPOTHESIS
   If we [change], then [metric] will [move by how much]
   for [audience], because [reason grounded in evidence].

3. PRIMARY METRIC
   Name, exact definition, data source.
   Baseline (last 4 to 8 weeks): ____
   Guardrail metric (must not get worse): ____

4. MINIMUM EFFECT WORTH ACTING ON
   Smallest lift that pays for the work: ____

5. SAMPLE SIZE AND DURATION
   Per variant: ____   Total: ____
   Weekly traffic: ____   Weeks needed: ____

6. DECISION RULE (written before launch)
   WIN: ____ -> we will ____
   LOSS: ____ -> we will ____
   INCONCLUSIVE: ____ -> we will ____

7. COST
   People-hours, ad spend, opportunity cost.

8. RESULT (filled in after END date only)
   Numbers, decision taken, what we learned, next test.

Sections 2 through 6 are the ones that matter. Sections 1, 7 and 8 keep a record so you stop repeating tests six months later.

How to fill out each field

Hypothesis

A useful hypothesis is falsifiable and names a mechanism. "Improve the pricing page" is a task. "If we replace the feature grid with three customer outcomes, trial signups from pricing-page visitors will rise from 3.0% to 4.2%, because win/loss calls show buyers can't tell which plan fits them" is a hypothesis.

The "because" clause is the part teams skip. Without it, a win teaches you nothing you can reuse. If you need a source of evidence, customer discovery interviews and a customer journey map are the cheapest places to find the drop-off worth testing.

Metric

Pick one primary metric. Not two, not a "basket." Choose the metric closest to money that you can measure within your test window. For a landing page, that is usually signup or purchase rate, not clicks and not time on page.

Add one guardrail, a metric that must not degrade. If you test a more aggressive offer, the guardrail might be refund rate or trial-to-paid conversion. A signup lift that destroys lead quality is not a win.

Write the exact definition. "Conversion rate" means nothing until you say: unique visitors who land on this URL and complete signup within the same session, excluding internal traffic.

Minimum effect worth acting on

This is the most underused field. Ask: what is the smallest lift that justifies the work and the rollout? Do the math in dollars. If a 5% lift is worth $400 a month and the build costs $6,000, a 5% lift is not worth detecting, and you should not design the test around it.

This number also drives your sample size, which is why it comes before it.

Sample size

You do not need a statistician. For a conversion-rate test at roughly 80% power and a 5% false-positive rate, use this rule of thumb:

Visitors per variant ≈ 16 × p × (1 − p) ÷ d²

Here p is your baseline conversion rate and d is the absolute lift you want to detect (a move from 3.0% to 4.2% is d = 0.012). Any online calculator will give slightly different numbers. The rule of thumb is close enough to tell you whether a test is feasible.

Here is what it produces for common baselines:

Baseline conversion	Detect a 50% relative lift	Detect a 25% relative lift
2%	about 3,100 per variant	about 12,500 per variant
5%	about 1,200 per variant	about 4,900 per variant
10%	about 600 per variant	about 2,300 per variant

Read the table before you pick a test. A site with 2% conversion and 4,000 monthly visitors cannot reliably detect a 25% lift in under three months. Your honest options are to test a bigger change, test somewhere with more volume, or not A/B test at all.

Decision rule

Write three outcomes and a concrete action for each:

Win: the variant beats control by at least the minimum effect, at 95% confidence, with the guardrail intact. Action: ship to 100% and schedule the follow-up test.
Loss: the variant is worse than control, or the guardrail breaks. Action: revert and log the learning.
Inconclusive: neither condition is met at the end date. Action: keep control, do not extend the test, and do not re-run the same idea with minor tweaks.

The inconclusive line is where most teams cheat. If you do not pre-commit to "keep control," you will extend the test until something looks good.

Worked example: a pricing-page test

A three-person marketing team at a B2B scheduling tool (about $1.1M ARR) wants more free-trial signups. Their pricing page gets roughly 6,000 visits a month and converts to trial at 3.0%, so about 180 trials a month. About 20% of trials become paying customers, and a customer is worth roughly $600 in first-year revenue. That makes each trial worth about $120.

Hypothesis: Replacing the feature grid with three outcome-based plan descriptions will lift trial signup from 3.0% to 4.2%, because win/loss calls show prospects can't tell which plan fits them.

First sample-size pass. The team's instinct is to detect a 25% lift (3.0% to 3.75%). The formula gives 16 × 0.03 × 0.97 ÷ 0.0075², or about 8,300 per variant and 16,600 total. At 6,000 visits a month, that is nearly three months. Too slow.

Second pass. The redesign is a large change, so they size it for a 40% lift (3.0% to 4.2%). That gives about 3,200 per variant, 6,500 total, or about five weeks at roughly 1,400 visits a week. They accept five weeks.

Value check. A 4.2% rate means 252 trials a month instead of 180: 72 extra trials, about 14 extra customers, roughly $8,600 in first-year revenue per month of traffic. The build costs about two people-weeks, near $4,000. The minimum effect worth acting on is a 20% lift (3.6%), which still pays back inside a few months.

Decision rule:

Win: variant at 3.6% or higher, significant at 95%, and trial-to-paid in the first 30 days no worse than 15%. Ship.
Loss: variant below control, or trial-to-paid under 15%. Revert.
Inconclusive: anything else at week five. Keep control, move on.

Result. After five weeks, control converted 101 of 3,250 visitors (3.1%) and the variant converted 135 of 3,250 (4.2%). The difference is significant at roughly 95%, and early trial-to-paid held at 19%. They shipped it, logged that outcome framing beat feature lists for this audience, and queued the next test: the same framing on the homepage.

Notice what made this work. The team changed the test design after seeing the sample size, not after seeing the result. That is the whole point of doing the math first.

Common mistake: peeking and stopping early

The most common way small teams ruin a test is checking the dashboard daily and stopping the moment the variant looks ahead. Early results swing wildly on small samples. If you check every day and stop at the first moment you cross 95% confidence, your real false-positive rate is far higher than 5%. You will "win" tests that were never wins.

This is partly a cognitive bias problem. Confirmation bias makes a good early number feel like proof, and a bad one feel like noise.

How to avoid it:

Set the end date and sample size in the template before launch.
Check only for technical problems (broken tracking, a variant that fails to load) until the end date.
Make the template's decision rule the only thing that can end the test.
If someone insists on an early look, have them look at traffic split and tracking, not at conversion.

A second mistake worth naming: changing several things at once and then claiming to know which one worked. If the headline, layout and offer all change, you have a redesign, not an experiment. That is fine as a bet, but record it as a bet and do not claim a mechanism.

Step-by-step: running the template each month

Pick the channel and the step. Use your funnel data to find the biggest drop-off, and make sure it has enough volume. If you have not chosen where to focus, the Bullseye framework helps you commit to one channel before you start testing inside it.
List three to five candidate ideas and score them quickly: expected impact in dollars, evidence behind them, and cost.
Pick one. Fill in the template through the decision rule. If the sample size is infeasible, redesign the test or drop it.
Get a second person to challenge it for 15 minutes. Their only job is to ask, "What would make you change your mind?"
Launch and leave it alone until the end date.
Decide using the rule, then write the result in section 8 within a day. Put the learning in a shared log.
Reuse the learning. A won test should seed the next one, such as applying the winning framing to another page.

A team of three can run this cycle once a month, sometimes twice if one test is on email and another on a higher-traffic page. That is plenty. Twelve conclusive tests a year will teach you more than forty ambiguous ones.

What to test and what to skip

Match the test to your volume and to the money at stake. If you are weighing awareness spend against conversion work, the brand awareness vs conversion comparison will help you decide which end of the funnel deserves your next test slot.

Good small-team candidates:

Offer and headline on your highest-traffic landing page
Subject lines and send times on your largest email list
Pricing page layout and plan framing
Ad creative angle, when spend is high enough to reach the sample size

Poor candidates:

Button colors and micro-copy on low-traffic pages
Anything where you would need 20,000 visitors per variant and have 3,000
Changes you will ship regardless of the result

If you will ship it either way, it is not an experiment. Just ship it and watch the trend.

When you value a win, use lifetime value rather than first-purchase revenue where you can. The guide on calculating and improving customer lifetime value shows how to get a number you can plug into the "minimum effect worth acting on" field.

Key takeaways

A growth experiment template fixes the hypothesis, one primary metric, sample size and decision rule before launch, so every test ends in a decision.
Size the test before you build it. If the traffic cannot detect the lift you need within about five to six weeks, test a bigger change or a higher-volume step.
Set the minimum effect worth acting on in dollars, then let it drive your sample size.
Write a rule for win, loss and inconclusive. Inconclusive means keep control and move on, not extend the test.
Do not peek at results mid-test. Check tracking, not conversion, until the end date.
Run one or two tests a month and log every result. A short, clean record of conclusive tests compounds faster than a long list of unfinished ones.