Experiment Engine

Overview

Running experiments without a system produces noise, not signal. At Dansu, I built an experiment engine that connected growth hypotheses to a structured test cadence, performance analysis, and explicit decisions about what to scale, kill, or iterate — turning ad hoc creative testing into a weekly operating rhythm.

The Problem

Early growth work was reactive — trying things, seeing results, not always understanding why something worked, and defaulting to repeating what felt good. Without structure, creative fatigue set in faster, winners weren't identified cleanly, and losing bets ran longer than they should have. The absence of a decision protocol meant experiments accumulated without conclusions.

What I Did

I treated experimentation as a system with three components:

Hypothesis queue: A structured backlog of growth hypotheses across channels — paid creative angles, organic content formats, Shopify CRO variants, creator outreach approaches. Each hypothesis had a clear test condition, a defined success metric, and a minimum run time before evaluation. Nothing entered the queue without a testable prediction.

Testing cadence: New experiments launched weekly. Existing experiments reviewed against predefined thresholds — cost per result, conversion rate, or engagement rate depending on the channel. No experiment ran indefinitely. The cadence forced decisions rather than deferring them.

Decision protocol: Three outcomes for any completed test — scale (increase budget or replicate across formats), iterate (adjust one variable and retest), or cut (kill the format and document the learning). Learnings fed the next hypothesis queue so the system compounded over time rather than starting fresh each week.

Growth loops tested included: UGC content funnels from creator partnerships into paid retargeting; proxy funnels using niche content accounts to reach cold audiences; Shopify product page variants (hook copy, image order, price anchoring position); ad hook variations across TOF formats (problem-led, social proof, lifestyle, product demo); and organic posting cadence and format tests across the multi-account network.

Impact

• Replaced ad hoc creative decisions with a repeatable weekly testing rhythm
• Identified winning formats faster and killed losing bets earlier through explicit decision thresholds
• Creative learnings from paid experiments informed organic strategy and vice versa — each channel's findings compounded across the other
• Reduced creative fatigue by treating formats as test variables with defined review windows rather than permanent choices

Note: Kept concise because the experimentation system was a methodology applied across the Meta ads, multi-account growth, and Shopify CRO workstreams rather than a standalone deliverable.