The Automation Stack I Actually Run

What I actually built

There's a lot of writing about automation that stays at the level of "use AI to save time". This isn't that.

At Dansu I built an automation stack that handled CRM, creator discovery, outreach, fulfilment checks, inventory forecasting, supplier comms, and ops workflows. The result was roughly 70-80% of recurring operational work automated, and weekly ops time dropping from around 40 hours to under 10. Here's the actual stack, the tools, and the decisions that made it work reliably — not just in theory.

---

The core stack

n8n is the workflow engine. Every pipeline runs through it — scraping jobs, enrichment flows, outreach sequences, fulfilment checks. I chose n8n over alternatives because it's self-hostable, the visual editor makes complex branching logic readable at a glance, and it has native support for webhooks, cron triggers, HTTP requests, and most of the APIs I needed. More importantly, it's easy to add error handling and conditional logic at the node level without writing a lot of custom code.

Supabase is the single source of truth. Every record that touches any pipeline — leads, creators, orders, jobs — lives in Supabase with a full status trail. This was a deliberate design choice: Supabase is not just storage, it's the state machine. Every pipeline reads current state from Supabase and writes state back. That's what makes the whole system resumable.

RocketAPI is the Instagram data layer. Creator discovery, engagement metrics, account lookups, reel performance data — it all comes through RocketAPI. The coverage is good enough for the use cases I needed: scraping by hashtag, pulling account metrics, and getting reel-level engagement signals for ranking.

Gmail API handles outreach sends and reply monitoring. Gmail API rather than a third-party email tool because I wanted direct control over sending behaviour, rate limiting, and threading logic without paying per-email fees at volume.

LLMs are components inside workflows, not the workflow itself. I use them for specific, bounded tasks: extracting structured fields from messy scraped text, classifying content categories from short bios, generating outreach drafts from a strict template using only the CRM fields present for that contact. The prompts are constrained — defined output schema, explicit "only use provided fields", no invented context.

---

How a pipeline actually works

The creator discovery pipeline is the clearest example.

A cron trigger in n8n fires on schedule. It pulls a batch of accounts from Supabase where status = 'pending' and last_scraped is null or stale. For each account, it calls RocketAPI to pull current metrics — follower count, recent reel views, average engagement, posting frequency over the last 30 days.

That data lands in an enrichment node. A scoring function runs: engagement rate against follower count (to filter inflated accounts), posting velocity over the past 30 days (favouring consistent posters over one-hit accounts), content category match against a predefined list of brand-adjacent niches. Each factor gets a weight. The output is a single score and a tier label: high, medium, low, or disqualified.

Records scoring above the high threshold get their status updated to outreach_ready. n8n picks those up in a separate workflow, generates a personalised opener using the LLM node (constrained to the actual account data), and queues the email via Gmail API. A send-time randomiser staggers the emails so they don't all land in the same minute. Reply detection runs via a Gmail watch webhook — any reply writes back to the Supabase record and pauses further outreach.

Every step writes a status and timestamp. If the RocketAPI call fails, the record gets status = 'scrape_failed' and a retry counter incremented. After three failures it goes to requires_review. Nothing disappears silently.

---

Reliability patterns

Reliability is the thing most automation writing skips. A pipeline that works 80% of the time is actually worse than doing the work manually, because the 20% failure rate is invisible and corrupts your data.

These are the patterns I applied across every workflow:

Idempotency by design. Every job can be rerun without creating duplicates or corrupting state. Dedupe keys are unique entity identifiers, not run IDs. Before any record is processed, there's a check: has this record already been processed in this state? If yes, skip. This means scraper reruns are safe — they'll pick up new records and skip ones already handled.

Job state machine. Every record has an explicit status column: queued, scraping, enriched, scored, outreach_ready, sent, replied, failed, requires_review. Workflows read status, act on specific statuses, and write new statuses. This means I can query Supabase at any time and see exactly where every record sits in the pipeline. No black boxes.

Retry with backoff. API calls fail. Rate limits hit. n8n's built-in retry nodes handle most of this, but I add explicit backoff logic for the APIs I hit hardest — RocketAPI and Gmail. Exponential backoff rather than fixed intervals. A job that fails at 2am retries at 2:01, then 2:03, then 2:07. If it still hasn't resolved by the time I wake up, the failure log is waiting.

Dead-letter storage. Failed jobs don't vanish. They get written to a failed_jobs table in Supabase with the error type, the payload at time of failure, and a timestamp. This means I can diagnose failures in bulk, fix the underlying cause, and requeue the affected records rather than losing them.

Error logging with categories. When a job fails, I log the error type: api_rate_limit, schema_mismatch, missing_required_field, timeout, unexpected_response. Categorical logging means I can see patterns — if schema_mismatch spikes, RocketAPI probably changed its response format. If api_rate_limit spikes, I need to reduce concurrency or add delays.

---

Proxy routing

Some scrapers need proxy routing to avoid blocks on volume scraping. I route these through residential proxies, not datacenter proxies — the latter get flagged more aggressively by most platforms. The routing logic lives inside the n8n workflow: if a request returns a 429 or a block signal, the workflow retries through a different proxy pool entry, then backs off.

This isn't necessary for every pipeline. It's necessary for the ones hitting public web targets at volume. The Gmail API and RocketAPI flows don't need it because they're authenticated API calls with rate limits I respect explicitly.

---

What I don't automate

Automation decisions are partly about what to automate and partly about what not to.

I don't send fully autonomous outreach to high-value partners — festivals I'm targeting for anchor B2B relationships, or large creators with serious commercial scale. Those drafts are generated and queued, but the send is gated on a manual review. The cost of a bad first message to a high-value target is higher than the cost of a few seconds of review.

I don't let the LLM write anything that goes out without a schema check. The prompt includes a required output structure. Before a draft is queued, a node validates that every expected field is present. If the output is missing a field or has added something unexpected, it goes into the review queue rather than the send queue.

I don't automate pricing decisions or supplier contract responses. Detection and triage — yes. First-touch with any commercial consequence — no.

---

The thing that makes it maintainable

Everything is observable. That's the real answer to "how do you maintain this without a team".

Supabase gives me a live view of pipeline state across every workflow. n8n has execution history for every workflow run, with the input and output of every node. Failures have categorical logs. Retry queues have counts.

If something breaks — and things break, always — I can open Supabase, filter to status = 'failed' with error_type = 'schema_mismatch', see exactly which records are affected, fix the schema mapping in the relevant n8n node, and requeue the batch. The whole debugging cycle takes minutes rather than hours because the system was designed to make its own state visible.

That observability is what separates automation that keeps working from automation that silently fails until you notice the pipeline has been producing nothing for three days.