AI companies are constantly hunting for new leads, but most burn cash on pricey platforms that promise magic and deliver fragility. In this guide you’ll see how to assemble a repeatable lead pipeline using cheap scrapers, simple email scripts, and no‑code automation—so you can keep costs under $50/mo while the pipeline runs itself. By the end you’ll know which tools to pick, how to wire them together, and where the usual failure points hide.
Why most scrapers get blocked after 500 requests
Many tutorials tell you to just spin up a headless browser and start crawling LinkedIn or Crunchbase. The reality is that sites throttle IP addresses after a few hundred requests, and they serve CAPTCHAs or empty pages. I’ve seen this happen with Puppeteer on a cheap VPS after roughly 450 profile scrapes.
The fix isn’t more compute; it’s smarter throttling and rotating residential proxies. I use Apify’s built‑in proxy rotation, which costs $49/mo for the basic plan. The scraper itself is a short JavaScript snippet that waits random intervals between 2 and 8 seconds, mimics human scroll, and stores each result in a key‑value store.
await page.waitForTimeout(Math.random() * 6000 + 2000);
// random pause to look less robotic
One concrete gripe: Apify’s free tier gives you only 10k compute units, which evaporates after scraping about 200 LinkedIn profiles. If you forget to monitor usage, the job stops mid‑run and you lose half a day’s work.
Concrete love: The platform’s UI shows a live graph of request rate and success ratio, so you can spot a throttling event before it turns into a block.
What most guides get wrong about email copy
Most advice says to write a “personalized” opening line and then pitch your service. In practice, recipients skim the first sentence and delete if it feels templated. I’ve tested dozens of variations and the only thing that consistently lifts reply rates is a single‑sentence observation about a recent post or product launch.
Here’s a real prompt I feed to Claude 3.5 to generate that line:
Given the LinkedIn post: "{{post_text}}", produce one sentence that shows I read it and found it interesting, without mentioning my product.
The output is usually 12‑18 words, e.g., “Loved your take on how AI agents can cut SaaS onboarding time by 40%—that’s exactly the kind of shift I’m watching.”
I think paying for a dedicated AI‑agent framework just to generate these lines is overkill; a simple prompt plus a cheap LLM API does the job for under $0.005 per message.
Mild aside: (which, yes, is annoying) you still need to watch for rate limits on the LLM provider; I hit a 429 after 150 calls in five minutes on the free tier of OpenAI.
How to debug when this breaks
When the pipeline stops, the first place to look is the Apify run log. If you see “HTTP 429” or “CAPTCHA detected”, the proxy pool is exhausted. Increase the pause between requests or upgrade to the residential proxy add‑on ($19/mo extra).
If the email step fails, check SendGrid’s activity feed. A common error is “blocked because of spam‑like content”. That usually means the generated line slipped into a template that contains a tracking URL or a spammy phrase like “free trial”. I fix it by stripping any URL from the prompt output and rerunning the step.
Finally, verify the Make scenario (formerly Integromat) didn’t hit its operation limit. The free plan allows 1,000 operations/mo; each lead uses about four operations (scrape, generate email, send, log). At 200 leads you’re already at 800 ops, leaving little room for retries.
One concrete love: Make’s visual scenario builder lets you click a module and see the exact input and output data, which cuts debugging time from ten minutes to under two.