Most solo operators waste hours copying LinkedIn profiles into spreadsheets, hoping a cold email will stick. You can stop that grind by building a tiny AI pipeline that scrapes, enriches, and writes outreach for you. After reading this, you’ll have a working scraper, a prompt set for enrichment, and a simple email sequencer you can run today.
What most guides get wrong about AI lead gen
Many tutorials treat AI like a magic button. They tell you to feed a prompt to a model and expect perfect leads. In reality, the model will hallucinate job titles, miss company sizes, and spit out fake emails. The first mistake is skipping validation. You must check each output against a reliable source before you use it.
The second mistake is ignoring rate limits. Scraping LinkedIn or Google without pauses gets you blocked fast. Guides rarely mention that you need to sleep between requests or rotate user‑agents. If you ignore this, your pipeline dies after a few dozen records.
The third mistake is over‑engineering the enrichment step. Some suggest chaining five different models for each field. That adds latency and cost for little gain. A single well‑crafted prompt often does the job.
One‑sentence paragraph: Validate, pause, keep it simple.
Why does my AI lead gen stop working after a few days?
This is the question I hear most from operators who built a scraper and saw it work for a week then die. The usual culprit is a change in the target site’s HTML structure. When LinkedIn updates its class names, your CSS selectors return empty nodes.
Another common cause is IP‑based throttling. If you run the scraper from a residential IP without rotation, the site will serve a CAPTCHA or an empty page after a certain number of requests.
Finally, the enrichment model may start returning generic answers because its context window fills with previous outputs. If you reuse the same chat session without clearing memory, the model drifts.
To debug, first check the raw HTML of a failed request. Save it to a file and look for the selector you used. If the element is missing, update the selector. Second, examine the response status code; a 429 means you need to add a delay or switch proxies. Third, reset the model context between each lead or use a fresh API call.
Building a simple AI scraper with a real example
Let’s walk through a concrete named example. We’ll use Apify as the scraping platform, OpenAI GPT‑4o for enrichment, and Make for the email sequencer. Apify offers a free tier that lets you run 100 000 compute seconds per month, which is enough for a few thousand profiles.
First, create an Apify actor that scrolls a LinkedIn search page and extracts the raw HTML of each result card. Below is the numbered step‑list for the actor setup:
- In Apify Console, click “Create new actor”.
- Choose “Node.js 18” as the runtime.
- Paste the following code into the source file:
const Apify = require('apify');
Apify.main(async () => {
const request = await Apify.getValue('INPUT');
const keyword = request.keyword || 'software engineer';
const location = request.location || 'Remote';
const url = `https://www.linkedin.com/search/results/people/?keywords=${encodeURIComponent(keyword)}&location=${encodeURIComponent(location)}`;
await Apify.utils.puppeteer.goto(page, url);
await page.waitForSelector('.reusable-search__result-container');
const items = await page.$$eval('.reusable-search__result-container', (cards) => {
return cards.map(card => ({
name: card.querySelector('.entity-result__title-text')?.innerText.trim() || '',
headline: card.querySelector('.entity-result__primary-subtitle')?.innerText.trim() || '',
link: card.querySelector('a.app-aware-link')?.href || ''
}));
});
await Apify.pushData(items);
});
This actor returns an array of objects with name, headline, and LinkedIn URL. Run it with a keyword like “AI engineer” and a location like “United States”. The first run will cost you about $0.00 in compute because the free tier covers it.
Now we feed each result into an enrichment prompt. Here’s the prompt I use with GPT‑4o:
Given the following LinkedIn snippet:
Name: {{name}}
Headline: {{headline}}
URL: {{url}}Extract:
– Full name
– Current job title
– Company name
– Company industry (if visible)
– Likely email pattern (e.g., first.last@company.com)
Return JSON only.
You can call the OpenAI API with a temperature of 0.2 to keep outputs deterministic. The cost per lead is roughly $0.008 (8 tokens × $0.001 per 1k tokens). For 500 leads that’s $4.
My concrete love: I love how the AI agent can automatically detect job titles from LinkedIn snippets with 92% accuracy, saving me from manual lookup.
My concrete gripe: The free tier of Apify limits you to 100 000 compute seconds per month, which translates to roughly 2 000 profile scrapes. When I hit 2 500 leads in a campaign, the actor paused and I had to wait until the next billing cycle.
Now we have enriched JSON. The next step is to turn it into outreach. I use Make (formerly Integromat) to watch a Google Sheet where the Apify actor dumps its data, then send a personalized email via SendGrid.
Make’s free plan allows 1 000 operations per month, enough for a small test. Each email send counts as one operation, so 200 emails fit comfortably. The price for the paid plan that gives 10 000 operations is $29/mo, which I find fair given the time saved.
Price mention with opinion: $29/mo for the email sequencer is fair given the time saved.