Building a Design Arena AI: Automated Creative Testing for Operators

What is a Design Arena AI, Exactly?

Last quarter, I burned through $5,000 in ad spend testing creative that my gut told me was good. My gut was wrong. The problem wasn’t the copy or the targeting; it was the endless, expensive cycle of manual design tweaks and subjective human feedback. So I built a system to automate the creative feedback loop itself: a design arena AI.

The concept is simple. Instead of you or your designer creating one ad image and hoping it works, an AI system generates multiple variations. Then, a second, separate AI agent acts as a ruthless creative director, analyzing and scoring each one against a set of business goals. It’s a survival-of-the-fittest environment for ideas, and it runs while you sleep.

This isn’t about replacing human designers. It’s about killing bad ideas faster and cheaper than any human focus group ever could. It lets you test a dozen visual concepts before a human even opens Figma. The core components are an image generator, a vision model for the critique, and an orchestrator to glue it all together.

The Core Stack: What You’ll Actually Need

You don’t need a team of data scientists for this. You just need to connect a few APIs. Here’s the stack I use for my production systems.

• Image Generator: Everyone talks about Midjourney, but its automation story is a mess (and good luck getting API access if you’re a solo dev). For a real, automated pipeline, you need an API. The DALL-E 3 API from OpenAI is reliable and integrated directly into their main API, making it easy to call. Alternatively, you can use Stable Diffusion via a service like Replicate, which gives you access to tons of fine-tuned models. For this build, we’ll stick with DALL-E 3 for simplicity.
• Critic Agent (Vision Model): This is the most important piece. You need an AI that can *see* and *reason* about an image. OpenAI’s GPT-4V(ision) is a solid choice. However, I’ve found that Claude 3 Opus from Anthropic currently has the edge in nuanced visual analysis and following complex instructions. Its ability to dissect visual hierarchy and emotional tone is uncanny.
• Orchestrator: This is the glue. You could write custom Python code, but a no-code AI tool is faster for this kind of workflow. I use Make (formerly Integromat) (formerly Integromat). It’s a visual workflow builder that connects to the OpenAI and Anthropic APIs without much fuss. Honestly, its free plan is a joke, but the Core plan at $29/month is fair for the amount of time it saves. It’s more than enough to run this entire design arena.

What Most Guides Get Wrong About AI Creative

Most tutorials on AI image generation are just collections of cool-looking prompts. They completely miss the point. The prompt that generates the image is only 20% of the battle. The real leverage comes from the system you build *around* the generation.

The critic prompt is where the money is made.

A generator prompt gets you a picture. A critic prompt gets you a business outcome. The mistake is treating this as a one-shot process. You don’t just generate an image and run it. You generate ten, have your AI critic score all ten, and then you analyze the top two. The arena model is what creates the value, because it surfaces the outliers you wouldn’t have thought to create yourself.

Another massive error is ignoring brand consistency. Letting an AI generate whatever it wants is a fast way to dilute your brand. You have to bake your brand guidelines—colors, fonts, tone, and negative constraints (e.g., “never show people in suits”)—directly into both the generator and critic prompts. Without these guardrails, you’re just making pretty, useless art.

Building the ‘Critic’ Agent: A Concrete Prompt Example

This is where the magic happens. You need to give your vision model a persona and a very specific scorecard. You can’t just ask, “Is this a good ad?” You’ll get generic, unhelpful answers. You need to force it to think like a seasoned professional with KPIs to hit.

Here is a prompt I’ve used with Claude 3 Opus to critique ad creative for a SaaS product. You would send this prompt to the API along with the image you want it to analyze.

You are an expert Direct Response Marketing Creative Director with 20 years of experience scaling e-commerce and SaaS brands. Your only goal is to identify ad creative that will drive conversions at a low cost. You are ruthless in your analysis. You will analyze the provided image and score it on a scale of 1-100 across four distinct categories. Provide your reasoning for each score in 1-2 sentences. Output your response ONLY as a valid JSON object. Do not include any other text or pleasantries.

The four scoring categories are:
1. **Clarity_of_Value_Prop (1-100):** How quickly and clearly can a user understand what is being offered and why it's valuable? Is the core message instantly obvious or buried?
2. **Visual_Hierarchy (1-100):** Does the image guide the user's eye effectively towards the most important element (the value prop or the CTA)? Is it cluttered or clean?
3. **Emotional_Resonance (1-100):** Does the image evoke a specific, desired emotion (e.g., relief, excitement, trust)? Or is it generic and forgettable?
4. **Brand_Consistency (1-100):** Does the image adhere to the brand's established visual identity? (Primary color: #0033FF, Font style: Sans-serif, clean, modern).

Your final output must be a JSON object with four keys: "clarity_score", "hierarchy_score", "emotion_score", and "brand_score", each with an integer value and a nested string key "reasoning".

By forcing the AI into a role and demanding a structured output like JSON, you turn a fuzzy creative process into cold, hard data. You can then use your orchestrator (Make.com) to parse this JSON, log the scores in a spreadsheet, and automatically flag the highest-scoring images for human review.

How Do You Handle Brand Consistency?

This is a frequent question, and it’s where these AI systems often fall apart. Getting an AI image generator to perfectly replicate your brand’s logo or use a specific font is a nightmare. My biggest gripe with the current generation of models is their inability to render text. They are shockingly bad at it. You’ll ask for a simple headline on an image, and it will come back as a garbled, misspelled mess. It’s a constant frustration that means you almost always need a final human touch-up in Canva or Figma to fix text and logos.

But there’s a clever workaround for policing other brand elements. This is a feature I absolutely love. You can use the vision model as a brand compliance officer. In your workflow, after an image is generated, you pass it to your critic agent with a simple prompt: “Here is our brand guide. Does the attached image violate any of these rules? Be specific and list any violations.”

I feed it a text block with our color codes, approved imagery styles, and negative constraints. The AI is surprisingly good at catching subtle violations, like using a slightly off-brand shade of blue or showing a visual metaphor that contradicts our brand’s tone of voice. It’s not perfect, but it automates 80% of the brand-check process.

How to Debug When This Breaks

This AI agent framework will fail. It’s not a matter of if, but when. Here are the three most common failure modes I’ve encountered and how to fix them.

1. The generated images are generic and boring. Your generator prompt is too vague. “An image of a person using a laptop” will give you stock photo garbage. You need to add texture. Specify the art style (e.g., “in the style of a minimalist vector illustration”), the color palette (“using a duotone color scheme of navy and coral”), the lighting (“dramatic side-lighting”), and the emotional context (“a frustrated office worker finding a moment of clarity”). Specificity is your best defense against mediocrity.
2. The critic gives everything a high score. Your critic prompt is too nice. If you don’t explicitly tell the AI to be harsh, it will default to being an agreeable assistant. Add phrases like “Be ruthless,” “Your job is on the line,” or “Find flaws that a junior designer would miss.” Also, forcing it to justify its scores, as shown in the prompt example, makes it harder for the AI to be lazy. If the reasoning is weak, the score is meaningless.
3. The automation times out or errors out. Image generation APIs can be slow, sometimes taking 30-60 seconds to return an image. If your orchestrator isn’t set up for this, the whole workflow will crash. In Make.com, you need to use asynchronous modules or webhooks that wait for the process to complete before moving to the next step. It’s a slightly more advanced setup, but it’s essential for a reliable system that doesn’t fall over when an API is having a slow day.

Building a design arena AI is a perfect example of an operator playbook. It’s not a single tool; it’s a system of tools working together to solve a real business problem: the high cost of creative testing. It takes some setup, but once it’s running, it gives you a powerful advantage.

We cover this in more depth elsewhere — deeper coverage of AI agent platforms.

If you’d rather skip the build and deploy a working version in an afternoon, we’ve packaged this entire workflow as a deployable AI automation blueprint at deepusecase.com/vault.