Have you ever received an email that felt so personalized, so tailored to your interests and experiences, that you couldn't help but be intrigued? What if I told you that email wasn't crafted by a human, but by an artificial intelligence (AI) agent?
That's exactly what happened in an experiment I recently conducted.
As founder, I'm always exploring innovative ways to scale my business operations. So when I started learning about AI agents and their potential, I couldn't resist trying them out for a task that traditionally requires human effort - personalized email outreach.
The idea was simple yet ambitious.
Could an AI agent craft compelling emails that would capture people's attention and drive engagement, all while maintaining a level of personalization that feels human? I decided to find out.
I used AI agents to send out nearly 1,000 personalized emails to developers with public blogs on GitHub.
The goal? To pique their interest in my wisp, a CMS that can simply the ways they manage their blogs.
And you know what? It worked remarkably well, especially for a cold email campaign.
Out of those 970 emails, the campaign achieved a staggering 20% click-through rate, with 180+ recipients clicking through to check out wisp in the span of a single day.
In fact, more than 10 signups in the day was attributed to this campaign itself.
As someone familiar with the challenges of cold outreach, those results were pretty impressive - especially for an experiment that took a day.
Developers are a notoriously tough crowd to sell to, yet the AI agents managed to capture their interest.
Here's what happened in my little experiment.
The AI Workflow Architecture
At the heart of this experiment was a trio of AI agents, each with a distinct role to play:
The WebInfoAgent scoured the target blogs, GitHub repositories, and websites to gather contextual information about the recipients' projects, interests, and writing styles.
Using that research, the EmailAgent then crafted personalized emails that built rapport by referencing specific details about the recipients' work.
Finally, the JobReportAgent documented the entire process, filing summaries into a database to track which emails were sent, any failures, and other relevant data.
From a technical standpoint, I used tools like LangChain's LangGraph for building the AI agents, LangSmith for monitoring and observability, and supabase's database to manage jobs and reports. The scripts were written in Node.js and TypeScript.
As someone with a development background, I have to admit - setting up this AI workflow was no easy feat. Agentic workflows are still an emerging concept, with limited documentation and resources available, especially in the Node.js ecosystem I was working in.
Overcoming Hurdles
Data Feeds
One of the biggest challenges was conceptualizing the workflow itself and finding suitable data sources to feed into the agent workflow. I spent hours trying different data sources before deciding on searching GitHub to identify active developers with public blogs for this experiment.
Training Agents to Write Good Emails
The real hurdle was ensuring the emails seemed genuinely personalized and not spammy. I knew that if recipients detected even a whiff of a generic, mass-produced message, they'd tune out immediately.
So a lot of time was spent on find-tuning the tone and structure of the email. In the end, I have a 700-words prompt on how to write good emails. In it, I provided the email agent with examples of personalized, rapport-building emails and emphasized the importance of referencing specific details about each recipient's work.
Multi-lingual Agents
While running through test cases, I've also noticed that many of the developers were non-English natives and writes in either Chinese, Korean, Spanish or French . So I've also prompted the agent to respond to them in their language. It took multiple tries to finally have the agent write emails in different language but the results were amazing. I've had multiple response from developers receiving these messages replying in Chinese and Korean!
This is a sample exchange between the agent's initial email and the response from the developer that loosely translates into:
I’m glad that my article can be helpful to you.
The WISP you recommended to me is a magic tool for rapid development and deployment for a website owner. I am glad that you recommended it to me. I will try to use WISP when I develop new websites in the future.
Thanks again for your recommendation!
Email Deliverability
Something that completely slipped my mind was email deliverability. Although I was using a different domain from my mail product, I've never sent emails at this volume before so the experiment impacted my email deliverability and some emails were landing in spam.
It didn't help that I forgot to perform basic screen on the email before sending it. It was bouncing off the most silly email addresses like example.com and noreply emails.
Looking back, it's probably better to run a much smaller experiment in the future but I was eager to test out the scalability of this approach.
Despite these challenges, the results speak for themselves.
Not only did the open and click-through rates exceed my expectations, but many developers even replied expressing their appreciation for the personalized messages.
Incredibly, not a single recipient seemed to detect that the emails were AI-generated.
Breakdown on Costs
Looking at the numbers, the cost-effectiveness of this AI-powered approach is staggering. The entire experiment, which sent out around 970 emails, cost a little over $200 to run using GPT-4o API from Azure OpenAI.
That's about $0.20 per personalized email sent.
To put that into perspective, let's compare it to traditional methods:
If I had hired human sales development representatives (SDRs) or a business process outsourcing (BPO) partner for this task, the cost would have been exponentially higher.
When I attempted to do this manually a few days prior, each email took me 15-20 minutes of research, writing, and sending. At that rate, I might have managed 50 emails per day at best.
A human SDR or BPO would take almost the full month to do what the agent did in a single day and cost upward of $2000.
With the AI agents, the median time per email was just 26 seconds. No human could match that efficiency and output. Not to mention that the script can be scaled easily without training and performance management.
Even if you look at it from an advertising spend perspective, the $1 cost-per-click (CPC) is pretty low compared to $2-$5 for software products. And remember, this was just the first run with no optimization on cost yet.
The implications are huge, especially for small businesses and startups like mine. By leveraging AI, we can supplement our workforce without scaling prematurely. Tasks like research, outreach, and business processes that typically require human effort can now be automated and streamlined.
Unexpected Results & Lessons Learnt
While the performance metrics were impressive, there were some unexpected results and insights too:
The cost, while low, was higher than I anticipated initially. The main cause is due to the agents using excessive tokens for their task. There's definitely room for further optimization.
Many recipients replied expressing appreciation for the thoughtful, personalized messages. As a founder, I actually felt a tinge of guilt knowing an AI crafted those emails!
Not a single person detected they were corresponding with an AI. Some asked how I found their email, but no one questioned the authenticity of the messages.
Sending 900+ emails in a day impacted email deliverability, as email providers flagged it as potential spam. A valuable lesson on warming up mailboxes gradually.
The Scalability Factor
One of the most exciting aspects of this experiment is just how scalable the AI-powered solution is. Once the workflow was set up, running new campaigns for different audiences was a matter of hours, not days or weeks.
I could easily connect this system to various data sources, searching for highly qualified leads beyond just developers. The possibilities are endless – entrepreneurs, startup founders, anyone whose pain points align with my product's value proposition.
And the best part?
Optimizing and iterating on the AI agents is far easier than trying to retrain or performance-manage human teams. I'm already exploring ways to reduce costs further through more efficient token usage and workflow improvements.
The implications for small businesses are profound. We can now punch far above our weight, automating processes that were previously too costly or time-consuming. As this technology becomes more accessible, it could level the playing field and empower lean teams to compete with industry giants.
The Future of AI Marketing
While this experiment yielded exciting results, we're still in the early days of agentic workflow and AI marketing automation. The documentation and resources are limited, making it challenging for non-technical teams to implement similar solutions.
However, I foresee a future where no-code, drag-and-drop tools will make building AI workflows accessible to anyone. Startups like FirstQuadrant and Dify are already working on platforms to democratize this technology - whether through an integrated sales stack or generative AI development platform.
As the technology matures and costs continue to drop, I anticipate more small businesses and startups will leverage AI agents for marketing, sales, and operations. The ability to automate processes at scale while maintaining a human touch will be game-changing.
Side story: One of my email actually landed in the inbox of Anand who's building FirstQuadrant and that's how I discovered them.
Advice for AI Marketing Pioneers
If you're a technology enthusiast eager to experiment with AI marketing solutions, my advice is: go for it! The potential rewards are immense, but be prepared for a steep learning curve.
For those without technical resources in-house, keep an eye on emerging no-code platforms like DIFY.AI. While not quite there yet in terms of quality and flexibility - I gave it shot trying to re-implement my agents after the experiment - these tools are rapidly evolving.
Ultimately, this experiment reinforced my belief that AI will fundamentally transform how businesses operate. The key is striking the right balance between automation and the human touch – using AI to augment our capabilities, not replace the ingenuity and emotional intelligence that sets us apart.
Reshaping the Job Landscape
As exciting as the possibilities of AI marketing are, we must also confront the potential impact on certain job roles and functions. Roles like sales development representatives (SDRs) and business process outsourcing (BPOs) could be significantly disrupted by AI agents capable of automating their core tasks.
Think about it: the primary responsibilities of SDRs are researching prospects, gathering information, filling CRMs, and booking meetings for account executives. My AI workflow handled all of those tasks with ease, and at a fraction of the cost.
The same goes for BPOs, whose manual processes for administrative tasks could easily be replicated and scaled by AI agents following defined workflows.
Now, I don't want to sound alarmist – we're not quite at the stage where AI will render these roles obsolete overnight. However, as the technology becomes more accessible and cost-effective, the threat to such jobs will only intensify.
There's also the question of ethical considerations around using AI for mass personalized outreach. While my experiment yielded positive results, with recipients appreciating the personalized touch, there's a potential slippery slope.
As AI-generated communications become more widespread, will we reach a point where people can't distinguish human from machine? Where every email, social media interaction, or customer service chat is powered by an AI agent?
It raises concerns around authenticity, consent, and whether we're entering a world of human-perceived relationships and experiences without the actual human element.
Navigating the AI-Powered Future
Make no mistake, the genie is out of the bottle. AI capabilities will continue advancing at a blistering pace, transforming how businesses operate and people work. The onus is on us – the pioneers and early adopters – to steer this technology in an ethical, responsible direction.
We must have thoughtful conversations about the implications for job displacement and develop strategies to reskill and transition impacted workers. We should explore frameworks that balance automation with protecting human roles that require emotional intelligence and creativity.
At the same time, we need to establish guidelines around transparency and consent for AI-driven communications at scale. Deception through omission is still deception – people should be aware when they're interacting with an AI agent versus a human.
These are complex challenges without easy answers. But by being proactive and intentional in our development and deployment of AI solutions, we can work towards a future where this powerful technology uplifts and empowers rather than diminishes and displaces.
The Beginning of an AI-Powered Journey
This experiment was just the first step in my journey to explore the transformative potential of AI for business operations. While the results were exciting, I know we've only scratched the surface of what's possible.
To further share these learnings, I've decided to open source the code used to build these AI agents. My goal is to help others understand how to implement similar workflows and spark ideas for innovative AI use cases - beyond spam of course.
In a separate technical blog post, I'll do a deep dive into the code itself, as well as the core concepts behind building effective AI agents. For those interested in such technical walkthroughs, stay tuned!
If this story piqued your curiosity about the leading edge of applied AI, I encourage you to share it with friends, colleagues, or anyone who may find it insightful. After all, we're in the midst of a technological shift that will impact how businesses of all sizes operate.
As I reflect on this experiment, I can't help but feel a sense of awe at the pace of AI's evolution. What seemed like science fiction is now a reality that any business can harness. The possibilities are endless – if you can dream it, AI may soon be able to power it.
However, with great power comes great responsibility. As we navigate this AI-powered future, we must do so thoughtfully and ethically. Displacement of human roles, consent, and transparency should remain at the forefront of these conversations.
For now, I'm energized to continue exploring, learning, and pushing boundaries. Because if this experiment taught me anything, it's that the era of autonomous AI agents revolutionizing how we work is already upon us.
