*A professional-services firm in downtown Orlando was buried under 600 emails a day. We built an AI triage system that auto-classified, extracted actions, and routed messages—except for one category we kept human.*
(Client details are anonymized and some specifics composited at the client’s request.)
It was a Tuesday morning in April when the owner of a 40-person professional-services firm in downtown Orlando called me. Their email inbox was a disaster. Six hundred messages a day—client requests, internal memos, vendor invoices, spam. Their four-person admin team was spending 80% of their day sorting and forwarding, leaving little time for actual client work. The owner said, “I need a system that can read, sort, and act on these emails. I don’t want my people to drown anymore.”
I’d seen this before. Small and mid-market firms often treat email as a fire hose they can’t turn off. But this time, the volume was extreme. We needed something that could understand intent, extract key actions, and route messages to the right person—without losing the nuance that only a human can catch.
The Situation: What Was Breaking
The firm had grown fast over five years, from 12 to 40 employees. Their client base was mostly other Central Florida businesses—real estate agencies, law firms, medical practices. Every client sent requests via email: scheduling, document requests, billing questions, project updates. The admin team had a manual triage process: read, decide who should handle it, forward. But with 600 emails a day, they were always behind. Average response time for a client request was 8 hours. Some emails fell through the cracks entirely.
I sat down with the admin team for a week. We shadowed their workflow. They showed me their shared inbox—hundreds of unread messages, flagged ones they’d “get to later,” and a folder system that no one used. One admin told me, “I spend my first two hours just deleting spam and forwarding the rest. By the time I’m done, new emails have piled up.” They were burning out.
What They’d Tried Before
They’d tried a few things. A rule-based email filter that flagged messages with certain keywords—but it missed context. A shared spreadsheet for tracking requests—but no one updated it. They even hired a part-time assistant, but the volume only grew. The problem wasn’t the people; it was the process. They needed something that could understand language, not just keywords.
I recommended an AI readiness assessment to identify the right approach. We ran a two-week audit of their email patterns: types of messages, response times, common actions. The data confirmed what they felt: 40% of emails were information-only (no action needed), 30% required a simple reply (like confirming receipt or giving a status), 20% needed routing to a specific department, and 10% were urgent client escalations. That last category—escalations—was the one we decided to keep human.
The AI Work We Did
We built an agentic email triage system using a combination of intent classification, entity extraction, and automated routing. Here’s the plain-English breakdown:
Intent Classification with a Fine-Tuned Model
We started with a pre-trained language model (like GPT-3.5 but fine-tuned on their historical emails). We labeled 2,000 example emails into categories: “Client Request,” “Internal Task,” “Vendor Invoice,” “Spam,” “Urgent Escalation.” The model learned to assign a confidence score for each category. Anything below 80% confidence got flagged for human review.
Action Extraction Using Entity Recognition
For emails classified as “Client Request,” we needed to pull out the specific action: “Schedule meeting,” “Send document,” “Update project status.” We used a named-entity recognition (NER) pipeline to extract dates, names, project IDs, and action verbs. This gave the system a structured summary: “Client X wants to meet on June 15 at 2 PM to discuss contract renewal.”
Routing Rules via a Low-Code Automation
We connected the classification output to an n8n workflow (you can think of it as a visual automation tool). Based on the intent and extracted entities, the email went to the right person or team. Billing questions went to accounting; project updates went to the project manager. The system also auto-replied to confirm receipt and set expectations: “We’ve recieved your request and routed it to [person]. You’ll hear back within 4 hours.”
The One Category We Kept Human
Urgent escalations—emails with phrases like “critical,” “urgent,” “ASAP,” or from known high-priority clients—were never auto-replied. Instead, they were tagged and sent to a dedicated Slack channel where a human could review within 15 minutes. Look, a machine can’t judge emotional tone or business politics. One false positive could damage a client relationship.
Where We Deliberately Kept a Human in the Loop
Beyond escalations, we also kept humans for ambiguous emails. If the model’s confidence was low (below 80%), the email went to a human reviewer with a suggested classification. The reviewer could accept, reject, or modify. We logged every correction to retrain the model monthly. Over three months, accuracy went from 85% to 94%.
I also insisted that the admin team have a “stop button.” At any point, they could pause the automation and manually triage. This gave them control and trust in the system.
“I used to spend my mornings just sorting. Now I spend them actually helping clients. The AI handles the noise.” — Admin team member
The Measured Results
After three months, we measured the impact. The numbers came from their email system and time-tracking tool:
- Time saved: The admin team collectively saved 22 hours per week on email triage. That’s over a full work week per month.
- Response time: Average client response time dropped from 8 hours to 1.5 hours. For urgent emails, it was under 30 minutes.
- Backlog cleared: The inbox went from 600 unread to under 50 by end of day. No more “I’ll get to it later” emails.
- Accuracy: The model correctly classified 94% of emails. The 6% that needed human review were caught quickly.
Here’s a specific example: a real estate client in Winter Park sent an email at 9 PM asking for a contract amendment. The system classified it as a client request, extracted the action “amend contract,” routed it to the legal team, and auto-replied with a confirmation. By 8 AM the next day, the legal team had the draft ready. Previously? That email would’ve sat until someone spotted it at 10 AM.
What We’d Do Differently
Honestly, one thing was harder than expected: handling email threads with multiple topics. A single email might ask a billing question, request a document, and complain about a project delay. The model would classify it as one intent, missing the others. We eventually added a “multi-intent” flag that split the email into seperate tasks. That took an extra two weeks of development.
Also, we underestimated the training data needed. 2,000 emails wasn’t enough for edge cases. We should’ve labeled 5,000 from the start. The first month had more false positives than I liked.
If I were doing this again, I’d involve the admin team earlier in the design. They had great insights about which emails were truly urgent versus just noisy. Their input improved the model significantly.
Is This Right for Your Business?
If your team is drowning in email—whether you’re a law firm, a medical practice, or a service company—this approach can work. But it’s not a plug-and-play solution. You need a clear understanding of your email patterns, a willingness to train the model, and a commitment to keeping humans in the loop for critical decisions.
We started with an AI readiness assessment to scope the project. That step saved us from building a system that didn’t fit their needs. If you’re curious about what an AI triage system could do for your business, reach out. We can audit your email flow and give you a realistic plan.
For now, the downtown Orlando firm is running smoothly. Their admin team is less stressed, clients are happier, and the owner sleeps better at night. That’s the kind of outcome I love to see.
“I used to spend my mornings just sorting. Now I spend them actually helping clients. The AI handles the noise.” — Admin team member
Frequently asked questions
How long did it take to build the email triage system?
The initial build took about 6 weeks, including data labeling, model fine-tuning, and workflow automation. The first month included a human-in-the-loop period for model refinement.
What tools did you use?
We used a fine-tuned language model (similar to GPT-3.5), n8n for workflow automation, and a custom entity extraction pipeline. The system integrated with their existing email platform via API.
How accurate is the triage system?
After three months of retraining, the model achieved 94% accuracy on intent classification. For emails below 80% confidence, human review is triggered.
Can this work for my type of business?
Yes, if you have a high volume of email with clear categories and action items. Professional services, healthcare, real estate, and legal firms are good candidates. We recommend starting with an AI readiness assessment.
What happens if the AI misclassifies an urgent email?
We designed a safety net: any email flagged as urgent or with low confidence goes to a human within 15 minutes via a Slack channel. We also review misclassifications monthly to improve the model.
How much did this cost?
Costs vary based on email volume and complexity. For this firm, the monthly AI processing cost was under $500, plus the initial development fee. The time savings paid for it within 2 months.
Ready to talk it through?
Send a one-line description of what you are trying to do. I will reply within one business day with a plain-English next step. Email or use the form →