Most requests to an AI automation company start with a tool in mind: “we want an AI agent” or “we need a chatbot.” The projects that go well usually start somewhere else, with one specific process, a count of how often it runs and a clear idea of what a mistake costs. This guide is for that starting point. It walks through choosing which process to automate first, picking the right kind of automation for it, and the questions to put to a firm about what happens after go-live, which is where most automation budgets quietly leak.
If you are buying an off-the-shelf AI product rather than having a workflow built, our guide to finding the right AI vendor covers security reviews, pricing and lock-in in more depth.
Step 1: Choose the process before you choose the firm
A firm can only quote and design well if you hand it a defined process. “Automate accounts payable” is a department. “Match incoming supplier invoices to purchase orders and route mismatches to a clerk” is a process. Write yours at that level before you talk to anyone.
This matters more than it sounds. In a RAND study based on interviews with 65 experienced data scientists and engineers, misunderstanding or miscommunicating the problem to be solved was the most common reason AI projects failed; a lack of suitable data and inadequate infrastructure were also leading causes. A well-scoped process addresses the first of those directly and exposes the second early.
When you have several candidates, score them. The four factors below predict most of whether an automation will pay off:
- Volume: how many times a month the process runs. Automating something that happens ten times a month rarely recovers the build and maintenance cost.
- Rule clarity: could a new hire do it correctly from a written checklist, or does it rely on judgment that lives in one person’s head?
- Error cost: what happens when the output is wrong. A misfiled support ticket is cheap; a wrong payment or a missed compliance deadline is not. Here a lower cost scores higher.
- Data availability: are the inputs already digital, consistent and accessible through a system you control, or do they arrive as phone calls, handwritten notes and screenshots?
Process scoring worksheet
Score each candidate 1 to 3 on each factor and add them up. A total of 10 to 12 is a strong first candidate; 7 to 9 is workable with design effort; 6 or below usually means fix the process or the data before automating anything.
| Factor | 1 point | 2 points | 3 points |
|---|---|---|---|
| Volume | Occasional, a few dozen times a month or less | Daily, steady | High and constant, or spiky enough to cause backlogs |
| Rule clarity | Mostly judgment; staff disagree on the right answer | Clear rules with frequent exceptions | Rules can be written down; exceptions are rare and recognizable |
| Error cost (lower is better) | Financial, legal or safety impact if wrong | Rework or customer annoyance if wrong | Easily caught and cheaply fixed |
| Data availability | Paper, phone or scattered across inboxes | Digital but inconsistent formats | Structured, in a system with an API or export |
Two practical notes on using it. First, have the people who do the work fill it in, not only their managers; the people doing it know where the exceptions are. Second, record a baseline for the winner now: monthly volume, hours spent, error or rework rate and turnaround time. You will need these numbers to judge any proposal and to know later whether the project worked.
Step 2: Match the approach to the process
“AI automation” covers several quite different technologies. A good firm will recommend the simplest one that fits, even if it is not the most impressive one to sell. Use the table below to check whether a proposal fits your process or the vendor’s favorite toolset.
| Approach | Fits when | Watch out for |
|---|---|---|
| Rules-based scripts or RPA (software robots that follow fixed steps, often through a user interface) | Inputs are structured, steps are fixed, and the systems involved have no usable API | Breaks when screens, field names or file layouts change; needs active maintenance |
| Workflow and integration tools (low-code platforms that connect apps through APIs and triggers) | The work is mostly moving data between systems on a trigger, with simple branching and approvals | Logic spread across many small flows becomes hard to see and audit; per-task pricing can climb with volume |
| LLM-based or agent-based automation (a language model reads, classifies, drafts or chooses next steps) | Inputs are unstructured text, such as emails, varied documents or free-form requests, and some judgment is needed | Outputs vary from run to run; needs confidence thresholds, testing on real samples and human review for high-cost errors |
| Custom machine learning (a model trained on your historical data) | You need a prediction or score, such as demand, churn risk or fraud likelihood, and have enough labeled history | Needs data preparation, retraining and drift monitoring; longest path to first result |
In practice many working automations combine layers. A common shape is a language model that reads an incoming email and extracts fields, deterministic rules that check those fields against your records, and a workflow tool that updates systems or routes the case to a person. Ask the firm to draw this for your process and to explain why each step uses the technology it does.
If agents are on the table, read our explainer on how agentic AI works in real applications first, and compare packaged options in the AI agent tools directory. For internal task and approval flows, an off-the-shelf tool from the project management tools category may already cover what you need, and is worth ruling out before paying for a custom build.
Where the automation has to read from and write to several existing systems, the integration work often outweighs the AI work. Our guide to AI integration services covers those patterns and the effort they involve.
Step 3: What to ask an automation firm
General vendor checks still apply: relevant past work, security practices, clear pricing, written ownership of code, data and workflows, and no promises of guaranteed results. Keep that part of the evaluation short and spend your time on the three areas below, because that is where automation projects differ from other software projects and where proposals are most often vague.
Exception handling
Every process has cases the automation will not handle: a missing field, an unfamiliar document layout, a customer who replies with a question instead of an answer. What the system does with those cases decides whether staff trust it.
- What happens when the automation cannot complete a case? Does it stop, skip, guess or hand off to a person?
- Where do exceptions land, who is notified, and what context does the reviewer see?
- For LLM steps, what confidence or validation check decides whether output is used automatically or sent for review?
- What share of cases do you expect to go to exceptions at launch, and how was that estimated? Ask them to test on a sample of your real cases, not a demo dataset.
- Can a person correct an exception and have the case continue, or does it start over?
Designing the review step itself, meaning who approves what and how to avoid rubber-stamping, is covered in our post on human-in-the-loop AI.
Monitoring
Automations fail quietly. A login expires, a supplier changes its invoice template, a model starts misclassifying a new kind of request, and nobody notices until a customer complains. Ask:
- What is logged for each run, and can we see it without asking you?
- Which alerts exist for failures, slowdowns, unusual exception rates or no runs at all?
- For AI steps, how will accuracy be checked over time? Is a sample of outputs reviewed regularly?
- Will we get a simple report of volume handled, exceptions and time saved against the baseline we recorded?
For a structured way to discuss AI risk with a firm, the NIST AI Risk Management Framework, released in January 2023 for voluntary use, is a useful shared reference. You do not need to adopt it formally to ask a vendor how their monitoring maps to it.
Who maintains it after go-live
This is the question most often left until the end, and the one that most affects total cost. Automations depend on systems and rules that keep changing, so maintenance is ongoing work, not an optional extra.
- Who fixes it when it breaks, and within what response time?
- Is maintenance a retainer, a block of hours, or billed per change? What counts as a fix versus a new feature?
- Who pays for usage costs such as platform licenses, bot runtime or model API calls, and how will those scale with volume?
- What documentation, credentials and source code do we receive so that our team or another firm could take over?
- Who on our side owns the process and decides when business rules change?
What a sound proposal looks like
When proposals arrive, the strongest ones tend to share a few traits. They restate your process in their own words and get it right. They name the approach for each step and say why. They give an expected exception rate tested on your data, or propose a short paid discovery to establish one. They include a monitoring plan and a maintenance arrangement with a price, not just a build quote. And they define success against the baseline you recorded, not against a generic return-on-investment figure.
Be wary of proposals that lead with an agent that will “handle everything,” skip the question of exceptions entirely or treat support as something to discuss later. If you are still deciding whether to build this capability internally instead, our comparison of agencies versus in-house developers walks through that trade-off.
When you have a scored process and a baseline in hand, you can compare firms that build this kind of work in the AI/ML development provider directory.
Ready to Transform Your Business with AI?
Explore top providers, collaborate with trusted partners, and take the next step toward innovation.
Frequently Asked Questions
Pick one that runs often, follows rules a new employee could learn from a checklist, has a tolerable cost when a mistake slips through, and already has its inputs in digital form. A process that scores well on all four usually pays back faster than a bigger, messier one. Avoid starting with work your team still argues about, because automation will copy that disagreement.
If the inputs are structured and the decision logic can be written down as rules, classic rules-based automation or RPA is cheaper to run and easier to audit. Language models earn their place when inputs are unstructured, such as free-text emails or varied documents. Many working automations combine both: a model reads and classifies, and deterministic rules decide and act.
Someone has to watch it, fix it when an upstream system changes and update it when the business rules change. Agree in writing who that is, how quickly they respond, how changes are requested and priced, and what documentation and access you receive so another team could take over if needed.
Record a baseline before the build: volume per month, hours spent, error or rework rate and turnaround time. After go-live, track the same figures plus the share of cases the automation completed without a person touching them. Without a baseline, any savings claim from you or the vendor is a guess.
Written by: AIML Marketplace Team
