Build or Buy Your AI? What does this mean?

I had an awesome time at KonnectHouse Tech in Chicago this week, learned a lot about enterprise AI adoption. One of the central themes was build or buy, use AI or not…what was less talked about was HOW you actually build this stuff. I can punch a question into a chatbot window and get a result. But does that result happen every month, every day, every week? If I come back next Friday and the answer is different, how do I find out what changed? Who is managing and improving the rules and the data they run against? And perhaps before we start asking those questions, what is the technology and infrastructure and company specific guardrails / policies you need to use and understand to do any of this?

What I loved most about Day 2 of KHT was hearing people get into the process. One comment stuck with me.

I can ask AI to evaluate an RFP response. Getting that to work every time somebody submits one is a different job. What triggers the review? Where does the response live? Which decisions can I write a rule for, and where do I need the AI to make a judgment? That’s the application somebody has to build.

This is very similar to the exercise you went through (or should have) before you took an internal process or business function like let’s say Accounts Payable Invoice management, and moved it to an offshore provider. Or when you renewed or took this work out for RFP - another great time to reset or ‘mature’ a process and a contract by putting pressure on your vendor to agree to changes.

Before this task goes, can you reengineer (i.e. improve) the process to automate / use technology to eliminate a step? Which part needs somebody in another department, and can the offshore team work directly with them? And what ‘level’ do I outsource at - does the entire Accounts Payable team all get laid off and that work lifts and shifts over to the vendor (or into the new AI enabled application) or does the ‘approval’ process need to stay with a smaller internal team that gets to keep their job?

This is the level of detail you need to go to when evaluating enterprise processes to be automated with AI - it’s not a C-level activity - follow the org chart in my drawing down from CFO - accounting - AP - THAT is where your list of processes begins…Invoice Submission, approval, disbursement. Say somebody tells me they built an AI agent for AP. Which of those three things does it do? And if I give it an invoice today, how do I know the supplier got paid?

Those are questions a buyer can ask even if they have never written a line of code. And the answers would change what I was willing to pay.

What changed in the build-or-buy conversation

Early on, a company could make AI usage go up by giving people access and encouraging them to use as much of it as possible. “Token maxing” makes a certain amount of sense if the object is to get people trying the technology. It makes for a pretty interesting conversation about the bill when nobody agreed on what work was supposed to get done.

Buying an application for a particular function or process management step is now possible and there are great tools available for agent orchestration / process management ‘in a box’. It’s like hooking up the end point in a line of stuff you need to connect to and with (people process tools) in order to implement the desired AI outcome whether that is less humans, faster processing times / ‘more output’, less errors, or all of the above.

Starting at one end of the line with a good ‘process in a box’ is a better place than trying to start in the middle so if you have a targeted use case that will drive a lot of value especially for a process like e.g. contract intake which doesn’t formally exist in a tool or application enterprise wide yet, you can buy this with an orchestration tool.

Now practitioners are taking a second look at building parts of those applications themselves. The people who know their business particularly well can see places where a pretty good general-purpose product still leaves a surprising amount of work for their team - a process within the application. And the tools for building something specific to their process are more accessible than they were when they first made that buying decision.

There was a real range of experience in the room. Amer Chowdhry's presentation on Dell was incredible, the sequence he described, simplify, standardize, automate, would be familiar to a lot of people who've moved work to a service provider. Amer also alluded to the fact that Dell also has support and client services businesses doing this kind of work at a scale most companies don't have access to, and a cloud computing hardware empire at its disposal.

The Wednesday build-or-buy panel started talking about application layers and the connectors people were using, which means even ‘normal’ non-tech companies are catching on. But that gap exists still across the world in terms of education and readiness for any of this, two companies can sit through exactly the same presentation and come home with different answers because of the people, systems and management capability they have had in place since the advent of AI.

What happens to a contract after you upload it?

A week before KHT, I was talking to my eng team about improving the data we can get from a set of signed contracts. I could already give an AI a file and ask it to classify things, but when we open this up to more and more metadata fields what would happen if we had to manually check every field of every contract, and worse what if I was the one who had to do it (I already knew the answer)...How capable would AI models be at doing this across THOUSANDS of contracts and a massive scale.

Say you’re the buyer and a supplier calls about an unpaid invoice. The invoice says net 30. You pull up the signed contract and the terms are net 30 there too. Then the category manager sends you an amendment changing the terms to net 45. The question I’d give the AI is a little more specific now. Which terms applied to THIS invoice, and where in the signed documents did you find them?

Let’s say the amendment took effect July 1 and applies to invoices issued on or after that date. A June 28 invoice and a July 2 invoice from the same supplier have different terms. If your supplier database has one field called “payment terms” and you change the 30 to a 45, what happens when AP opens the June invoice again? This is what people mean by the ‘schema’. You decide what the database needs to remember: which agreement, which version, the effective date and what starts the payment clock. Somebody in AP or procurement has to explain that last part. Thirty days from receipt and thirty days from the invoice date are two different deals.

This in my opinion is a good representation of the ‘frontier’ of the procurement / outsourcing AI boom we are in, it’s a seriously complicated and vast thing to solve for at not just thousands of companies but thousands more departments and unique scenarios that matter…something to lose sleep over.

How does AP get those terms? A connector can use an API (the way two applications exchange records) to look them up. The contract might name one supplier, but AP has three vendor IDs for three subsidiaries. Which ID did we buy under? Somebody has to map those records before AP can use the answer. That’s integration work.

What are you buying when you buy the drag-and-drop tool?

In Microsoft’s Copilot Studio you can draw the flow: invoice received, look up the supplier, apply the rule, send for approval. Put the AP lead in front of that drawing. Where does the invoice go when the person who has to approve it is on vacation? She can tell you what’s missing before anybody starts building. Microsoft’s flow designer · Run history

What access comes with that connector? The agent needs to read payment terms. Does that give it permission to change bank details too? Who can do that today? If two employees have to approve a bank change, decide where both approvals happen before giving the new application write access.

So what part would I buy? If a vendor can already read the invoice and send the fields into our finance system, great. What does the vendor do when the supplier resends it under a new company name? Our AP specialist may know from experience that both copies are the same bill. Does the product catch it? Does the vendor work that exception for us? Or is our AP specialist still doing exactly what she did yesterday, now with another queue to check?

I’d give both copies to the vendor and to our internal team. If one quotes me for reading two documents and the other quotes me for getting one approved invoice into AP, I’ve got two different scopes of work. Fix that before comparing prices.

Where does AP’s job actually end?

Go back to the three boxes in my notebook. Submission, approval, disbursement. If I outsource all three, what am I paying the provider to deliver? For this example I’d write it into the agreement this way: the right supplier gets the approved amount, within policy, by the date we agreed to pay them. Now I can take an invoice all the way through and count whether we got it right.

Say the budget owner approved the invoice and Treasury sent the payment file. AP’s screen says ‘complete.’ The bank returned the payment. Did we pay the supplier? Anybody who has had to take that phone call knows the answer. Treasury has the settlement record. AP has the open invoice. Somebody has to connect the two and own the follow-up, even if those people report to different departments. Where would that time show up in your vendor’s price? Where would it show up in your internal build estimate?

Think about Dell’s examples of doing complicated work processing like this. By testing these with AI and how it’s currently done side by side, or the ‘parallel run’ as we called it in Google terms, where you don’t let the new provider charge you until they prove they are hitting the same SLAs as the outbound vendor who gets to keep charging you in transition support.

If the new vendor or end to end AI enabled process hits 100% SLA on day one…are you producing the same output? If so congrats, start looking at the next task, the next process, the next functional department. If not…if the new AI process hit 50% vs. the old’s 90%, what sub step went wrong? What data did the AI use, what did the old process use / have that the AI didn’t? What ‘fixing’ needed to be done? Tweak the ‘rules’ (like the prompt if you want to think about it that way or the GPT “project rules” document) and try it again next time, maybe it will get even worse! Wire the thing into your databases, keep trying, testing, running the results and run logs through different models asking different questions until you find the problem. You will.

What does it cost us to do this today?

When I did a job family analysis for outsourcing, we’d put the people down the left side of a spreadsheet and the processes across the top. Sit down with the AP lead. How much of her week goes to submission, approval, disbursement? If she spends Friday afternoon sorting out payment exceptions, that time belongs to the payment process even if her title is ‘Finance Manager.’ Allocate her fully loaded pay across everything she does. The row has to add up to 100%. Only the portion spent on AP belongs in this business case.

Do the same with the software license and the vendor fee. Here’s where I’ve seen outsourcing math go wrong. We buy a team, then our manager still assigns their work, answers their questions and checks what comes back. I’ve called that ‘shadow temp operations.’ If our manager directs the vendor’s people day to day, I’d ask Legal about co-employment too. If the vendor fee goes in the spreadsheet, the manager’s time has to go in too. Otherwise nobody’s priced the whole job.

Let’s put some fictional dollars against those three boxes. Submission costs $64,000 a year, approval $55,500, disbursement $42,500. We spend $162,000 and receive 12,000 invoices. That gets me $13.50 per invoice. But only 11,400 payments met the conditions we just wrote into the agreement. What did it cost me to pay one invoice correctly and on time? $14.21. Which number would you use to judge whether the AI was effective?

Keep it, build it or buy it?

Keep doing what we’re doing for three years and the cost is $486,000. Now take just submission. If AI can handle 80% of that $64,000 job, can we take 80% of the money out? Say we can avoid three quarters of the cost of the work it covers. That’s $64,000 × 80% × 75%, or $38,400. Do that for approval and disbursement too. In this example $65,700 a year can come out across all three steps, and $96,300 stays with us. Both teams still have to deliver the same 11,400 good payments.

Suppose my team needs $72,000 to build it and $16,000 a year to keep it running. The vendor wants $8,000 to implement and $44,000 a year after that. Add the $96,300 of work we keep to each option. Now look at what happens depending on which year you’re asking Finance to fund.

On a one-year budget I’d buy this one. Buying saves $13,700 against today. Building costs $22,300 more than today. But the build costs $28,000 less than buying every year it runs. It takes 27.4 months to earn back that extra $64,000 up front. Give me three years at the same volume and service level and the build saves $77,100 against today, $20,000 more than buying saves. If we’re replacing the finance system in two years, I’d bring that up before somebody signs a three-year business case.

What’s inside that $16,000 a year? Who fixes the connector when the finance system changes? Who updates the rule, runs the two invoices through it again and checks the result? Put somebody’s time against each job. I’d ask the vendor the same questions about the $44,000. A contractual commitment to maintain the connector could be worth paying for. A support number I can call so somebody can tell my team how to fix it is a different purchase.

One more question for Finance. Of that $65,700, which expense can we actually stop paying? If AP still has the same people on payroll, the same vendor under contract and the same licenses, our three-year cash spend is $606,000 to build or $626,000 to buy. We may still have a good reason to do it. Maybe the same people can handle next year’s volume and we can cancel a planned hire. Put the role, the date and the avoided hire in the case. That’s a claim the CFO can come back and check.

Which task would you test first?

Go back to the AP boxes in my notebook and pick one task inside submission. Can we check an invoice against the signed contract? Sit down with the person doing that check today. Give them the invoice, the PO and the amendment that changed net 30 to net 45. Which terms apply to this invoice? What gets recorded if the documents agree, and who gets the invoice if they don’t? AP can define what a correct answer looks like before we ask anybody to build or sell us that piece.

Now give the internal team and the vendor that same task. Have each run an ordinary invoice and one with conflicting terms. Where does a person have to step in, and did their time make it into the price? Next Friday, have somebody else run it. If they can explain which terms applied and why, we’ve got some evidence for a build-or-buy decision on that part of AP. We can leave payment approval with the person who owns it and move on to the next task.

Tell me which step you’d keep in-house and why