For years, your IT consultant kept the lights on. They patched servers, migrated you to the cloud, and probably saved you from a few disasters you never heard about. None of that qualifies them to build your AI strategy, and the gap between those two jobs is wider than most companies realize until they’re six months into a stalled pilot.
The incentive problem nobody names
Most IT consultancies get paid to keep things stable. Their whole model runs on long contracts, predictable maintenance fees, and clients who don’t rock the boat. Surprises are bad for business.
AI work is close to the opposite. You try things quickly, some of them are set up expecting to fail, and when something isn’t working you pull the money and move it somewhere else.
So if you ask your existing IT vendor to run an AI program properly (kill projects early, shift budget around, admit nobody really knows how this will play out), you’re asking them to undercut their own revenue. They know that. What you get instead is AI dressed up as another infrastructure rollout: a multi-year roadmap, a steering committee, and a slide deck with a title like “AI Value Realization” that nobody can argue with because it doesn’t actually commit to anything.
Everyone in the room knows it has no teeth. The team that built it was never set up to deliver something with teeth in the first place.
Data readiness is where the real work starts
Most generalist consultants treat an AI engagement like a server migration. They go through the logs, look over the network setup, maybe run a security audit. What they usually skip is the question that actually matters: is your data in any state to be used by a model?
Before anyone gets into which model to pick, the project lives or dies on the data itself. Is it accurate? Do you know where it came from? Who can access it, and is any of it documented?
In practice, AI projects fail far more often because of messy data than bad algorithms. Customer records scattered across three systems, fields nobody can explain, dates formatted four different ways. A specialist looks at that first. A generalist starts with the infrastructure, because that’s the part they know how to check.
Governance and cost exposure that generic IT frameworks miss
AI introduces new challenges for cost, security, and compliance that aren’t easy to solve using traditional IT strategies. For example, hyperscaler platforms charge per token per cycle, so a poorly configured model can blow your compute budget in days. There will always be unknowns for mission-critical applications, but the need to be upfront and transparent about what you know-and-what-you-don’t about your AI models and data is a legal obligation according to the European Union’s proposed AI Act, as well as GDPR’s requirements for fit-for-purpose data.
And while data residency, data protection, and data retention requirements may all be regularly addressed within your current security policy, having an audit trail that demonstrates bias accountability may not be. If you don’t have an in-house AI team, the pragmatic move is to evaluate the best ai consulting firms rather than defaulting to the same partner who manages your network – because the firm patching your firewalls isn’t the firm that should be setting your model governance policy.
The technical gap is bigger than it looks
Developing AI in production demands expertise not typically found in a standard IT consultant’s repertoire. For instance, implementation of retrieval-augmented generation, which requires a model to retrieve information from your enterprise data rather than infer it, involves the sort of architecture-level choices that many jack-of-all-trade practitioners have never been responsible for.
Monitoring, retraining, and general care and feeding for a live model – often lumped into the term MLOps – is an entirely different role than being on call for a user’s issue with the six nines system you stood up. Someone has to track hallucination rate, sample output and relevance, and many other measures of the model to know if it’s doing what you paid for, and generalists most likely do not have recommendations for what those rates and performances should be. If they can’t tell you how they would determine if their solution is working, they certainly can’t evaluate if it is performing as intended.
The often-underestimated prompts and reinforcement learning avenues can sound like skills until you’ve had a rival try to prompt-inject your customer-facing chatbot. Build versus buy – deciding if a SaaS tool or API or a specific model offers the best route for what you’re trying to accomplish – is often passed over by generalists as just another procurement style decision, rather than a make-or-break, high-visibility management problem with real cost and risk tradeoffs that show up months later.
Why speed matters more than scope
The most powerful AI programs do not resemble traditional IT projects at all. They build a prototype of one specific application, tie the results – costs saved, increased revenue, reduced service requests – to a cash value, and then within a few weeks determine if it should be scaled up or shut down. Gartner estimated in 2024 that by the end of 2025 30% of generative AI programs would be stopped after a successful proof-of-concept phase due to unmanageable cost, lack of data quality, or failure to show a clear business case. But that’s not a failure of AI. It’s a poorly designed plan, often imposed by a vendor who wanted to give you a long, expensive project instead of a short, fast test.
General consultants offer you a timeline. They can and will bill you by the hour, no matter how long the project takes. AI experts sell you a product. They’re stuck with the cost of a wasted experiment if you reject it. These things fail because, when you investigate what went wrong, you find that the neat output of the model couldn’t be integrated with the software that your team uses every day rather than that the model itself didn’t work. That’s a scoping issue, not a technology issue.
The real question to ask your current vendor
If you are considering engaging your current IT partner for another AI transformation project, there are three things you should ask them: how do they measure the hallucination rate, what would be their approach to decide between build and buy for your anticipated solution, and what KPI in the first 90 days would be tied to a commercial term in dollars. If everything you hear still sounds like infrastructure with AI dressing added to it, you are done.