The CTO just got asked about AI. Not in a meeting where she had time to prepare. A board member mentioned it. The CEO mentioned it. A large customer mentioned it. Now the company wants a roadmap, and the CTO has to explain either why they are building AI features or why they are not. The hardest part is that both options require a defensible answer.
The temptation is to treat this as a technology question. Which model should we use. Which framework should we build on. Which vendor should we pay. Those questions have answers. But they are the wrong questions. The failure modes are rarely about the model.
Two industry-scale research efforts from 2025 and 2026 show this clearly. Gartner, surveying technology professionals and organizations across all sectors, forecasted that over 40 percent of agentic AI projects will be cancelled by the end of 2027. The reasons Gartner cites are escalating costs, unclear business value and inadequate risk controls. That is not a model problem. That is a systems problem.
MIT's Project NANDA released its GenAI Divide research in July 2025, analyzing 300 public AI deployments, conducting 52 structured executive interviews and collecting 153 survey responses. The headline is sharp: 95 percent of enterprise generative AI pilots produced no measurable profit and loss impact, despite an estimated 30 to 40 billion dollars of enterprise investment. The research carries caveats worth naming directly. The study is preliminary and has not been peer reviewed. Critics have pressed on the six-month measurement window and the sample size. But the directional finding is solid and consistent across multiple independent sources. When AI is added to a product, something is going wrong often.
The Condition That Determines Readiness
A mature B2B product is ready for an AI feature only when seven conditions hold at once: worthwhile business value, usable workflow fit, representative and governed data, feasible integration, measurable evaluation criteria, acceptable risk, and sustainable operating economics. If any one of the seven is missing, the correct next step is a scoped proof of concept or a discovery phase, not production development.
The shift in how technology leaders think about this is important, and it is relatively new. Five years ago, the question was whether the model was capable enough. Now the model is almost certainly capable. The question has moved entirely to the organization and the product around the model. Google's 2025 DORA research, drawing on survey responses from nearly 5,000 technology professionals, described AI as an amplifier of the surrounding system. The research summary is direct: AI does not fix a team, it amplifies what is already there. Strong teams use AI to become stronger. Struggling teams find that AI only intensifies their existing problems.
That is the framework that matters for every AI readiness question. The model is not the constraint. The system around the model is constraint.
Seven Things That Have to Be True
Business Value: What Problem Does This Solve and Is AI Better Than Alternatives?
Ask yourself concretely. What measurable problem does this AI feature solve? What happened today without it? Is the problem frequent enough to matter for the business? Does solving it save time, reduce cost, improve the user experience or create new revenue? Is AI materially better than deterministic automation or ordinary software for solving this problem?
Good indicators that a feature has real business value include measurable reduction in support hours, processing time cut in half, higher customer conversion rates, visibly reduced user effort or increased adoption of a feature that previously was underused. These are things you can track before you build.
The single biggest red flag is this: competitors are adding AI. That is a market signal, not a business case. If the problem does not connect to something your customers are actively struggling with, then building to match a competitor is how you spend your engineering budget without moving revenue.
A support team that handles hundreds of tickets daily and cuts volume by automating routine classification has real business value. That is a concrete problem the team experiences every day. An ERP that adds an AI recommendation engine just to say it has one is not solving a problem. It matches a feature list.
Workflow Fit: Does AI Enter Where Users Really Work?
AI rarely makes work simpler. It usually changes how work is done. Where exactly does the feature enter the existing workflow? Does it recommend, summarize, classify, extract, generate or execute? Can the user verify the output easily when it matters? What happens when the AI is wrong, and how much does the user have to check?
This is where mature B2B products face a specific risk. Your users already have established processes. They know the shortcuts. They know the gotchas. An AI feature that looks impressive in a demo but requires users to double-check everything might create more work than it removes.
A knowledge assistant that searches product documentation and drafts an answer for internal support staff to refine has good workflow fit. The staff can verify the answer quickly and push back if it is wrong. A feature that makes the user's job harder because it introduces an extra step or an extra screen has poor workflow fit regardless of how accurate the model is.
Data Readiness: Is Your Data Useful and Legally Accessible?
Do you have enough representative data? Is it standardized so the model can make sense of it? Is it trustworthy? Is important context trapped in customer-specific configurations, in PDFs nobody has structured, in databases that do not talk to each other? Can the AI legally and securely access the data it needs? Is the source information current enough to be useful?
This is where many projects discover their first hard constraint. The model is capable. The data is not. A healthcare software vendor might have years of clinical data but be unable to send it to external models because of privacy rules or customer contracts. This is the same fragmentation challenge that surfaces when teams apply AI to legacy AS400 data. The model could work if you could feed it the data. You cannot feed it the data. That is not a temporary limitation. That changes the modernization path entirely.
A financial institution might have transaction data spread across systems that do not sync. Classification or analysis looks straightforward until you realize that a single customer's data lives in four different schemas. The work to standardize that data often exceeds the work to build the feature itself.
Integration Feasibility: What Has to Connect and How Hard Is It?
The AI rarely operates alone. It usually needs access to something. CRM data. ERP data. Electronic Health Record systems. Your product database. Customer records. Support knowledge bases. APIs. User permissions. Workflow engines. Document stores. If building those connections is difficult, adding AI exposes existing integration debt rather than solving it.
This is the point where many teams discover that choosing the AI model was simple compared to integrating it with the rest of the product. The model you choose might have excellent capability, but if it cannot reach the data it needs to work with, the capability does not translate into a product.
Evaluation Readiness: How Will You Know It Is Good Enough?
Before building production AI, answer this question precisely: How will we know this feature is good enough? Possible metrics include extraction accuracy on representative test sets, task completion rate, user acceptance rate when they have a choice, hallucination or error rate, escalation rate when the system hands work to a human, response latency, cost per completed task, time saved compared to the manual process, or support ticket reduction.
The critical mistake is building first and evaluating later. Teams often judge a feature from a handful of good demo prompts and assume it will work at scale. Representative evaluation cases have to exist before you deploy. If you cannot define what success looks like, you cannot build it and you cannot know when you have achieved it. Once a feature reaches build, the same discipline applies to verifying AI-generated code before you ship it.
Risk and Compliance: What Could Go Wrong and Who Gets Hurt?
Risk depends heavily on what the AI is doing. Searching product documentation is lower risk. Drafting internal summaries is lower risk. Suggesting support responses for a human to refine is lower risk. Extracting structured information from documents for human review is lower risk because the human is in the loop.
Clinical or financial recommendations carry higher risks. Automated actions affecting customers carry higher risks. Regulatory classifications carry higher risks. Anything where an incorrect answer is difficult to detect or difficult to reverse carries higher risk.
Some teams deliberately avoid building certain AI capabilities specifically because they would increase exposure in medico-legal liability or medical device regulation. That is not timidity. That is a rational engineering decision built on known constraints.
Operating Economics: What Does This Cost at Scale?
The proof of concept is rarely the expensive part. Production costs are different. Production cost includes model and API usage, embeddings and vector storage, GPU or inference infrastructure, observability and monitoring, evaluation and testing, retry loops when the AI fails the first time, human review of edge cases, security controls, and the ongoing work to maintain prompts and models as things change.
Many teams build proof of concept that costs two hundred dollars a month per developer and then discover that production deployment at full scale costs two thousand dollars per developer per month. The cost per token is cheap. The cost per business task completed is what matters. If the AI has to be called ten times to complete one task, and each call costs something, the economics changes.
Measure cost per successful business task rather than cost per token. Connect that back to the business value question at the beginning. If the task saves an hour of manual work and costs thirty dollars in AI infrastructure, the math works. If the task saves five minutes and costs fifty dollars, it does not.
Scoring the Idea
Take a single proposed AI feature and score it from one to five on each of the seven dimensions. One means problematic, five means strong. A score of five means the feature scores well across a dimension and nothing is blocking that area. A score of one means there is a significant constraint that has not been resolved.
The scoring thresholds are an editorial framework rather than an industry standard. Publish them plainly, so readers know these are guidelines not commandments.
A total score of 28 to 35 means the feature is a candidate for production planning. The risks are known and manageable. Start engineering.
A score of 21 to 27 means running a focused proof of concept first. Use the proof of concept to test the highest-risk assumptions and gather enough evidence to decide whether production development makes sense.
A score of 14 to 20 means investigating prerequisites first. One or more of the seven dimensions has a significant gap. Do not start production development until that gap is filled.
A score below 14 means deferring. The feature is not ready. That is not forever. It is a signal to focus on something else and revisit this idea in six months when the constraints might be different.
Not sure where your AI idea scores?
We run structured AI discovery for established software products, so you know what the feature will take before it reaches an engineering roadmap.
Three Ideas, Scored
Knowledge Assistant Over Product Documentation
A product engineering team wants to build an AI assistant that searches for product documentation and drafts answers to internal questions. The goal is to reduce the time support staff spend hunting through documentation.
Business value scores are high. The team can measure support hours before and after. Workflow fit scores high because support staff can verify and refine the draft in seconds. Data is in-house documentation, structured and current, so data readiness scores high. Integration is straightforward because the tool sits on top of existing docs. Evaluation is clear: did the draft answer match what an experienced staff member would have written. Risk is low because a human reviews everything before it reaches customers. Economics are reasonable because you are replacing labor with infrastructure cost.
A score of 28 to 32. Production candidate. Start engineering.
Automated Contract or Document Extraction
An enterprise software company wants to use AI to extract key terms from customer contracts. The business case is time savings for legal and operations staff.
Business value is real but constrained. You save time only if every customer sends documents in a consistent format. Data readiness is the constraint. Half the customers send PDFs scanned from paper. Half send Word documents. A few send email attachments. A few have custom formats because they built the document in their own system. Validation rules have to be defined per customer variant, not globally. Integration depends on how the extracted data flows downstream. Is it only for display or does it affect business logic?
A score of 16 to 22. Not production ready. Run a proof of concept against the messiest document set first, not the cleanest. See if the approach holds up when documentation is imperfect.
Autonomous Decision-Making in the Customer's Domain
A financial software vendor wants to use AI to make recommendations about investment or credit decisions for customers. The business case is significant if it works.
Business value scores well on paper. Workflow fit is unclear because you need to understand how customers would use autonomous recommendations versus reviewing human ones. Data has to be representative of real decisions, not just historical transactions. Integration might require security and audit controls that do not exist yet. Risk scores poorly because wrong decisions have a measurable financial impact and the customer bears the risk. Evaluation is the constraint: you cannot know whether the model's decisions are sound at scale without extensive testing against held-out data representing the full distribution of cases the model will encounter.
A score of 15 to 20. Defer. This is not a 2026 project. It is a 2027 project if you invest in evaluation infrastructure and risk control in 2026.
What to Validate in a Four-to-Six-Week Discovery
When a feature scores in the 21 to 27 range, run a discovery phase before committing to production engineering. The discovery should be specific and time boxed.
Assemble a representative evaluation set with real examples that include edge cases, not just the clean cases. Test data access and quality against actual customer data, not sanitized test sets. Prototype the single highest-value workflow entry point, not the full feature. Measure cost per completed task at realistic volume, accounting for failure retry and human review. Document the failure modes and when the system would require human intervention.
Four to six weeks is enough time to answer whether you should proceed. If you cannot answer in that time, the feature is not ready.
The Signals That Say You Are Not Ready Yet
Nobody can name the metric that would change the decision if you moved it. The data lives in customer-specific configurations and is not portable. No one has defined what a wrong answer costs to the business. The only cost estimate is the token price from the API vendor. The use case originated in a board slide rather than in a support queue or a customer conversation.
FAQ
What is AI product readiness?
AI product readiness is the assessment of whether an existing software product can support an AI feature in production. It covers business value, workflow fit, data quality, integration feasibility, evaluation criteria, risk exposure and operating economics rather than model selection alone. The readiness is a property of the product and the organization around it, not of the model.
Why do most enterprise AI pilots fail?
MIT's 2025 GenAI Divide research attributes the failure rate to integration and workflow problems rather than model quality. Gartner separately cites escalating costs, unclear business value and inadequate risk controls as the drivers behind projected agentic AI cancellations through 2027. In both cases, the problems are in the system, not in the model.
How do you calculate ROI on an AI feature?
Measure cost per successful business task rather than cost per token. Include inference, storage, retries, human review, evaluation and maintenance on the cost side. Quantify the time, support load or manual effort removed on the value side. If the task saves two hours per week and the feature costs three hundred dollars per month, the break-even is when the saved time exceeds the cost. Calculate that before you build.
Should we build an AI feature just because competitors have one?
Competitive pressure is a market signal, not a business case. If the feature cannot be tied to a measurable problem your users encounter frequently, a narrow proof of concept answers the board's question at a fraction of the cost of production development. Start there rather than starting with the assumption that matching a feature list is the same as meeting a customer's need.
