Enterprise software vendor evaluation needs AI scrutiny, JPMorgan VP says

Enterprise software vendor evaluation should start with how a system will be used inside the business, not with a vendor’s feature grid, Sudhakar Shivaraju, VP of global real estate systems at JPMorgan Chase, wrote in a July 27 guest post for CIO Dive. His argument is aimed at CIOs facing a crowded software market where AI claims, roadmap promises and migration costs can distort buying decisions.

Shivaraju based the guidance on two formal enterprise systems evaluations he said he has led across different industries and at different points in his career. No specific vendors, contract values, implementation budgets or performance benchmarks were disclosed.

The central warning is familiar to enterprise buyers: a scorecard can look rigorous while measuring the wrong things. Shivaraju wrote that CIOs should organize comparisons around functional domains tied to actual operating needs, rather than evaluating products line by line against sales collateral.

How should CIOs evaluate enterprise software vendors?

Shivaraju said CIOs should assess vendors across domains such as security and compliance, integration and API architecture, user experience, support, roadmap transparency, AI capability and total cost of ownership. He argued that these categories should then be weighted according to the organization’s constraints, because a global regulated enterprise will not have the same priorities as a smaller company operating in one region.

That weighting is where many evaluations can go wrong. Security, compliance and data residency may deserve heavier emphasis in regulated environments, while support responsiveness may matter more for organizations that need rapid issue resolution at scale. Shivaraju’s point is that a generic template can produce a decision that appears defensible but is misaligned with the business.

AI claims need their own scorecard

Shivaraju singled out AI as a category that should not be reduced to a yes-or-no feature. Enterprise vendors now frequently market AI functions, but buyers need to separate production capability from roadmap language.

He said CIOs should ask whether a vendor’s AI works against the company’s own data or sits as a generic layer, how data privacy and handling change when AI is involved, and whether the feature is already live or still promised. Those questions matter more for organizations subject to data residency rules or other regulatory obligations.

The same skepticism applies to product roadmaps. Shivaraju said conferences, roadmap sessions and direct conversations with vendor executives can provide useful evidence, including release timing and commitments to address known gaps. But anticipated features should be scored separately from current capabilities, rather than treated as if they already exist.

Support, roadmaps and switching costs are separate risks

Shivaraju also warned against blending vendor support with roadmap transparency. A company may be clear about future releases while still underperforming on support tickets. If support speed is a material risk, he said buyers should address it directly, including through negotiated service-level commitments.

Total cost of ownership should include the disruption that comes with moving systems, not just license comparisons. Shivaraju listed data migration, retraining, workflow rebuilding and re-establishing integrations as costs that are often underweighted because they are harder to quantify.

That can leave an incumbent vendor in a stronger position than a feature comparison suggests. According to Shivaraju, even a vendor with known weaknesses may remain the better option if the operational cost of switching outweighs the benefits.

The broader takeaway for technology leaders is procedural rather than vendor-specific. Shivaraju said the structure of the evaluation, including domain selection, weighting, treatment of roadmap claims and accounting for switching costs, often influences the quality of the decision more than the vendor field itself.

This story draws on original reporting from CIO Dive.