Addressing the AI Readiness Gap in Healthcare
- by Jeff Collins
Key takeaways
- More than half of healthcare providers and payers are already using AI in some capacity.
- New initiatives can stall when AI workloads run on infrastructure designed for different requirements.
- Implementing a hybrid environment with dedicated, single-tenant servers plus cloud resources enables healthcare organizations to match each AI workload with the right infrastructure.
Healthcare organizations are moving ahead rapidly with AI adoption. In fact, according to a recent survey, 56% of healthcare providers and payers are actively using AI today—up from 43% the year before.1
Most AI initiatives in healthcare start small. Organizations often launch a pilot or proof of concept, exploring a single use case. To host these new workloads, they might use the cloud or other infrastructure that is already available.
That’s a reasonable way to start. But what happens when it’s time to scale? Many organizations do not have the infrastructure in place to support the use of AI in production—and the public cloud isn’t always the answer. Addressing that readiness gap with the right infrastructure will be critical for realizing the potential benefits of AI without incurring excessive costs.
Finding Cracks in the Foundation
AI workloads are not entirely different from typical healthcare applications. Electronic health record (EHR) systems and patient portals, for example, have largely predictable, steady demand. There are no huge sudden spikes that require more resources. Similarly, AI inference workloads run steadily once in production, though they have much higher and more continuous resource needs than EHR systems or patient portals.
Organizations sometimes launch AI pilots in the cloud so they can experiment while avoiding upfront infrastructure expenditures. If a cloud-based pilot is successful, it seems like a logical step to run the production workload in the cloud as well. But hyperscale cloud environments are built to reward workloads that spike and settle. An AI inference workload, for example, does neither. It just runs and keeps running.
When healthcare organizations run those AI workloads in the cloud, they find that their bills scale just as fast as their workloads. Instead of paying a flat fee, they are paying more and more every month.
Understanding Why AI Initiatives Stall
The number of AI workloads in production is certainly growing. But there are many initiatives that stall before they can start delivering the benefits they promise.
Not all problems are technical. Rising cloud costs—which appear because of the mismatch between workload needs and the cloud’s elastic pricing model—can quickly put an end to an AI initiative. Some initiatives never make it out of the pilot stage because IT cannot accurately predict what it will cost to run a steady-state workload in the cloud. Others are canceled when those large cloud bills start to roll in.
Aligning Workloads with Infrastructure
To reduce the obstacles that can bring AI initiatives to a halt, organizations must align each workload to the right infrastructure. In some cases, the cloud will in fact be the best fit for AI workloads. For example, cloud infrastructure might make sense for using AI to analyze images in a picture archiving and communication system (PACS) or to find aggregate patient trends in large data lakes. The cloud’s elastic storage capacity and on-demand compute resources are designed to support these types of workloads.
By contrast, data pipelines needed for AI initiatives and production-scale inference workloads are better matched with dedicated, single-tenant servers. Those servers can support these steady-state workloads while eliminating the elastic pricing that can lead to surprise bills.
For many healthcare organizations, then, a hybrid environment—with a mix of dedicated servers and cloud infrastructure—is often the best approach. Organizations can capitalize on the advantages of each type of infrastructure and minimize penalties for running particular AI workloads in the wrong places.
The hybrid approach can work well overall for healthcare organizations, which have a diverse range of workloads. They can use dedicated, bare metal hardware for non-elastic workloads, such as patient portals or telehealth applications. Meanwhile, they can use the cloud for bursting workloads, like those that train AI models or assess risks for payers.
Preparing Infrastructure for AI
Having the right infrastructure in place is critical for succeeding with AI initiatives. But of course, what is right for one workload is not right for all. Carefully evaluating workload requirements and implementing the most appropriate infrastructure can help organizations move initiatives out of the pilot phase into production while avoiding unexpected costs. For most healthcare organizations, a hybrid model will help cover a full range of needs and provide the best foundation for delivering results from AI initiatives.
Learn how Hivelocity can help your healthcare organization build the infrastructure foundation for your AI initiatives.
FAQ
Q: How can infrastructure decisions cause AI initiatives in healthcare to stall?
A: If organizations select cloud environments, with elastic pricing, for steady-state AI workloads, they can be surprised by large cloud bills. While a cloud environment might be the right choice for an initial pilot, moving steady-state workloads, including AI inference, to dedicated bare metal servers can help rein in costs.
Q: Does moving AI workloads to bare metal mean giving up the cloud entirely?
A: No. Experimental workloads or workloads that burst, such as large-scale model training, can benefit from the elastic resources of cloud infrastructure. But workloads that run continuously, without bursting, are a better fit for dedicated, single-tenant servers.
Q: Why should healthcare organizations make infrastructure decisions early?
A: Organizations are eager to move from AI pilot to production, so they can start benefiting from AI capabilities. But without adequate infrastructure planning, initiatives could stall in production, as the true cost of running workloads at scale becomes clear. Making infrastructure decisions while creating an AI strategy helps avoid that point of failure.
Citation
1 NVIDIA, State of AI in Healthcare and Life Sciences: 2026 Trends, January 2026
