Build, Colo, Cloud or On-Prem: How Should the CIO Decide?

For years, infrastructure strategy often sounded simpler than it really was. Move to the cloud, reduce dependence on owned data centers, gain flexibility and avoid large capital investments. That logic worked for many organizations, especially when the priority was speed, but CIOs are now entering a more complicated phase.

Cloud costs are under greater scrutiny, AI workloads require unusually large amounts of compute and power, data sovereignty rules are becoming more important, and some applications still depend on predictable performance, local control or regulatory boundaries. At the same time, building or leasing new data center capacity can require enormous capital and long planning horizons.

The question for the CIO is therefore no longer whether cloud is good or bad. The better question is which workload belongs where, and the answer may be different for every major application, data set and AI workload.

The first wave of cloud strategy was about migration

A recent discussion on cloud modernization describes how many enterprises approached the first wave of cloud adoption. The goal was often straightforward: move workloads out of traditional data centers and into public cloud infrastructure as quickly as possible.

Lift and shift made that possible because applications could be moved without substantial redesign. That gave organizations easier access to infrastructure and helped reduce some of the friction associated with owning physical systems, but CIOs later discovered that moving an old application into a cloud server did not automatically modernize it.

Legacy applications could remain difficult to scale, consume more cloud resources than expected, create monitoring complexity and become expensive. The important lesson is that hosting location is only one part of the architecture, because moving a workload does not necessarily improve the workload itself.

The CIO is entering a workload placement era

The same cloud modernization discussion describes a second phase in which CIOs are asking more detailed questions about application redesign, cloud spending, AI and data platforms, and how multiple providers should be managed. The source argues that many enterprises are adopting more structured multicloud strategies and evaluating workloads individually according to performance, security and cost.

That shift reframes the problem. Instead of choosing one infrastructure model for the whole organization, CIOs can evaluate several possible destinations such as public cloud, private or on-premises infrastructure, colocation and purpose-built owned data centers.

The objective is not to choose a winner. It is to place each workload where its requirements are best served, and to make that decision using a consistent set of principles rather than individual team preference.

Public cloud still solves important problems

Public cloud remains attractive because organizations can access infrastructure quickly, experiment without waiting for physical hardware and expand capacity without immediately building new facilities. Modern cloud services can also provide access to data platforms, AI services, managed databases and development tooling that would be expensive or complicated to recreate internally.

For workloads with uncertain demand, public cloud can be particularly useful because capacity can scale without the organization committing to long-term physical infrastructure. The cloud modernization source also highlights that public cloud delivers more value when applications are designed to take advantage of cloud capabilities rather than simply being transferred unchanged.

That means the CIO should distinguish between two questions: can this application run in the cloud, and does this application actually benefit from running there? Those are not the same question, and the second one is usually more important.

Cloud can become expensive when architecture does not change

The same source describes a common pattern in which organizations migrate quickly, resources expand, applications remain monolithic, operational complexity increases and cloud bills start climbing. The response is increasingly FinOps, stronger governance and application modernization.

This matters because cloud pricing can hide an architectural problem. If an application continuously consumes a large amount of compute, storage and network capacity, the variable pricing model that makes cloud attractive for flexible workloads may become less attractive over a long period.

The CIO therefore needs to think about workload economics across several years, not simply compare monthly hosting costs. A workload that is inexpensive to start may become expensive to operate continuously, while a workload that requires expensive physical infrastructure may become cheaper over a long enough period.

Colocation sits between ownership and cloud

Colocation is often treated as a middle ground because the organization does not build the entire facility but retains more control over its own infrastructure than it would in public cloud. It can deploy its own servers, networking and storage while relying on the colocation provider for physical space, power, cooling and facility operations.

This model becomes attractive when an enterprise wants more infrastructure control but does not want to fund and operate an entire data center. For AI infrastructure, that can be particularly useful because an organization may want dedicated GPU systems while avoiding the complexity of securing land, utility power, cooling systems and facility operations.

But colocation introduces its own dependencies. The provider must have sufficient power, the site must support the required rack density, cooling architecture must match the hardware and future capacity must be available when the customer needs it.

Colocation therefore does not eliminate the capacity problem. It transfers part of that problem to another operator and makes provider due diligence more important.

Building infrastructure creates control, but also commitment

Owned infrastructure offers a different set of advantages. The organization can control physical architecture, network design, security boundaries and operational processes, and it can design facilities around specific workloads.

For stable, predictable and highly strategic workloads, that control may be valuable. But the financial commitment can be enormous, especially when the infrastructure includes high-density AI capacity and the electrical and cooling systems required to support it.

The Oracle example illustrates the scale of that challenge. According to CIO, investment bank TD Cowen estimated Oracle's AI infrastructure commitments would require roughly $156 billion in capital expenditure, while the report described financing pressure, rising borrowing costs and data center projects struggling to secure sufficient capital.

This is an extreme hyperscale example, but the underlying lesson applies more broadly. Owning infrastructure means owning more of the capital risk, so a CIO considering owned capacity needs confidence in long-term demand.

Infrastructure financing has become part of technology risk

The Oracle report contains an important observation for CIOs: enterprises should treat a provider's infrastructure expansion as a shared infrastructure risk. The logic is simple, because if a provider cannot finance the capacity, it cannot build the capacity, and customers may not receive the infrastructure they expect.

That expands the CIO's vendor risk model beyond security, availability, service levels and lock-in. For infrastructure-intensive AI workloads, another question becomes relevant: can the provider actually fund and deliver the future capacity the enterprise expects to consume?

This becomes especially important when an organization signs large long-term commitments. Capital markets, bank lending and infrastructure financing can eventually affect technical capacity, even when those topics sit outside traditional IT architecture discussions.

Multi-vendor strategy can reduce infrastructure dependency

Analysts recommend multicloud and multivendor approaches as a way to reduce dependence on any single provider. That does not mean every workload should run simultaneously across several clouds, because doing so can create substantial complexity and cost.

A more practical interpretation is that the enterprise should avoid designing its entire technology strategy around the assumption that one provider will always have the required capacity, pricing and regulatory fit. For some workloads, portability may matter more than active distribution, while other critical workloads may require a second provider or a combination of public cloud and privately controlled infrastructure.

The objective is resilience at the portfolio level. CIOs should be able to change placement when economics, regulation or capacity conditions change.

Sovereignty adds another dimension to workload placement

Cost and performance are not the only factors. Data sovereignty can determine where workloads are legally or strategically allowed to operate, especially in regulated sectors such as defense, aerospace, healthcare and financial services.

Wind River describes sovereign cloud as an environment designed to keep computation, storage, processing and control within defined sovereign boundaries. The source also notes that sovereign requirements increasingly extend beyond where data is stored and may include where the data is processed and who has legal or operational control over the environment.

That makes infrastructure placement much more complicated. A data center may physically sit inside a country while the underlying provider is still subject to another jurisdiction, so location alone may not satisfy the organization's sovereignty requirements.

Cloud location and cloud control are different questions

This distinction is one of the most important ideas in sovereign infrastructure planning. CIOs need to ask where the hardware is located, who operates it, who controls the software stack, who can access the data and which legal jurisdiction can compel access.

Wind River specifically references the U.S. CLOUD Act as an example of why organizations are paying attention to the relationship between provider jurisdiction and data location. For some organizations, saying that the data is hosted locally may therefore be insufficient.

A more complete sovereignty review may need to examine ownership, operations, control and legal exposure. This can make sovereign cloud, local providers or owned infrastructure more attractive for particular workloads.

AI is making infrastructure placement harder

AI adds pressure to almost every decision dimension because workloads can require very large amounts of compute, significant power and cooling, specialized GPUs and predictable long-term capacity. Some AI workloads involve sensitive enterprise data, while government or regulated workloads may require sovereign control.

Training workloads can also have different requirements from real-time inference. The result is that "put AI in the cloud" is too broad to be a strategy, because different AI workloads may belong in entirely different environments.

A CIO might place experimentation in public cloud, sensitive enterprise inference on private infrastructure, large training workloads on dedicated cloud or specialized providers, sovereign workloads on local infrastructure, and stable, heavily utilized GPU clusters in owned or colocated systems. The correct answer depends on workload characteristics rather than a single enterprise-wide slogan.

The CIO should evaluate four dimensions first

A useful workload placement framework can begin with four questions. These four questions can eliminate many poor choices before detailed cost modelling even begins, and they create a common language for comparing very different infrastructure models.

Then evaluate economics

Once workload requirements are clear, the CIO can compare cost models. Public cloud often shifts spending toward operating expense, owned infrastructure requires more capital and colocation sits somewhere between the two, but simple comparisons can be misleading.

A better model should consider compute, storage, network transfer, licensing, power, cooling, facility costs, staffing, hardware refresh, financing, capacity reservation, migration cost, exit cost and downtime risk. The cheapest option in year one may not be the cheapest in year five, while a more expensive option may offer better control or lower risk.

Infrastructure strategy therefore becomes a portfolio decision rather than a single cost optimization exercise. The CIO needs to understand how economics change over the full life of the workload.

Avoid making the decision application by application without governance

If every engineering team chooses its preferred environment independently, the enterprise can end up with fragmented infrastructure. One team chooses AWS, another chooses Azure, another builds Kubernetes internally, another signs a colocation contract and another begins using a specialized AI provider.

Individually, each decision may make sense, but collectively the environment can become difficult to govern. The cloud modernization source describes this problem and argues for more structured multicloud strategies with clearer workload placement decisions and stronger governance.

The CIO therefore needs principles that guide individual teams. Flexibility without governance creates complexity, and complexity eventually becomes an operating cost.

The infrastructure portfolio needs a common operating view

Hybrid infrastructure creates another challenge because workloads span cloud, colocation and owned facilities. Different environments expose different metrics, different teams own different layers and different vendors use different management models.

That makes cross-environment visibility increasingly important. A platform such as Sensaka can help provide a shared operational view across infrastructure domains, but the CIO still needs clear policies about workload placement, ownership and risk because technology can improve visibility without replacing the decision framework.

A practical workload placement scorecard

CIOs can evaluate major workloads against a common set of criteria. Each criterion asks a different question about the consequences of placing the workload in a particular environment, and together they produce a much stronger discussion than simply asking whether a workload should go "to the cloud."

The scorecard also makes tradeoffs explicit and easier to revisit when conditions change.

There is no single correct infrastructure model

The three sources point toward the same conclusion from different directions. The cloud modernization discussion shows that migration alone does not solve architecture, cost or complexity problems, the sovereign cloud discussion shows that legal jurisdiction and operational control increasingly affect infrastructure placement, and the Oracle example shows that even very large providers face financing and capacity constraints.

Together, they suggest that infrastructure strategy is becoming more selective. The CIO is no longer deciding whether the organization is cloud-first or data-center-first, but deciding what each workload actually needs.

The question becomes portfolio design

A mature infrastructure strategy might use public cloud for rapidly changing applications and experimentation, colocation for dedicated infrastructure without full facility ownership, owned infrastructure for strategic and predictable high-control workloads, and sovereign cloud or local infrastructure where jurisdiction is critical. Multiple providers may also be appropriate where concentration risk becomes unacceptable.

That mixture will differ by organization. The important point is that each choice should have a reason and that the organization should be able to explain why the workload belongs where it runs.

Three questions the CIO should answer before choosing

Before deciding where a workload should run, the CIO should be able to answer what the workload requires, what risk the organization is accepting by placing it there, and how difficult it will be to change the decision later. Performance, capacity, sovereignty, security, availability, vendor dependency, financing risk and operational complexity all belong in those answers.

These questions are more useful than declaring allegiance to any particular infrastructure model, because they force the organization to think about both entry and exit. Infrastructure decisions become dangerous when they are easy to enter and expensive to reverse.

The future is likely hybrid because the requirements are hybrid

Cloud remains essential, owned infrastructure remains relevant, colocation solves a real problem between the two, and sovereign environments are becoming more important. AI makes all of these choices more consequential because capacity, data sensitivity and power requirements can vary dramatically between workloads.

The CIO's task is therefore moving from infrastructure selection to infrastructure portfolio management. The goal is to place each workload where the business receives the best combination of flexibility, control, capacity, sovereignty and cost, rather than trying to force every workload into one operating model.

Sources: GitHub Issue Discussion, Wind River, Sovereign Cloud Stats Every CIO Needs, CIO.com, Oracle May Slash Up to 30,000 Jobs to Fund AI Data Center Expansion.

Related resources: explore Data Center Observability, review our approach to IT Operations, and see how teams manage distributed sites with Remote Infrastructure Management.