What Is an AI Factory?

AI data centers run five broad kinds of work, often side by side in the same halls:

• Training: building a model from scratch, often for weeks on hundreds or thousands of accelerators that exchange data over a high-speed fabric.
• Fine-tuning: adapting an existing model to a company's own data, language or task, on a much smaller share of the hardware.
• Inference: serving models to applications such as chat and search assistants, code assistants, document processing and agents. Output is counted in tokens, and the load runs around the clock.
• Retrieval-augmented generation: models that look up a knowledge base before answering, so answers can be traced to a source.
• Other accelerated work: computer vision, recommendation systems, simulation and scientific computing.

Training comes in large runs that start and finish. Inference is continuous and follows user traffic. A facility that does both has to share its accelerators between them, which is why pooling and scheduling sit at the center of AI factory operations.

A general-purpose data center runs CPU servers for business systems: databases, ERP, web services. Its operators measure availability and capacity. An AI factory runs dense accelerator servers. Racks draw far more power, many need liquid cooling, and servers are linked by a scale-out fabric so accelerators can exchange data during training.

That is why operations has two meanings here. Infrastructure operations, or O&M, keeps the facility and hardware running. Business operations runs compute as a service: who gets which accelerators, how usage is metered, and whether the facility earns back what it costs. A platform for an AI factory has to serve both groups from the same data.

• Facility: power distribution, UPS, cooling units, liquid cooling loops, and temperature, humidity and leak sensors.
• Hardware: accelerator servers, CPU servers, switches and storage, read in-band through the operating system and out-of-band through each server's BMC.
• Compute: accelerators pooled across servers and vendors, offered as a few standard specs and handed to jobs through queues and quotas.
• Model and token services: models deployed as inference services behind a gateway.
• Business: tenants, projects, quotas, GPU-hours, cost per token, and the reports finance and management read.

A configuration management database (CMDB) links the layers: which job runs on which accelerator, in which server, rack and power feed, for which tenant. Without that link a fault in a cooling loop and a failed training job look like two separate problems. See CMDB integration.

Provisioning. New servers are initialized through the BMC, installed over the network with PXE, given drivers and a container runtime, and stress-tested at a burn-in gate before they join the GPU pool.

Reliability. Long training runs are expensive to restart. Operators watch for early degradation such as rising ECC memory errors or heat, take suspect nodes out of scheduling, and resume jobs from the last checkpoint. Alert correlation turns dozens of alerts from one fault into one event.

Accounting. GPU-hours and tokens are metered per project, tenant and model, with idle capacity and cost per token alongside. One utilization percentage hides four different counts, so the GPU utilization guide separates them. Management usually reads these figures on an operations cockpit.