What Is MFU (Model FLOPs Utilization)?
MFU, or model FLOPs utilization, is the share of a system's peak compute that a training job turns into the model's own math: tokens processed per second, times the FLOPs each token needs, divided by the peak FLOPs of the hardware. It measures how efficiently a job uses its GPUs, which a busy percentage cannot.
Where MFU comes from
Google researchers proposed MFU in the 2022 PaLM paper as an efficiency measure that does not depend on implementation. The older measure, hardware FLOPs utilization (HFU), counts every operation the hardware runs, including activations recomputed to save memory, so two teams training the same model could report different figures for the same work. MFU counts only what the model needs for its forward and backward passes. The numerator is the observed throughput, and the denominator depends only on the architecture and the hardware's published peak.
How to calculate MFU
MFU = (tokens per second × FLOPs per token) ÷ (GPUs × peak FLOPs per GPU)
For dense transformer training, each token needs about six FLOPs per parameter: two for the forward pass and four for the backward pass, plus an attention term that grows with sequence length. Use the GPU's dense peak at the training precision, such as BF16, not the higher figure some datasheets quote with sparsity.
A hypothetical example: a 7-billion-parameter model needs about 42 GFLOPs per token. If 8 GPUs with a dense peak of 1,000 TFLOPS each process 50,000 tokens per second together, they achieve 2,100 TFLOPS out of 8,000 possible, an MFU of about 26%.
What MFU looks like in published runs
The PaLM paper reported 46.2% MFU for PaLM 540B on 6,144 TPU v4 chips, with an HFU of 57.8%, and calculated 21.3% for GPT-3 on V100 GPUs by the same method. Meta's Llama 3 paper reported 38% to 43% BF16 MFU for Llama 3 405B pre-training on 8,192 to 16,384 GPUs. These are heavily tuned runs by large teams; smaller jobs and less-tuned code should not expect the same. MFU is mainly a training measure; inference efficiency is usually judged by throughput and latency instead.
MFU vs GPU utilization
GPU utilization counts the time a kernel was running. MFU counts the math that was done. A job can show 100% utilization and an MFU of 30%: kernels run all the time, but many are memory-bound, small or waiting on data inside the kernel. Allocation is a third measure again: whether the GPU was assigned to the job at all. An operator needs all three to understand what a training cluster delivers.
What lowers MFU, and what raises it
- communication between GPUs: gradient all-reduce, pipeline bubbles, waits on slow nodes
- data loading that cannot keep up with the GPUs
- memory-bound operations and small matrix sizes that leave compute units waiting
- small micro-batches per GPU
- recomputed activations, which cost time but do not count as model FLOPs
- thermal or power throttling on the accelerators
Teams raise MFU by tuning the mix of tensor, pipeline and data parallelism, using larger micro-batches, fused kernels and memory-efficient attention, overlapping communication with computation, and feeding the GPUs from a fast data pipeline. Some of the causes sit outside the code: a congested network link, slow storage or a hot accelerator shows up as lower MFU too.
Where infrastructure monitoring fits
MFU is calculated from the training job, because only the framework knows the model's FLOPs per token and the tokens it processed. Infrastructure can remove the causes outside the code. Sensaka for AI Data Centers puts GPU utilization, temperature, power and ECC errors next to network and storage metrics on one timeline, so a job that slows because a link drops packets or storage throughput falls can be traced to that cause. It also meters allocated and idle GPU-hours per project, the cost side of the same efficiency question, and runs on-premises.
Common questions
What does MFU stand for?
Model FLOPs utilization: the model FLOPs a training job achieves per second, divided by the peak FLOPs per second of the hardware it runs on, expressed as a percentage.
What is the difference between MFU and HFU?
Hardware FLOPs utilization (HFU) counts every floating-point operation the hardware executes, including activations recomputed to save memory. MFU counts only the operations the model itself needs for the forward and backward passes, so it does not change with implementation tricks and is always the lower of the two.
What is a good MFU?
Published large training runs report roughly 38% to 46%: Meta reported 38-43% for Llama 3 405B, and the PaLM paper reported 46.2% for PaLM 540B. These were heavily tuned runs at very large scale. There is no universal target; compare a job with its own history and with published runs of similar size.
How do I measure MFU with DeepSpeed?
DeepSpeed's FLOPS profiler reports the FLOPs of a model step and the achieved throughput. Divide the achieved FLOPS per GPU by the GPU's dense peak at the same precision. Check whether the count includes recomputed activations: if it does, the result is closer to HFU than MFU.
Try it on your own hardware
The Sensaka pilot is free for 30 days and runs on-premises, inside your data center, with success criteria agreed before it starts. Integrators and resellers can also ask about the partner programme, which pays a 25% margin in the first year.