我们构建了一个约束感知型 GPU 资源分配器，并在七个基准测试场景中将其与先进先出（FIFO）调度器进行对比评测。在完全相同的硬件上运行完全相同的工作负载时，GPU 利用率最高提升了 33 个百分点，且所有场景中的优先级加权输出均有所提升，最高达 105%。硬件本身未作任何变更；发生变化的仅是资源分配决策的执行顺序。需说明的是：以下所有增益数值均表示相对于同一场景下 FIFO 调度器结果的改进幅度。

We built a constraint-aware GPU allocator and benchmarked it against a FIFO scheduler across seven benchmark scenarios. On identical hardware, running identical workloads, GPU utilization rose by as much as 33 percentage points, and priority-weighted output rose in every one of them, by as much as 105%. Nothing about the hardware changed. What changed was the order in which allocation decisions get made. One note on measurement before the numbers start. Every gain below is expressed as improvement over the FIFO result on the same scenario.

我们构建了一种约束感知型 GPU 分配器，并在七个基准场景中将其与 FIFO 调度器进行了对比评测。在相同硬件上运行相同工作负载的情况下，GPU 利用率最高提升了 33 个百分点，所有场景中的优先级加权输出均有所提升，最高达 105%。硬件本身未作任何变更。发生变化的，仅仅是资源分配决策的执行顺序。 在呈现具体数据前需说明一点：以下所有增益值均以同一场景下 FIFO 调度器的结果为基准计算得出。利用率以百分点（percentage points）表示；价值则以优先级加权输出的百分比增幅表示。 “让 GPU 保持忙碌”并非系统可直接执行的决策。真正需要做出的决策更为具体，也更为困难：在每个时间步，哪块 GPU 执行哪个任务、按何种优先级执行。形式化而言，这是针对每组 GPU–任务–时间步组合所做的一项二元选择，其输出为一张调度网格——横轴为全部 GPU，纵轴覆盖整个调度周期，每个单元格中填入对应任务名称，或为空。 核心难题在于：两种互不兼容的任务形态在同一时间步争夺同一硬件资源。第二种异构性则存在于单一任务类型内部：即使基于相同基础模型，训练任务的持续时间可从数小时延伸至数天，所需 GPU 数量亦可从一块扩展至数十块。 本研究全程以基于 FIFO 的调度器作为对照基准：实时推理服务由固定预留资源保障，其余所有任务均按到达顺序排入队列，完全不考虑优先级。在特定条件下，该策略确属合理。当集群存在富余容量时，分配顺序对利用率不构成影响——无论任务以何种序列排布，所有任务均可容纳，因此 FIFO 与任何更复杂的调度策略所能填充的资源池比例完全一致。 但一旦出现资源争用，这种顺序成本便不再隐形，而开始切实损耗可用容量。此时，其代价体现在两个相互强化的方面，值得分别审视。 首先是顺序问题。在真实争用情形下，哪些任务能够被容纳，不仅取决于现有容量总量，更取决于任务的排布顺序。顺序并非在容量判定完成之后才启用的平局决胜机制；它本身就是一项容量决策。FIFO 在每个任务到达时即刻为其分配资源，既不评估该任务的价值，也不检查调度周期内尚待容纳的其他任务，导致高优先级工作被迫等待先到者，且早期分配所占用的容量可能无法被后续任务有效利用。 其次，二者相互叠加。为满足当日最大实时需求而预留的整块 GPU 资源，在每一小时内均对队列中所有批处理任务不可用；剩余资源则仅按请求到达顺序分发。这相当于航空公司将飞机优先分配给最先致电的包机客户，随后发现已无运力执飞真正盈利的定期航线。而为仅持续数小时的峰值需求全天预留 GPU，则正如前文所述的“停场飞机”——处于待命状态，零收益，且对其他任何用户均不可用。 在五个专为真实争用场景构建的基准测试中，该分配器同步提升了两项指标：利用率由原先的 52–85% 区间提升至 72–88% 区间；优先级加权价值增幅介于 24.6% 至 105.1%，平均提升 52%。所有场景下，两项指标均实现提升，且无需以牺牲任一指标为代价进行折衷解释。 表现最强的单个案例为面向训练密集型负载的 8 块 GPU 配置：利用率从 53.6% 提升至 87.0%，价值增幅达 105%，实现翻倍以上增长。这 33 个百分点的提升，源于回收了长期处于待命状态的预留容量，并将剩余资源按优先级顺序重新分配。（该数值基于单一基准排序得出。） 利用率衡量的是占用率：即已分配的 GPU 时间占可用 GPU 时间的比例。它不包含关于所分配任务价值的任何信息。某一场景彻底分离了这两项指标，其差距方向极易被忽视：一个未对优先级进行定价的目标函数，可能使集群填充率完全相同、完成任务数量完全一致，却仍交付更低的整体价值。前文曾指出，占用率并非衡量集群盈利能力的良好指标；本处数据即为此论断的实证版本。 替代方案并非堆砌更多启发式规则。某些约束仅在全局层面具有意义，任何局部规则均无法表达：例如要求连续 GPU 块、限定整个调度周期内可接受的 GPU 切换总量、或确保正在运行的任务永不被抢占。为满足此类约束，必须将整个问题建模为一个统一整体。 目标函数包含两项：将某块 GPU 分配给类批处理任务，可获得等于其优先级乘以时间衰减权重的奖励；未能满足实时需求，则产生与短缺量成正比的惩罚。这也正是实时需求弹性处理得以安全实施的原因：分配器可在低谷期将 GPU 分配给批处理任务，因为未来实时需求未被充分满足所招致的惩罚，远高于该批处理任务当前所能产生的全部收益——起保障作用的是动态惩罚机制，而非静态预留。 时间权重沿调度周期递减，这一设计仅在在线系统中具备合理性：下一轮调度启动时，新任务必将抵达。当前使用的容量，其价值高于未来承诺的容量。 该启发式方法并非通用贪心分配器。其规则即为形式化模型的结构性约束，这意味着其所生成的每张调度网格，从构造上即保证为合法分配。它通常并不“有效”，而是“设计即有效”。 正是这种设计——将调度视野覆盖整个周期，而非逐个响应到达任务——带来了利用率的提升。分配器在安置任一任务前，即可通览全部排队任务；它能将空闲资源池维持为剩余任务实际可占据的形态：当某批处理任务需要指定大小的连续 GPU 块时，轮到其执行时该空间依然保留。优先级决定谁对该空间拥有优先索取权。FIFO 则不具备上述任一能力：它将容量率先承诺给最先提出请求的任务；而稍后到达、但需要特定形态资源的任务，可能因前期分配而彻底失去容纳空间。

We built a constraint-aware GPU allocator and benchmarked it against a FIFO scheduler across seven benchmark scenarios. On identical hardware, running identical workloads, GPU utilization rose by as much as 33 percentage points, and priority-weighted output rose in every one of them, by as much as 105%. Nothing about the hardware changed. What changed was the order in which allocation decisions get made. One note on measurement before the numbers start. Every gain below is expressed as improvement over the FIFO result on the same scenario. Utilization is reported in percentage points; value is reported as a percentage increase in priority-weighted output. "Keep the GPUs busy" is not a decision a system can execute. The decision is narrower and much harder: which GPU runs which job, in which timestep, at what priority. Formally it is one binary choice per combination of GPU, job and timestep, and the output is a grid — every GPU, across the whole scheduling horizon, with a job name in each cell or nothing at all. Two incompatible shapes competing for the same hardware in the same timestep is the core problem. A second heterogeneity sits inside a single type: for the same base model, training jobs range from a few hours to several days, and from one GPU to dozens. The comparison point throughout is a FIFO-based scheduler: real-time inference served from a fixed reservation, and every other job placed in arrival order, without regard for priority. Under the right conditions, that is a reasonable policy. When the cluster has slack, allocation order costs nothing in utilization, everything fits regardless of sequence, so FIFO and anything more sophisticated fill the same fraction of the pool. Contention is where that ordering cost stops being invisible and starts costing capacity too. It then becomes expensive in two separate ways, and they are worth taking one at a time. The ordering. Under real contention, which jobs fit at all depends on the order you place them, not just on how much capacity exists. Order is not a tiebreaker applied after the capacity question is settled. Order is a capacity decision. FIFO places each job as it arrives, without weighing what that job is worth and without checking what else still has to fit inside the horizon, so high-priority work waits behind whatever asked first and capacity gets committed in placements that later jobs cannot use. The two compound. The block held for the day's maximum real-time demand is off the table for every batch job in the queue, in every hour, and whatever remains is handed out in the order the requests happened to arrive. It is the GPU equivalent of an airline assigning aircraft to whichever charter called first, then finding nothing left to fly the route that actually pays. And GPUs reserved all day for a peak lasting a couple of hours are the grounded aircraft from the previous piece in the most literal sense: on standby, earning nothing, unavailable to anyone else. Across five benchmark scenarios built for genuine contention, the allocator improved both axes at once. Utilization moved from a 52–85% band to a 72–88% band. Priority-weighted value rose between 24.6% and 105.1%, averaging 52%. Every scenario, both metrics, no tradeoff to explain away. The strongest single case was a training-heavy workload on 8 GPUs: utilization went from 53.6% to 87.0%, and value more than doubled, up 105%. Thirty-three points of a fixed, already-depreciating asset, recovered by reclaiming reserved standby capacity and placing the rest in priority order. (This figure reflect a single baseline ordering.) Utilization measures occupancy: what fraction of available GPU-time is allocated to something. It carries no information about what that something is worth. One scenario pulls the two apart completely, and the gap runs in a direction that is easy to miss. An objective that does not price priority can fill the cluster to exactly the same level, finish exactly as many jobs, and still deliver less. The previous piece argued that occupancy is a poor read on whether a cluster is earning; this is the measured version of that claim. The alternative is not a longer list of heuristic rules. Some constraints only mean anything globally, and no local rule can express them: contiguous blocks, a budget for how much GPU churn is acceptable across the entire horizon, a guarantee that running work is never preempted. To honor those, the problem has to be written down as one thing. The objective function has two terms. Allocating a GPU to a batch-like job earns a reward equal to its priority multiplied by a time-decay weight. Failing to meet real-time demand incurs a penalty proportional to the size of the shortfall. It is also what makes the elastic treatment of real-time demand safe. The allocator can hand a GPU to batch work during a trough because underserving real-time demand later is priced so far above whatever that batch work earns — the penalty, not a static reservation, is what protects availability. The time weight decays across the horizon for a reason that only makes sense in an online system: by the next scheduling run, new jobs will have arrived. Capacity used now is worth more than capacity promised later. That heuristic is not a generic greedy allocator. Its rules are the formal model's structural constraints, which means every grid it produces is a legal allocation by construction. Not usually valid. Valid by design. That design, applied across the whole horizon rather than one arrival at a time, is what produces the utilization gain. The allocator sees every queued job before it places any of them, it can hold the free pool in shapes the remaining work can actually occupy, a batch job needing a contiguous block of a given size still has room when its turn comes. Priority decides who gets first claim on that room. FIFO has neither view: it commits capacity to whichever job asked first, and a job that arrives later and needs a specific shape may find nothing left that fits, so it goes unscheduled and the GPU-hours it would have consumed go unclaimed. It runs in 1 to 2 milliseconds on the five contended scenarios, and 15 milliseconds at 64 GPUs and 30 jobs — fast enough to run on every incoming request. The system exposes two modes. Fast mode runs the allocator alone and returns its grid; this is the hot path. Full mode uses that grid as a starting point for the formal model, which attempts to improve on it — suited to periodic review rather than per-request decisions. Utilization improved in every scenario but one, where it tied exactly. Value improved in all seven. The scale test matters because it holds at size: 64 GPUs, 30 jobs, 15 milliseconds, 15.9% more value. The uniform-priority test matters because it addresses the obvious skeptical reading. Override every job to identical priority, so that no priority signal distinguishes any of them, and the allocator still moves utilization from 76.8% to 87.5% and value up 23.1%. The gain is not purely an artifact of ordering by priority. Planning placements across the horizon contributes on its own. Everything above assumes the scheduler knows how many GPU-hours each job needs and how much real-time traffic is coming. Both are predictions, not inputs, and a scheduler is only as good as they are. A single generic estimator does not work, because the four workload types have qualitatively different cost drivers. This is where the specialization argument from the previous piece reconnects: the same logic that makes a task-specific model outperform a generalist applies to the estimators feeding the scheduler. This closes back to the ordering argument. Better demand estimates are what make priority-aware placement possible in the first place, you cannot sequence jobs well without knowing what they will consume. The failure mode to avoid has a name worth borrowing: the end-of-world effect