时间序列基础模型正在改变预测系统的构建方式。用户无需为每个数据集单独训练和维护模型,而是可直接使用预训练模型进行零样本预测。该模型的权重、架构、推理流程以及复现基准测试结果所需的全部代码均已公开。
Time-series foundation models are changing the way forecasting systems are built. Instead of training and maintaining a separate model for every dataset, users can use a pretrained model and generate forecasts zero-shot. The model weights, architecture, inference pipeline, and code needed to reproduce the benchmark results are all available.
时序基础模型正在改变预测系统的构建方式。用户无需为每个数据集单独训练和维护一个模型,而是可直接使用预训练模型进行零样本(zero-shot)预测。该模型的权重、架构、推理流程以及复现基准测试结果所需的全部代码均已公开提供。本文博客将介绍该模型,深入剖析基准测试结果与模型架构,探讨训练数据来源与许可协议,并提供代码示例以说明如何使用该模型。最后,我们还将重点说明 Granite Time Series 系列模型如何借助 Confluent 产品,在生产环境中应用于流式处理场景。 基础模型最有价值之处在于其能泛化至未在训练中显式接触过的时序数据。因此,我们首先聚焦于零样本性能表现。 图3:GIFT-Eval CRPS 指标(同时考虑零样本与可复现预训练模型)。 图4:GIFT-Eval MASE 指标(同时考虑零样本与可复现预训练模型)。 PatchTST-FM-r2 延续了 PatchTST 系列赖以高效的基于分块(patch-based)的表征方式,但对其内部架构进行了重新设计,以更高效地建模长程与短程依赖关系,并平滑块间预测——这两项改进均显著提升了误差指标表现。其中一项关键变更,是将标准 Transformer 层替换为融合卷积操作与多头自注意力机制的层;此类层被称为 Conformer 层,其最初源于语音处理应用。PatchTST-FM-r1 的模块由多头自注意力机制与前馈网络组合构成;而在 r2 版本中,该模块被替换为一种 Conformer 风格模块,其结构为:两个半步前馈层分别位于多头自注意力机制两侧,并额外引入一个时间卷积层。 图6:使用 ETTh1 数据集真实样本,对比 Transformer(左)与 Conformer(右)版本所捕获的注意力模式。 经上述改进,最终模型参数量约为 3.85 亿,支持长达 8,192 步的上下文窗口,并可在灵活的预测长度下输出 99 个分位点。该模型既提供点预测(point forecasts),也提供分位数输出,从而支持对预测分布及不确定性区间的建模。 对于面向实际应用的基础模型而言,模型质量仅是考量因素之一。开发者日益需要明确了解模型所用训练数据、基准测试数据是否可能泄露至训练过程,以及模型部署所涉潜在影响。对企业级采用者而言,此类透明度的重要性,不亚于排行榜上额外提升的几个百分点。这虽不能替代组织自身开展的模型治理与许可审查流程,但相较完全不透明的预训练语料库,它已为用户提供了远为充分的信息支撑该项审查工作。 如您所见,该流程无需微调(fine-tuning),亦无需针对特定任务拟合模型。整个流水线仅需输入序列的近期历史数据,即可生成未来预测结果,包括用户指定的各分位点。 本次发布隶属于 IBM Granite Time Series 模型系列的更广泛布局,为该整体产品组合新增一款模型。该集成方案通过 Confluent Cloud 上的 Apache Flink,将基础模型推理能力直接嵌入流式处理应用。预测结果与异常检测结果均可直接从实时数据流中生成,无须团队另行搭建并迁移数据至独立的机器学习环境。
Time-series foundation models are changing the way forecasting systems are built. Instead of training and maintaining a separate model for every dataset, users can use a pretrained model and generate forecasts zero-shot. The model weights, architecture, inference pipeline, and code needed to reproduce the benchmark results are all available. In this blog we describe the model, dive deeper into the benchmarking results and the model architecture, discuss the training data and licensing, and provide code examples illustrating how to use the model. Finally, we also highlight how the models from the Granite Time Series family can be used in streaming applications in production setting leveraging Confluent product. A foundation model is most useful when it generalizes to time series it has not been specifically trained on. For that reason, we focus first on zero-shot performance. Figure 3. GIFT-Eval CRPS when both zero-shot and pretrained replicable models are considered. Figure 4. GIFT-Eval MASE when both zero-shot and pretrained replicable models are considered. PatchTST-FM-r2 retains the patch-based representation that made the PatchTST family effective, but the internal architecture is redesigned to capture long- and short-term relationships efficiently and to smoothen inter-patch predictions — both of which substantially improve error measures. One change is the move from standard transformer layers to layers which incorporate convolution along with the multi-head self-attention. These layers are referred to as conformer layers and has its origin in speech processing applications. A PatchTST-FM-r1 block combines multi-head self-attention with a feed-forward network. In r2, we replaced this with a conformer-style block containing two half-step feed-forward layers surrounding multi-head self-attention and a temporal convolution layer. Figure 6. Comparing attention patterns captured in the transformer (left) and conformer versions(right) using real samples from the ETTh1 dataset. With these changes, the resulting model has approximately 385M parameters, supports very long contexts of up to 8,192 steps, and predicts 99 quantiles over flexible forecast lengths. The model provides both point forecasts and quantile outputs for forecasting distributions and uncertainty intervals. For foundation models intended for real applications, model quality is only one consideration. Developers increasingly need to understand what data went into a model, whether benchmark data may have leaked into training, and what the implications are for deploying the model. For enterprise adopters, this kind of transparency can be as important as another few points on a leaderboard. This does not eliminate the need for an organization's own model-governance and licensing review, but it gives users considerably more information with which to perform that review than an opaque pretraining corpus would. As you can see, no fine-tuning and no task-specific model fitting is needed. The pipeline consumes just the recent history of the series and generates the future forecast, including the requested quantiles. This release connects to a broader effort around IBM Granite Time Series models, adding a new model to the broader portfolio. The integration brings foundation-model inference directly into streaming applications through Apache Flink on Confluent Cloud. Forecasts and anomaly-detection results can be generated from live streams rather than requiring teams to set up and move data to a separate ML environment.