Subscribe to the blog
Predicting what comes next matters more than ever. Retailers are trying to forecast demand across thousands of products without creating stockouts or wasting money on excess inventory. Financial teams need to anticipate risk as markets shift. Manufacturers want to catch equipment failures before they cause expensive downtime.
Traditional time series methods have done solid work for years. But they're stretched thin now. The data we're working with today is messier, bigger, and more complex than what these classical tools were built for. That's where newer approaches come in. In the last few years, foundation models, transformer architectures, and continuous-time modeling have fundamentally changed what's possible in time series forecasting.
These aren't just small accuracy improvements. They change how we handle temporal patterns, deal with irregular data, and make predictions across different domains. If you're working as a data scientist, ML engineer, or analytics lead and want to modernize your forecasting, this guide breaks down what you actually need to know.
What Classical Methods Do (And Where They Break)
For decades, we've relied on ARIMA, exponential smoothing, and seasonal decomposition. These methods work by modeling time dependencies as linear combinations of past observations, trends, and seasonal components. They're clean, interpretable, and fast to run.
But they have hard limits. ARIMA assumes stationarity, meaning your data's statistical properties stay constant over time. If you've got trends or seasonality, you need to manually transform everything through differencing. Do it wrong and you strip away patterns you actually need.
Classical models also assume linear relationships. That breaks down fast when you hit market shocks, promotional spikes, or feedback loops. They struggle with multivariate signals, irregular sampling, and long-range dependencies spanning hundreds of time steps. In practice, you end up doing tons of feature engineering, manual tuning, and domain-specific workarounds. Modern approaches sidestep a lot of this.
The Breakthrough: Foundation Models and Transformers
Here's the core idea. Instead of hand-crafting temporal representations, we train large neural architectures to learn them directly from data. The breakthrough uses transformer models, originally built for language tasks, adapted for time series. These models treat sequences of time points like tokens and use self-attention to capture dependencies across any time lag without needing stationarity or linearity assumptions.
Recent research from ICML 2025 shows that simpler transformer designs often beat complex ones. The real insight? The most useful forecasting information comes from tracking patterns within individual variables over time, not trying to model every possible cross-variable interaction in high dimensions.
Techniques like patch-wise tokenization help a lot. You treat a short window of consecutive time points as one token. This improves both speed and the model's ability to catch localized patterns.
Multi-resolution approaches go further by using multiple transformer branches at different time scales simultaneously, learning both short-term fluctuations and long-term seasonal trends in one shot.
Then there's the rise of foundation models for time series.
Google's TimesFM is a good example. It's a decoder-only transformer pretrained on massive amounts of diverse time series data, designed for zero-shot forecasting. That means it generates accurate predictions on brand new datasets without any fine-tuning specific to your task.
On public benchmarks, TimesFM outperforms traditional models like ARIMA and exponential smoothing, and it rivals deep learning baselines that were explicitly trained on target data. It does this by learning generalizable temporal patterns during pretraining, then adapting to new domains through in-context learning where you just provide a handful of example time points at inference.
There's also parallel innovation with neural ordinary differential equations. Instead of treating a neural network as discrete layers, Neural ODEs model the hidden state as a continuous dynamical system. This lets the model evaluate at any arbitrary time point, making it naturally suited for irregular time series where observations don't arrive at uniform intervals.
Here's a simplified pipeline for transformer-based forecasting:
Step 1: Preprocessing
Normalize your time series. Segment sequences into overlapping patches (say, length 16). Add positional encoding so the model knows the order.
Step 2: Model Setup
Input your sequence of patches. Run through 2-6 layers of multi-head self-attention with feed-forward networks and residual connections. Output your forecast horizon.
Step 3: Training
Use Mean Squared Error or Mean Absolute Error as your loss. Train with AdamW optimizer and learning rate scheduling. Add dropout and weight decay for regularization.
Step 4: Evaluation
Check MAE, MAPE, and scaled MAE (normalized by a naive baseline). Use rolling-window validation on hold-out data to simulate production.
Step 5: Deployment
Serve via REST API or batch inference. Monitor prediction error and data drift in production.
When does this win? It's great when you have large volumes of historical data, complex nonlinear patterns, multivariate dependencies, or irregular sampling. It's also perfect for transfer learning, where you pretrain on one domain and fine-tune on another. The tradeoff is compute. You'll need GPUs, larger datasets, and you lose some of the immediate interpretability you get from ARIMA coefficients. That said, attention weights do show which past time steps matter most.
Examples
Retail Demand:
A global retailer had chronic inventory problems because their forecasts kept missing seasonal spikes and promotions. They switched to advanced transformer models
trained on historical sales, promotional calendars, weather, and local events. The new model captured trends and seasonality far better than classical approaches. Result: improved accuracy across thousands of products and stores, optimized ordering, lower inventory costs, and fewer stockouts. The lesson is that modern architectures generalize across product hierarchies in ways that univariate ARIMA can't.
Financial Risk:
During the 2008 crisis, classical ARIMA models showed 30% higher prediction errors compared to normal conditions. Recent transformer deployments in finance handle volatile periods better by learning nonlinear interactions between asset classes and macro indicators. They incorporate multiple features (price, volume, sentiment) and adapt to regime changes more smoothly.
Predictive Maintenance:
Industrial sensor data is tough. Measurements arrive irregularly, sensors fail, and noise is high. Neural ODE models work well here because they handle sparse, irregular data without requiring imputation. One manufacturing team deployed a continuous-time model for equipment monitoring and reduced unplanned downtime by catching vibration and temperature anomalies earlier.
How to Actually Implement This
Start with your data pipeline. Modern models need data, so invest in infrastructure that captures high-frequency signals, metadata, and relevant covariates. Feature engineering still matters. Thoughtful normalization, handling missing values, and encoding calendar effects (holidays, weekdays) improve convergence and accuracy.
Build your training with rolling-window cross-validation to simulate production. Hold out recent periods, not random samples, to test forward-in-time generalization. Pick metrics that align with business goals. MAE and MAPE are common, but scaled MAE (normalized by a naive baseline) lets you compare across datasets with different scales.
In production, monitor for model performance and data drift. Time series features naturally drift, and ground truth for validation can take weeks or months. You're exposed if the model degrades silently. Implement robust monitoring and update procedures.
For teams serious about upskilling, formalized training accelerates adoption. The need for AI skills is growing year over year. Companies like Salesforce and Google are hiring heavily but still face talent shortages. ATC's Generative AI Masterclass is a hybrid, hands-on program (10 sessions, 20 hours total) covering no-code generative tools, AI for voice and vision, working with multiple agents, and culminating in a capstone where participants deploy an operational AI agent. Currently, 12 of 25 spots remain. Graduates receive an AI Generalist Certification and transition from passive consumers to confident creators of AI workflows with the fundamentals to think at scale.
Compute matters. Training transformers on millions of time steps needs GPUs and careful memory management. Use gradient checkpointing, mixed-precision training (FP16), and tune batch sizes. For inference, consider distillation or quantization if latency is critical. If you're using foundation models like TimesFM, check whether zero-shot or few-shot learning meets your accuracy bar before investing in full fine-tuning.
Risks and Pitfalls
Overfitting is still a risk, especially with small training sets. Data snooping, where test information leaks into training, creates overly optimistic benchmarks. Be strict about train-test splits and don't tune hyperparameters on final evaluation data.
Deep learning models can be brittle. Performance may tank if production data shifts from training distribution. This hits hard in finance or healthcare, where biased predictions have serious consequences. Run bias audits during retraining, document changes clearly,and engage stakeholders in validation.
If forecasts drive automated decisions (inventory orders, trading signals, clinical alerts), a bad model propagates errors at scale. Add guardrails like confidence thresholds, anomaly detection on predictions, and human review for high-stakes calls.
Classic vs Breakthrough
| Attribute | Classical (ARIMA, ETS) | Breakthrough (Transformers, Foundation Models) |
| Data needs | Low (hundreds of points) | High (thousands to millions) |
| Interpretability | High (coefficients, diagnostics) | Moderate (attention weights) |
| Latency | Fast (milliseconds) | Moderate (model-dependent) |
| Scalability | Limited (manual tuning per series) | High (transfer learning, zero-shot) |
| Accuracy | Good for linear, stationary data | Better for complex, nonlinear, multivariate |
| Use cases | Simple univariate, stable processes | Multivariate, irregular sampling, cross-domain |
Wrapping Up
Time series analysis is shifting. Foundation models, transformers, and continuous-time architectures are delivering real improvements in accuracy and generalization. These aren't just upgrades. They fundamentally change how we represent temporal patterns and handle messy real-world data.
For practitioners, the path forward is investment in data infrastructure, experimentation with modern architectures, and rigorous evaluation and monitoring. Don't throw out classical methods entirely. They still work for simple, low-latency forecasts. But for complex, high-stakes applications, the breakthrough approaches covered here offer something meaningfully better.If you're ready to build hands-on skills and deploy operational AI workflows, structured programs like the ATC Generative AI Masterclass (currently 12 of 25 spots remaining) combine fundamentals, practical tooling, and guidance to accelerate your journey from consumer to confident creator of production time series models.
