Typed Forecast Results#

What you’ll build

A workflow using ForecastResult and ForecastCollection objects - typed containers that dispatch automatically to the correct evaluation and plot routines, replacing ad-hoc DataFrame wrangling with a structured, type-safe interface.

Prerequisites

01 - Getting Started
06 - Neural Networks
07 - Quantile Regression
08 - Parametric Distributions

Learning objectives

By the end of this notebook you will be able to:

Understand why typed results matter - no ad-hoc branching on forecast kind in downstream code
Create a ForecastResult from raw arrays and inspect its kind, loc, and timestamps fields
Use ForecastCollection to hold multi-model output and iterate over results by name
Call .evaluate() without knowing the forecast kind - dispatch happens automatically
Use .plot() dispatch to visualise point, quantile, and parametric results uniformly

Setup#

Key concept - typed forecast results

Every call to forecaster.forecast() returns a ForecastCollection whose values are ForecastResult objects. Each result carries a kind field - one of "point", "parametric", "quantile", "interval", or "samples" - that encodes what the result contains.

Because kind is set at construction time, downstream code never has to ask “did this model return a mean or a quantile?”. Methods like .evaluate() and .plot() inspect kind internally and dispatch to the correct implementation. You write one loop; Twiga handles the branching.
# Without typed results - manual branching
if model_type == "quantile":
    lower, upper = preds["q0.1"], preds["q0.9"]
elif model_type == "normal":
    lower = preds["mean"] - 1.96 * preds["std"]
    ...

# With ForecastResult - no branching needed
for name, result in collection.items():
    result.evaluate()   # dispatches automatically
    result.plot()       # dispatches automatically

import logging

from great_tables import GT
from lets_plot import LetsPlot
import numpy as np
import pandas as pd
from sklearn.preprocessing import RobustScaler, StandardScaler

LetsPlot.setup_html()

from twiga import DataPipelineConfig, ForecasterConfig, TwigaForecaster
from twiga.core.utils.logger import get_logger

log = get_logger(__name__, level=logging.INFO)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 14
     10 
     11 from twiga import DataPipelineConfig, ForecasterConfig, TwigaForecaster
     12 from twiga.core.utils.logger import get_logger
     13 
---> 14 log = get_logger(__name__, level=logging.INFO)

TypeError: get_logger() got an unexpected keyword argument 'level'

# ── Sample data ──────────────────────────────────────────────────────────────
np.random.seed(42)
n = 24 * 90  # 90 days of hourly data

timestamps = pd.date_range("2023-01-01", periods=n, freq="h")
target = 500 + 200 * np.sin(2 * np.pi * np.arange(n) / 24) + 30 * np.random.randn(n)
temp = 15 + 10 * np.sin(2 * np.pi * np.arange(n) / (24 * 365)) + np.random.randn(n)

data = pd.DataFrame({"timestamp": timestamps, "load_kw": target, "temperature": temp})

split = int(0.7 * n)
val_split = int(0.85 * n)
train_df = data.iloc[:split]
val_df = data.iloc[split:val_split]
test_df = data.iloc[val_split:]

log.info("train=%d  val=%d  test=%d rows", len(train_df), len(val_df), len(test_df))

1. Auto-populated NN dimensions#

Previously, creating an NN config required calling MLPFConfig.from_data_config(data_config) so that
forecast_horizon, lookback_window_size, and feature counts were populated.

Now, all dimension fields default to 0 (a sentinel) and TwigaForecaster fills them automatically.
You only need to set training hyperparameters:

# Old API
mlpf_config = MLPFConfig.from_data_config(data_config)
mlpf_config.max_epochs = 5

# New API
mlpf_config = MLPFConfig(max_epochs=5)   # dims filled by TwigaForecaster

from_data_config() still exists for standalone inspection or building configs outside a forecaster.

from twiga.models.nn import MLPFConfig

data_config = DataPipelineConfig(
    target_feature="load_kw",
    historical_features=["temperature"],
    calendar_features=["hour", "day_of_week"],
    period="1h",
    forecast_horizon=24,
    lookback_window_size=72,
    input_scaler=StandardScaler(),
    target_scaler=RobustScaler(),
)

train_config = ForecasterConfig(split_freq="days", train_size=45, test_size=7)

# Dims are 0 here — they will be filled when passed to TwigaForecaster
mlpf_config = MLPFConfig(max_epochs=3, rich_progress_bar=False)

log.info("forecast_horizon before TwigaForecaster: %d", mlpf_config.forecast_horizon)  # 0

forecaster = TwigaForecaster(
    data_params=data_config,
    model_params=[mlpf_config],
    train_params=train_config,
)

# The config stored in the forecaster has dims populated
stored_cfg = forecaster.models[0].model_config
log.info("forecast_horizon after TwigaForecaster : %d", stored_cfg.forecast_horizon)  # 24
log.info("lookback_window_size                   : %d", stored_cfg.lookback_window_size)  # 72
log.info("num_target_feature                     : %d", stored_cfg.num_target_feature)  # 1

2. The `distribution=` shorthand#

Instead of importing a dedicated class for each probabilistic variant, set distribution= on any base arch config.
TwigaForecaster resolves it to the concrete class automatically - Pydantic validates the value at construction time.

from great_tables import md
import pandas as pd

from twiga.core.plot.gt import twiga_gt

api_comparison = pd.DataFrame(
    {
        "Old (verbose)": [
            "MLPFNormalConfig.from_data_config(...)",
            "MLPGAMLaplaceConfig.from_data_config(...)",
            "MLPGAFConfig.from_data_config(...)  → QR variant",
        ],
        "New (shorthand)": [
            'MLPFConfig(distribution="normal")',
            'MLPGAMConfig(distribution="laplace")',
            'MLPGAFConfig(distribution="qr")',
        ],
    }
)

twiga_gt(
    GT(api_comparison)
    .tab_header(title=md("**distribution= Shorthand**"), subtitle="Old verbose API vs. new shorthand")
    .cols_label(**{c: md(f"**{c}**") for c in api_comparison.columns})
    .tab_source_note("Twiga Forecast"),
    n_rows=len(api_comparison),
)

arch_distributions = pd.DataFrame(
    {
        "Arch": ["MLPFConfig", "MLPGAMConfig", "MLPGAFConfig"],
        "Supported distributions": [
            "normal, laplace, lognormal, gamma, beta, qr, fpqr, crc",
            "normal, laplace, lognormal, gamma, beta, qr, fpqr, crc, studentt",
            "normal, laplace, lognormal, gamma, beta, qr, fpqr, crc, studentt",
        ],
    }
)

twiga_gt(
    GT(arch_distributions)
    .tab_header(title=md("**Valid distribution= Values per Architecture**"), subtitle="Set on any base arch config")
    .cols_label(**{c: md(f"**{c}**") for c in arch_distributions.columns})
    .tab_source_note("Twiga Forecast"),
    n_rows=len(arch_distributions),
)

from twiga.core.config import ConformalConfig
from twiga.models.nn import MLPFConfig, MLPGAMConfig

# Normal distribution on MLPF backbone
normal_config = MLPFConfig(distribution="normal", max_epochs=3, rich_progress_bar=False)

# Laplace on MLPGAM backbone
laplace_config = MLPGAMConfig(distribution="laplace", max_epochs=3, rich_progress_bar=False)

log.info("normal_config  name: %s", normal_config.name)  # mlpf
log.info("laplace_config name: %s", laplace_config.name)  # mlpgam

conformal_config = ConformalConfig(method="aci", score_type="unscaled", alpha=0.1)

forecaster_prob = TwigaForecaster(
    data_params=data_config,
    model_params=[normal_config, laplace_config],
    train_params=train_config,
    conformal_params=conformal_config,
)

# After resolution, the stored configs are the concrete probabilistic classes
for model in forecaster_prob.models:
    log.info("Resolved model class: %s", type(model).__name__)

forecaster_prob.fit(train_df=train_df, val_df=val_df)

3. `forecast()`: typed results via `ForecastCollection`#

evaluate_point_forecast() and predict() return raw DataFrames, which is convenient but loses type information.

forecaster.forecast(test_df) returns a ForecastCollection - a dict-like container of ForecastResult objects, one per model.

collection = forecaster.forecast(test_df)
result = collection["mlpfnormal"]  # ForecastResult

result.loc           # ndarray (n_windows, horizon, n_targets) - point predictions
result.timestamps    # list of timestamp arrays, one per window
result.model_name    # "mlpfnormal"
result.inference_time  # seconds

Key concept - ForecastCollection

A ForecastCollection is an ordered dict-like container: keys are model names, values are ForecastResult objects. It is returned by forecaster.forecast(test_df) - one entry per model registered in the forecaster.

Key operations:

collection["mlpfnormal"] - access a single result by name

for name, result in collection.items() - iterate over all results

collection.to_dataframe() - flatten all results into a single long-format DataFrame

Because ForecastCollection knows the kind of every result, ensemble strategies (mean, median, weighted mean) implemented in TwigaForecaster can operate over results without inspecting which kind each model produced.

from twiga import ForecastCollection, ForecastResult

collection = forecaster_prob.forecast(test_df=test_df)

log.info("Models in collection: %s", list(collection.keys()))

for name, result in collection.items():
    log.info("--- %s ---", name)
    log.info("  loc shape      : %s", result.loc.shape)
    log.info("  n_windows      : %d", len(result.timestamps))
    log.info("  inference_time : %.3f s", result.inference_time)

Inspecting a single result#

result = collection["mlpfnormal"]

# First forecast window
window_0_timestamps = result.timestamps[0]
window_0_forecast = result.loc[0, :, 0]  # (horizon,)

log.info("Window 0 — first 5 timestamps : %s", window_0_timestamps[:5])
log.info("Window 0 — first 5 predictions: %s", window_0_forecast[:5].round(1))

Converting to a flat DataFrame#

df = collection.to_dataframe()
log.info("Collection DataFrame shape: %s", df.shape)
log.info("\n%s", df.head(6).to_string())

4. Putting it all together#

Here is the minimal end-to-end pattern with the new API:

from twiga import DataPipelineConfig, ForecasterConfig, TwigaForecaster
from twiga.models.nn import MLPFConfig, MLPGAMConfig

data_config  = DataPipelineConfig(target_feature="load_kw", ...)
train_config = ForecasterConfig(split_freq="days", train_size=45, test_size=7)

forecaster = TwigaForecaster(
    data_params=data_config,
    model_params=[
        MLPFConfig(distribution="normal", max_epochs=20),   # probabilistic
        MLPGAMConfig(distribution="qr",   max_epochs=20),   # quantile regression
        MLPFConfig(max_epochs=20),                          # point forecast
    ],
    train_params=train_config,
)

forecaster.fit(train_df, val_df)

# Typed results
collection = forecaster.forecast(test_df)
for name, result in collection.items():
    print(name, result.loc.shape)

from great_tables import GT, md
import pandas as pd

from twiga.core.plot.gt import twiga_gt

summary_df = pd.DataFrame(
    {
        "Feature": ["NN dims", "Probabilistic variant", "Quantile variant", "Typed results"],
        "Old API": [
            "MLPFConfig.from_data_config(data_config)",
            "MLPFNormalConfig.from_data_config(...)",
            "MLPFQRConfig.from_data_config(...)",
            "raw pd.DataFrame",
        ],
        "New API": [
            "MLPFConfig() — auto-filled by TwigaForecaster",
            'MLPFConfig(distribution="normal")',
            'MLPFConfig(distribution="qr")',
            "forecaster.forecast(test_df) → ForecastCollection",
        ],
    }
)

twiga_gt(
    GT(summary_df)
    .tab_header(title=md("**API Evolution Summary**"), subtitle="Old verbose API vs. new concise API")
    .cols_label(**{c: md(f"**{c}**") for c in summary_df.columns})
    .tab_source_note("from_data_config() still available for standalone config inspection"),
    n_rows=len(summary_df),
)

Wrapping up#

What you did

Configured two probabilistic NN models using the distribution= shorthand
Used auto-populated dimensions - no from_data_config() call needed
Ran forecaster.forecast(test_df) and received a typed ForecastCollection
Inspected individual ForecastResult objects (loc, timestamps, inference_time)
Converted the collection to a flat DataFrame with .to_dataframe()

Key takeaways

ForecastResult.kind eliminates if-else chains in downstream code - dispatch happens automatically.
The distribution= shorthand keeps model configs concise while Pydantic validates values at construction.
NN input/output dimensions are auto-populated by TwigaForecaster - you only set training hyperparameters.
ForecastCollection is a dict-like container that supports iteration, indexing, and flat export.
collection.to_dataframe() gives a single long-format DataFrame suitable for custom analysis.

What’s next?#

# ruff: noqa: E501, E701, E702
from IPython.display import HTML

_TEAL = "#107591"
_TEAL_MID = "#069fac"
_TEAL_LIGHT = "#e8f5f8"
_TEAL_BEST = "#d0ecf1"
_TEXT_DARK = "#2d3748"
_TEXT_MUTED = "#718096"
_WHITE = "#ffffff"

steps = [
    {
        "num": "05",
        "title": "ML Point Forecasting",
        "desc": "CatBoost · XGBoost · LightGBM — point forecasts",
        "tags": ["catboost", "xgboost"],
        "active": False,
    },
    {
        "num": "08",
        "title": "Quantile Regression",
        "desc": "QR-LightGBM · FPQR — prediction intervals",
        "tags": ["quantile", "intervals"],
        "active": False,
    },
    {
        "num": "12",
        "title": "Ensemble Strategies",
        "desc": "Mean · median · weighted-mean ensembles",
        "tags": ["ensemble", "weighted"],
        "active": False,
    },
    {
        "num": "14",
        "title": "Typed Forecast Results",
        "desc": "ForecastResult · ForecastCollection · typed dispatch",
        "tags": ["typed", "ForecastResult", "collection"],
        "active": True,
    },
    {
        "num": "16",
        "title": "SHAP Feature Attribution",
        "desc": "TreeExplainer · feature ranking · timestep attribution",
        "tags": ["SHAP", "explainability"],
        "active": False,
    },
]
track_name = "Intermediate Track"
footer = 'Next: understand your model\'s decisions with <span style="color:#107591;font-weight:600;">SHAP Feature Attribution</span> (16).'


def _b(t, bg, fg):
    return f'<span style="display:inline-block;background:{bg};color:{fg};font-size:10px;font-weight:600;padding:2px 7px;border-radius:10px;margin:2px 2px 0 0;">{t}</span>'


ch = ""
for i, s in enumerate(steps):
    a = s["active"]
    cb = _TEAL if a else _WHITE
    cbo = _TEAL if a else "#d1ecf1"
    nb = _TEAL_MID if a else _TEAL_LIGHT
    nf = _WHITE if a else _TEAL
    tf = _WHITE if a else _TEXT_DARK
    df = "#cce8ef" if a else _TEXT_MUTED
    bb = "#0d5f75" if a else _TEAL_BEST
    bf = "#b8e4ed" if a else _TEAL
    yh = (
        f'<span style="float:right;background:{_TEAL_MID};color:{_WHITE};font-size:10px;font-weight:700;padding:2px 10px;border-radius:12px;">★ you are here</span>'
        if a
        else ""
    )
    bdg = "".join(_b(t, bb, bf) for t in s["tags"])
    ch += f'<div style="background:{cb};border:2px solid {cbo};border-radius:12px;padding:16px 20px;display:flex;align-items:flex-start;gap:16px;box-shadow:{"0 4px 14px rgba(16,117,145,.25)" if a else "0 1px 4px rgba(0,0,0,.06)"};"><div style="min-width:44px;height:44px;background:{nb};color:{nf};border-radius:50%;display:flex;align-items:center;justify-content:center;font-size:15px;font-weight:800;flex-shrink:0;">{s["num"]}</div><div style="flex:1;"><div style="font-size:15px;font-weight:700;color:{tf};margin-bottom:4px;">{s["title"]}{yh}</div><div style="font-size:12.5px;color:{df};margin-bottom:8px;line-height:1.5;">{s["desc"]}</div><div>{bdg}</div></div></div>'
    if i < len(steps) - 1:
        ch += f'<div style="display:flex;justify-content:center;height:32px;"><svg width="24" height="32" viewBox="0 0 24 32" fill="none"><line x1="12" y1="0" x2="12" y2="24" stroke="{_TEAL_MID}" stroke-width="2" stroke-dasharray="4 3"/><polygon points="6,20 18,20 12,30" fill="{_TEAL_MID}"/></svg></div>'

HTML(
    f'<div style="font-family:Inter,\'Segoe UI\',sans-serif;max-width:640px;margin:8px 0;"><div style="background:linear-gradient(135deg,{_TEAL} 0%,{_TEAL_MID} 100%);border-radius:12px 12px 0 0;padding:14px 20px;display:flex;align-items:center;gap:10px;"><svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="{_WHITE}" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg><span style="color:{_WHITE};font-size:14px;font-weight:700;">Twiga Learning Path — {track_name}</span></div><div style="border:2px solid {_TEAL_LIGHT};border-top:none;border-radius:0 0 12px 12px;padding:20px 20px 16px;background:#f9fdfe;display:flex;flex-direction:column;">{ch}<div style="margin-top:16px;font-size:11.5px;color:{_TEXT_MUTED};text-align:center;border-top:1px solid {_TEAL_LIGHT};padding-top:12px;">{footer}</div></div></div>'
)