Typed Forecast Results#

Intermediate Python Twiga Time

What you’ll build

A workflow using ForecastResult and ForecastCollection objects - typed containers that dispatch automatically to the correct evaluation and plot routines, replacing ad-hoc DataFrame wrangling with a structured, type-safe interface.

Prerequisites

Learning objectives

By the end of this notebook you will be able to:

  1. Understand why typed results matter - no ad-hoc branching on forecast kind in downstream code

  2. Create a ForecastResult from raw arrays and inspect its kind, loc, and timestamps fields

  3. Use ForecastCollection to hold multi-model output and iterate over results by name

  4. Call .evaluate() without knowing the forecast kind - dispatch happens automatically

  5. Use .plot() dispatch to visualise point, quantile, and parametric results uniformly

Setup#

Key concept - typed forecast results

Every call to forecaster.forecast() returns a ForecastCollection whose values are ForecastResult objects. Each result carries a kind field - one of "point", "parametric", "quantile", "interval", or "samples" - that encodes what the result contains.

Because kind is set at construction time, downstream code never has to ask “did this model return a mean or a quantile?”. Methods like .evaluate() and .plot() inspect kind internally and dispatch to the correct implementation. You write one loop; Twiga handles the branching.

# Without typed results - manual branching
if model_type == "quantile":
    lower, upper = preds["q0.1"], preds["q0.9"]
elif model_type == "normal":
    lower = preds["mean"] - 1.96 * preds["std"]
    ...

# With ForecastResult - no branching needed
for name, result in collection.items():
    result.evaluate()   # dispatches automatically
    result.plot()       # dispatches automatically

import logging

from great_tables import GT
from lets_plot import LetsPlot
import numpy as np
import pandas as pd

LetsPlot.setup_html()

from twiga import DataPipelineConfig, ExperimentConfig, TwigaForecaster
from twiga.core.utils.logger import get_logger

log = get_logger("Tutorial")

data = pd.read_parquet("../data/MLVS-PT.parquet")
data = data[["timestamp", "NetLoad(kW)", "Ghi", "Temperature"]]
data["timestamp"] = pd.to_datetime(data["timestamp"])
data = data.drop_duplicates(subset="timestamp").reset_index(drop=True)
# Restrict to 2019-2020 to keep tutorial execution fast
data = data[(data["timestamp"] >= "2019-01-01") & (data["timestamp"] <= "2020-12-31")].reset_index(drop=True)

train_df = data[data["timestamp"] < "2020-01-01"].reset_index(drop=True)
val_df = data[(data["timestamp"] >= "2020-01-01") & (data["timestamp"] < "2020-07-01")].reset_index(drop=True)
test_df = data[data["timestamp"] >= "2020-07-01"].reset_index(drop=True)

1. Auto-populated NN dimensions#

Previously, creating an NN config required calling MLPFConfig.from_data_config(data_config) so that
forecast_horizon, lookback_window_size, and feature counts were populated.

Now, all dimension fields default to 0 (a sentinel) and TwigaForecaster fills them automatically.
You only need to set training hyperparameters:

# Old API
mlpf_config = MLPFConfig.from_data_config(data_config)
mlpf_config.max_epochs = 5

# New API
mlpf_config = MLPFConfig(max_epochs=5)   # dims filled by TwigaForecaster

from_data_config() still exists for standalone inspection or building configs outside a forecaster.

from twiga.models.nn import MLPFConfig

data_config = DataPipelineConfig(
    target_feature="NetLoad(kW)",
    period="30min",
    latitude=32.371666,
    longitude=-16.274998,
    calendar_features=["hour", "day_night"],
    known_future_features=["Ghi"],
    forecast_horizon=48,
    lookback_window_size=96,
    stride=48,
    input_scaler="standard",
    target_scaler="robust",
)

train_config = ExperimentConfig(project_name="tutorial-14")


# Dims are 0 here — they will be filled when passed to TwigaForecaster
mlpf_config = MLPFConfig(max_epochs=3, rich_progress_bar=False)

log.info("forecast_horizon before TwigaForecaster: %d", mlpf_config.forecast_horizon)  # 0

forecaster = TwigaForecaster(
    data_params=data_config,
    model_params=[mlpf_config],
    cv_params=train_config,
)

# The config stored in the forecaster has dims populated
stored_cfg = forecaster.models[0].model_config
log.info("forecast_horizon after TwigaForecaster : %d", stored_cfg.forecast_horizon)  # 24
log.info("lookback_window_size                   : %d", stored_cfg.lookback_window_size)  # 72
log.info("num_target_feature                     : %d", stored_cfg.num_target_feature)  # 1

from twiga.core.config import ConformalConfig
from twiga.models.ml import LIGHTGBMConfig

conformal_config = ConformalConfig(method="residual", alpha=0.1)

forecaster_prob = TwigaForecaster(
    data_params=data_config,
    model_params=[LIGHTGBMConfig()],
    cv_params=train_config,
    conformal_params=conformal_config,
)

# After resolution, the stored configs are the concrete probabilistic classes
for model in forecaster_prob.models:
    log.info("Resolved model class: %s", type(model).__name__)

forecaster_prob.fit(train_df=train_df, val_df=val_df)

3. forecast(): typed results via ForecastCollection#

evaluate_point_forecast() and predict() return raw DataFrames, which is convenient but loses type information.

forecaster.forecast(test_df) returns a ForecastCollection - a dict-like container of ForecastResult objects, one per model.

collection = forecaster.forecast(test_df)
result = collection["mlpfnormal"]  # ForecastResult

result.loc           # ndarray (n_windows, horizon, n_targets) - point predictions
result.timestamps    # list of timestamp arrays, one per window
result.model_name    # "mlpfnormal"
result.inference_time  # seconds

Key concept - ForecastCollection

A ForecastCollection is an ordered dict-like container: keys are model names, values are ForecastResult objects. It is returned by forecaster.forecast(test_df) - one entry per model registered in the forecaster.

Key operations:

  • collection["mlpfnormal"] - access a single result by name

  • for name, result in collection.items() - iterate over all results

  • collection.to_dataframe() - flatten all results into a single long-format DataFrame

Because ForecastCollection knows the kind of every result, ensemble strategies (mean, median, weighted mean) implemented in TwigaForecaster can operate over results without inspecting which kind each model produced.

4. Putting it all together#

Here is the minimal end-to-end pattern with the new API:

from twiga import DataPipelineConfig, ExperimentConfig, TwigaForecaster
from twiga.models.nn import MLPFConfig, MLPGAMConfig

data_config  = DataPipelineConfig(target_feature="load_kw", ...)
train_config = ExperimentConfig(split_freq="days", train_size=45, test_size=7)

forecaster = TwigaForecaster(
    data_params=data_config,
    model_params=[
        MLPFConfig(distribution="normal", max_epochs=20),   # probabilistic
        MLPGAMConfig(distribution="qr",   max_epochs=20),   # quantile regression
        MLPFConfig(max_epochs=20),                          # point forecast
    ],
    cv_params=train_config,
)

forecaster.fit(train_df, val_df)

# Typed results
collection = forecaster.forecast(test_df)
for name, result in collection.items():
    print(name, result.loc.shape)

from great_tables import md
import pandas as pd

from twiga.core.plot.gt import twiga_gt

summary_df = pd.DataFrame(
    {
        "Feature": ["NN dims", "Probabilistic variant", "Quantile variant", "Typed results"],
        "Old API": [
            "MLPFConfig.from_data_config(data_config)",
            "MLPFNormalConfig.from_data_config(...)",
            "MLPFQRConfig.from_data_config(...)",
            "raw pd.DataFrame",
        ],
        "New API": [
            "MLPFConfig() — auto-filled by TwigaForecaster",
            'MLPFConfig(distribution="normal")',
            'MLPFConfig(distribution="qr")',
            "forecaster.forecast(test_df) → ForecastCollection",
        ],
    }
)

twiga_gt(
    GT(summary_df)
    .tab_header(title=md("**API Evolution Summary**"), subtitle="Old verbose API vs. new concise API")
    .cols_label(**{c: md(f"**{c}**") for c in summary_df.columns})
    .tab_source_note("from_data_config() still available for standalone config inspection"),
    n_rows=len(summary_df),
)

Wrapping up#

What you did

  • Configured two probabilistic NN models using the distribution= shorthand

  • Used auto-populated dimensions - no from_data_config() call needed

  • Ran forecaster.forecast(test_df) and received a typed ForecastCollection

  • Inspected individual ForecastResult objects (loc, timestamps, inference_time)

  • Converted the collection to a flat DataFrame with .to_dataframe()

Key takeaways

  1. ForecastResult.kind eliminates if-else chains in downstream code - dispatch happens automatically.

  2. The distribution= shorthand keeps model configs concise while Pydantic validates values at construction.

  3. NN input/output dimensions are auto-populated by TwigaForecaster - you only set training hyperparameters.

  4. ForecastCollection is a dict-like container that supports iteration, indexing, and flat export.

  5. collection.to_dataframe() gives a single long-format DataFrame suitable for custom analysis.

What’s next?#

# ruff: noqa: E501, E701, E702
from IPython.display import HTML

_TEAL = "#107591"
_TEAL_MID = "#069fac"
_TEAL_LIGHT = "#e8f5f8"
_TEAL_BEST = "#d0ecf1"
_TEXT_DARK = "#2d3748"
_TEXT_MUTED = "#718096"
_WHITE = "#ffffff"

steps = [
    {
        "num": "05",
        "title": "ML Point Forecasting",
        "desc": "CatBoost · XGBoost · LightGBM — point forecasts",
        "tags": ["catboost", "xgboost"],
        "active": False,
    },
    {
        "num": "08",
        "title": "Quantile Regression",
        "desc": "QR-LightGBM · FPQR — prediction intervals",
        "tags": ["quantile", "intervals"],
        "active": False,
    },
    {
        "num": "12",
        "title": "Ensemble Strategies",
        "desc": "Mean · median · weighted-mean ensembles",
        "tags": ["ensemble", "weighted"],
        "active": False,
    },
    {
        "num": "14",
        "title": "Typed Forecast Results",
        "desc": "ForecastResult · ForecastCollection · typed dispatch",
        "tags": ["typed", "ForecastResult", "collection"],
        "active": True,
    },
    {
        "num": "16",
        "title": "SHAP Feature Attribution",
        "desc": "TreeExplainer · feature ranking · timestep attribution",
        "tags": ["SHAP", "explainability"],
        "active": False,
    },
]
track_name = "Intermediate Track"
footer = 'Next: understand your model\'s decisions with <span style="color:#107591;font-weight:600;">SHAP Feature Attribution</span> (16).'


def _b(t, bg, fg):
    return f'<span style="display:inline-block;background:{bg};color:{fg};font-size:10px;font-weight:600;padding:2px 7px;border-radius:10px;margin:2px 2px 0 0;">{t}</span>'


ch = ""
for i, s in enumerate(steps):
    a = s["active"]
    cb = _TEAL if a else _WHITE
    cbo = _TEAL if a else "#d1ecf1"
    nb = _TEAL_MID if a else _TEAL_LIGHT
    nf = _WHITE if a else _TEAL
    tf = _WHITE if a else _TEXT_DARK
    df = "#cce8ef" if a else _TEXT_MUTED
    bb = "#0d5f75" if a else _TEAL_BEST
    bf = "#b8e4ed" if a else _TEAL
    yh = (
        f'<span style="float:right;background:{_TEAL_MID};color:{_WHITE};font-size:10px;font-weight:700;padding:2px 10px;border-radius:12px;">★ you are here</span>'
        if a
        else ""
    )
    bdg = "".join(_b(t, bb, bf) for t in s["tags"])
    ch += f'<div style="background:{cb};border:2px solid {cbo};border-radius:12px;padding:16px 20px;display:flex;align-items:flex-start;gap:16px;box-shadow:{"0 4px 14px rgba(16,117,145,.25)" if a else "0 1px 4px rgba(0,0,0,.06)"};"><div style="min-width:44px;height:44px;background:{nb};color:{nf};border-radius:50%;display:flex;align-items:center;justify-content:center;font-size:15px;font-weight:800;flex-shrink:0;">{s["num"]}</div><div style="flex:1;"><div style="font-size:15px;font-weight:700;color:{tf};margin-bottom:4px;">{s["title"]}{yh}</div><div style="font-size:12.5px;color:{df};margin-bottom:8px;line-height:1.5;">{s["desc"]}</div><div>{bdg}</div></div></div>'
    if i < len(steps) - 1:
        ch += f'<div style="display:flex;justify-content:center;height:32px;"><svg width="24" height="32" viewBox="0 0 24 32" fill="none"><line x1="12" y1="0" x2="12" y2="24" stroke="{_TEAL_MID}" stroke-width="2" stroke-dasharray="4 3"/><polygon points="6,20 18,20 12,30" fill="{_TEAL_MID}"/></svg></div>'

HTML(
    f'<div style="font-family:Inter,\'Segoe UI\',sans-serif;max-width:640px;margin:8px 0;"><div style="background:linear-gradient(135deg,{_TEAL} 0%,{_TEAL_MID} 100%);border-radius:12px 12px 0 0;padding:14px 20px;display:flex;align-items:center;gap:10px;"><svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="{_WHITE}" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg><span style="color:{_WHITE};font-size:14px;font-weight:700;">Twiga Learning Path — {track_name}</span></div><div style="border:2px solid {_TEAL_LIGHT};border-top:none;border-radius:0 0 12px 12px;padding:20px 20px 16px;background:#f9fdfe;display:flex;flex-direction:column;">{ch}<div style="margin-top:16px;font-size:11.5px;color:{_TEXT_MUTED};text-align:center;border-top:1px solid {_TEAL_LIGHT};padding-top:12px;">{footer}</div></div></div>'
)