Skip to content

feat(ai): add forecast_dataframe and impute_dataframe - #2876

Open
miladh wants to merge 3 commits into
ai-ts-forecasting/03-low-level-apifrom
ai-ts-forecasting/04-dataframes
Open

miladh wants to merge 3 commits into
ai-ts-forecasting/03-low-level-apifrom
ai-ts-forecasting/04-dataframes

Conversation

@miladh

@miladh miladh commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2875.

Description

DataFrame variants of forecast and impute, validated and converted like insert_dataframe:

  • Each numeric column is one time series, named by its label, so column names must be strings.
  • Datapoints keep the API's meaning. A NaN is a gap with no reading, sent as a datapoint without a value: the model doesn't see it, and impute doesn't return it.
  • missing= is an optional boolean DataFrame with the same index and columns, sent as the API's missing flag: the datapoints to hide from the model whatever their value, which impute predicts. Like in the API, nothing is marked by default.
  • cohort= takes one cohort for every column, or a {column: cohort} mapping. Columns not in the mapping are processed independently.
  • The result is to_pandas() of the result list: the median in one column per time series, with a timezone-naive DatetimeIndex in UTC like other SDK result DataFrames. include_all_quantiles=True gives (column, quantile level) columns for every level the API returns.
  • A DataFrame without a DatetimeIndex, with repeated, non-string or non-numeric columns (booleans included) or infinite values, a cohort mapping that names a column not in the DataFrame, or a missing that doesn't match the DataFrame, is rejected before anything is sent.

Usage

import numpy as np
import pandas as pd
from cognite.client import CogniteClient

client = CogniteClient()
df = pd.DataFrame(
    {
        "23-PT-1101": [42.1, 42.4, 42.9],
        "23-PT-1201": [39.8, 40.1, 40.0],
        "24-TT-3001": [31.0, np.nan, 30.9],   # NaN: a gap with no reading
    },
    index=pd.date_range("2026-10-01", periods=3, freq="1min"),
)

# Every column independently
forecast = client.ai.time_series.forecast_dataframe(df)
forecast["23-PT-1101"]   # the median forecast

# The two compressor pressures jointly; 24-TT-3001 stays independent
forecast = client.ai.time_series.forecast_dataframe(
    df, cohort={"23-PT-1101": "compression-train-a", "23-PT-1201": "compression-train-a"}
)
all_levels = client.ai.time_series.forecast_dataframe(df, include_all_quantiles=True)

# Predict the gaps, and fill them with the predicted median
imputed = client.ai.time_series.impute_dataframe(df, missing=df.isna())
filled = df.fillna(imputed)

pandas is an optional extra, so the new tests are marked dsl. The ai.time_series docstring test moves to its own dsl-marked test, like datapoints and raw.

Checklist:

  • Tests added/updated: TestDataFrames in test_api/test_ai_time_series.py, and DataFrame integration tests in tests_integration/test_api/test_ai_time_series.py.
  • Documentation updated: docstrings.
  • The PR title follows the Conventional Commit spec.

🤖 Generated with Claude Code

@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from bc78e48 to 8e32224 Compare November 2, 2026 11:15
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 9d62a18 to 1bb7339 Compare November 2, 2026 11:15
@miladh
miladh marked this pull request as ready for review November 2, 2026 11:23
@miladh
miladh requested review from a team as code owners November 2, 2026 11:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces forecast_dataframe and impute_dataframe methods to both the async and sync AI Time Series APIs, allowing users to forecast and reconstruct missing values directly from pandas DataFrames. Feedback on the implementation of the _dataframe_to_input helper function highlights the need for improved robustness, specifically regarding DatetimeIndex validation, timezone-aware index handling, and validating that all columns are numeric to prevent cryptic runtime errors.

Comment thread cognite/client/_api/ai/time_series.py Resolved
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 1bb7339 to 0fd9b72 Compare November 4, 2026 21:41
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from 8802d44 to bf68252 Compare November 4, 2026 21:50
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 0fd9b72 to c20d0a9 Compare November 5, 2026 09:23
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from bf68252 to 9118e3c Compare November 5, 2026 09:23
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from c20d0a9 to 71a6259 Compare November 6, 2026 11:40
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch 2 times, most recently from 567f58e to d0fb961 Compare November 6, 2026 11:48
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from e1937b4 to 4e8d9dc Compare November 9, 2026 12:08
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch 2 times, most recently from 0d5bc2d to 99e2230 Compare November 9, 2026 12:58
miladh and others added 3 commits November 9, 2026 13:40
DataFrame versions of forecast and impute, validated and converted like
insert_dataframe: each numeric column is one time series, named by its
label, so column names must be strings. cohort is one cohort for every
column, or a mapping from column name to cohort.

Datapoints keep the API's meaning. A NaN is a gap with no reading, sent as
a datapoint without a value, so the model doesn't see it and impute
doesn't return it. Both methods take missing, an optional boolean
DataFrame sent as the API's missing flag: the datapoints to hide from the
model whatever their value, which impute predicts. Like in the API,
nothing is marked by default.

The result is to_pandas of the result list: the median in one column per
time series, with a timezone-naive DatetimeIndex in UTC like other SDK
result DataFrames; a naive input index is taken to be in UTC, as in
insert_dataframe. A DataFrame without a DatetimeIndex, with repeated,
non-string or non-numeric columns (booleans included, as the API takes
numbers only) or infinite values, a cohort mapping that names a column
not in the DataFrame, or a missing that doesn't match the DataFrame, is
rejected before anything is sent. The docstrings note that the endpoints
are in beta.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dataframe

Like to_pandas, the DataFrame methods return the median only by default.
include_all_quantiles returns every quantile level the API returns
instead, as (column, quantile level) columns, and is passed on to
to_pandas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aframe

Forecasts a DataFrame with the median and with every quantile level, and
imputes the gaps marked in missing: the result has one row per marked
timestamp, and filling the gaps with it leaves no NaN.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from d55bb65 to 9e58935 Compare November 9, 2026 13:41
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from 99e2230 to 1daa4fc Compare November 9, 2026 13:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant