Repository navigation
feat(ai): add data classes for time series forecasting and imputation - #2873
Conversation
Request and result types for the upcoming ai/timeseries/{forecast,impute}
endpoints:
- InputDatapoint / InputTimeSeries: request objects (dataclasses, like
DatapointsQuery). Datapoints can be InputDatapoint, (timestamp, value)
tuples or API-shaped dicts; timestamps accept ms, datetime or time-shift
strings.
- QuantileDatapoint: one step with quantile level -> value. Non-numeric keys
are ignored on load.
- ForecastResult(List) / ImputeResult(List): one result per input label, with
quantile_levels and cohorts from the response metadata (like NodeList's
typing/debug), lookup by .get(label=...), and to_pandas() with a
DatetimeIndex.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ea1105b to
12e6a69
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces new data classes and helper functions in cognite/client/data_classes/ai.py to support forecasting and imputation features, including InputDatapoint, InputTimeSeries, QuantileDatapoint, ForecastResult, ForecastResultList, ImputeResult, and ImputeResultList, along with corresponding unit tests. Feedback on the changes includes handling empty inputs in _concat_columns to prevent pandas errors, supporting lists in addition to tuples for input datapoints, and safely handling potential null metadata values in API responses to avoid attribute errors.
pd.concat raised "No objects to concatenate" for a ForecastResultList or ImputeResultList with no results (e.g. an empty slice). Other SDK lists return an empty DataFrame, so do the same. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Re @niekbeckers' question on reusing — Written by Claude |
Direct pd.to_datetime calls are only allowed in _pandas_helpers.py (enforced by test_pandas_timestamp_constructors_only_used_in_pandas_helpers), so build the empty DatetimeIndex with to_pandas_datetime_index instead. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2873 +/- ##
==========================================
+ Coverage 93.49% 93.72% +0.22%
==========================================
Files 527 541 +14
Lines 55489 57533 +2044
==========================================
+ Hits 51879 53920 +2041
- Misses 3610 3613 +3
🚀 New features to boost your workflow:
|
The forecast and impute endpoints return their series under "items", like other CDF list endpoints, rather than "timeSeries". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ForecastInputTimeSeries InputDatapoint and InputTimeSeries were too generic for data_classes.ai, which also holds the document AI classes. The forecast and impute endpoints take the same request body, so both use these classes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a load method ForecastInputTimeSeries.datapoints accepted datapoints, tuples and dicts, which widened its type for little gain. It now takes ForecastInputDatapoint only, and ForecastInputDatapoint.load creates one from a tuple, a list or an API-style dict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API keys quantiles by their level only ("0.50", or more decimals when
needed, like "0.025"), so there are no other keys to skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
metadata, with quantileLevels and cohorts, is always in the response, and the levels are numbers already, so there is nothing to default or convert. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Results have no id, external id or instance id, so get takes just the label. Labels are unique, since duplicate labels in a request are rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_handle_unknown_arguments_when_loading adds a made-up key to every dict in a resource. Every key in a quantiles dict must be a quantile level, like "0.50", so loading fails on the made-up key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks @haakonvt for the useful feedback! It's addressed in 8772cc0e..9ab0e18f, with a reply on each thread. One question is still open, on the The range also includes a recent API change: the array of series is now called It also adds a |
Results hold every quantile level the API returns, which makes the DataFrame and the notebook view wide, and the levels can change over time. to_pandas now takes quantile_levels, the levels to include: the median (0.5) by default, a list of levels, or None for all of them. The notebook view uses the default. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Håkon V. Treider <haakonvt@gmail.com>
The supported formats are matched directly: a datapoint, a (timestamp, value) or (timestamp, value, missing) sequence, or a dict with a timestamp. A dict without a timestamp, or a sequence of another length, raises the same TypeError as any other unsupported input. The docstring of ForecastInputTimeSeries.datapoints says to load datapoints with it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The result lists took the API response in a separate _load_response. CogniteResourceList already has _load_raw_api_response for lists that need more than the items of a response, like NodeList with typing, so use it, with its signature. _load keeps taking the items only, as load passes them. The responses of all requests of a call are merged: their items, and their cohorts in the order the API lists them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o_pandas Taking quantile levels as floats is more than to_pandas needs. It now takes include_all_quantiles: by default the result holds the median in one column per time series, with no quantile level in the columns, and with include_all_quantiles every level is a separate column, as (label, quantile level) columns for a list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
haakonvt
left a comment
There was a problem hiding this comment.
LGTM, a few more nits, but nothing blocking as we are "a long stack away" from making these endpoints public 😅
…ists The API now returns quantileLevels and cohorts in a metadata object in every forecast and impute response. Keep that shape, following the SDK's precedents for lists with data beyond the items of a response: - ForecastMetadata holds both fields, required as in the API, and is a CogniteResource like TypeInformation, which NodeList holds for typing. - ForecastResultList and ImputeResultList have it as metadata, the key the API uses, instead of two separate attributes. - Like SyncRecordList, whose nextCursor and hasNext are always in the response, metadata is not optional, and a list built from results alone, like a slice or load(dump()), gets an empty default. - Like NodeList, dump_raw returns the items and the metadata as the API does, and _load_raw_api_response loads them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
haakonvt
left a comment
There was a problem hiding this comment.
🦄 one nit you can put in a later pr
| # order the API lists them. | ||
| metadata = ForecastMetadata( | ||
| quantile_levels=responses[0]["metadata"]["quantileLevels"], | ||
| cohorts=list(dict.fromkeys(c for response in responses for c in response["metadata"]["cohorts"])), |
There was a problem hiding this comment.
You can use from cognite.client.utils._auxiliary import remove_duplicates_keep_order
Description
First PR in a stack that adds SDK support for the
ai/timeseries/forecastandai/timeseries/imputeendpoints. The API is in beta.The stack builds the feature in two layers, both under
client.ai.time_series:forecast/impute: a low-level client that mirrors the API. PRs 1–4.forecast_with_retrieval/impute_with_retrieval: convenience methods for CDF time series by instance id. PRs 5–8.This PR only adds data classes, shaped like the API's request and response, with no API calls yet:
ForecastInputDatapointandForecastInputTimeSeries: the request objects, used by both endpoints. They're dataclasses, likeDatapointsQuery.ForecastInputDatapoint.loadcreates a datapoint from a tuple, a list or an API-style dict.QuantileDatapoint: one predicted point, quantile level -> value.ForecastResult(List)andImputeResult(List): one result per input series. The lists hold the response'smetadata(quantile levels and cohorts) asForecastMetadata, following SDK precedent:_load_raw_api_responseanddump_rawlikeNodeList, and a non-optional field with an empty default for lists built from results alone, likeSyncRecordList.to_pandas()gives one column per series with the median;include_all_quantiles=Truegives(label, quantile level)columns for every level.Usage (this PR on its own)
Checklist:
test_data_classes/test_ai.py. The pandas tests are markeddsl.🤖 Generated with Claude Code