Repository navigation
Add support for Apache.Arrow.Types.TimestampType #6809
Description
System Information (please complete the following information):
- Win 10
- Microsoft.Data.Analysis v0.20.1
- .NET Version: .NET 6
Describe the bug
I am getting below exception
Unhandled exception. System.NotImplementedException: timestamp
at Microsoft.Data.Analysis.DataFrame.AppendDataFrameColumnFromArrowArray(Field field, IArrowArray arrowArray, DataFrame ret, String fieldNamePrefix)
at Microsoft.Data.Analysis.DataFrame.FromArrowRecordBatch(RecordBatch recordBatch)
at Program.
at Program.(String[] args)
To Reproduce
Steps to reproduce the behavior:
- Take a .arrow file with one column of type Apache.Arrow.Types.TimestampType
- var dataframe = DataFrame.FromArrowRecordBatch(recordBatch);
- It will throw above exception
Expected behavior
The record batch should transform to Dataframe with appropriate tye to handle timestamp (Datetime maybe)
Activity
totalgit74 commentedon Sep 6, 2023
I'd like to see this implemented. Time-series require it and using the Arrow IPC/feather format to move data between .Net, R data.table and Python DataFrame as required is very handy in order to make language agnostic data stores/formats.
bhuesemann commentedon Oct 10, 2023
Same issue here while processing arrow data from BigQuery IPC. The timestamp datatyp is pretty much basic to many analysis use-cases and thus needs to be supported.