DataFrame - add support for vbuffer #5872

Activity

1. LittleLittleCloud

   added

   Microsoft.Data.Analysis

   on Jul 7, 2021
2. lqdev commented on Aug 11, 2021

   

   lqdev

   on Aug 11, 2021

   More actions

   To give an example. For the following dataset

   5.1	3.5	1.4	0.2	setosa
   4.9	3.0	1.4	0.2	setosa
   4.7	3.2	1.3	0.2	setosa

   Using the following IDataView schema

   type ModelInput = {
       [<LoadColumn(0,3)>] Features: float32 array
       [<LoadColumn(4)>] Label: string
   }

   Throws the following error when ToDataFrame is called.

   System.NotSupportedException: VBuffer`1 is not a supported column type.

   👍React with 👍1Reacted by Gleb Krivosheev - Moody's
3. LittleLittleCloud commented on Feb 28, 2022

   

   LittleLittleCloud

   on Feb 28, 2022

   MemberAuthor

   More actions

   @ericstj @eerhardt @michaelgsharp
   What it takes to add support for VBuffer or more generically: Object in DataFrame API, any roadmap for that
4. michaelgsharp commented on Mar 2, 2022

   

   michaelgsharp

   on Mar 2, 2022

   Contributor

   More actions

   This is going to need some further investigation to see what it would take. Its in our roadmap but we will be taking a look at it after we get TorchSharp resolved.

   👍React with 👍1Reacted by Eric StJohn
5. michaelgsharp

   added this to the ML.NET Future milestone on Mar 2, 2022
6. ericstj commented on Mar 2, 2022

   

   ericstj

   on Mar 2, 2022

   Member

   More actions

   @eerhardt have you thought of this before or are you aware of any discussion with Prasanth about it? If we have an idea of how it would work we could write that up here in case someone else might be interested in helping fix this.
7. eerhardt commented on Mar 2, 2022

   

   eerhardt

   on Mar 2, 2022

   Member

   More actions

   I really haven't given it deep thought. I know it is a problem, but I'm not sure how exactly to structure a DataFrameColumn that contains VBuffer instances. They are a little bit at odds, since VBuffer is supposed to be a "buffer" that changes as you "cursor" over the rows of an IDataView. Whereas DataFrame wants everything to be loaded at once in memory. But maybe we can have a column derived from DataFrameColumn that contains a distinct VBuffer for every row in the DataFrame.

   That's about as far as I've gone thinking of this.

   See also #5721

   👍React with 👍1Reacted by Eric StJohn
8. jeffhandley commented on Nov 27, 2023

   

   jeffhandley

   on Nov 27, 2023

   Member

   More actions

   @michaelgsharp or @JakeRadMSFT -- Was this completed with #6409, or is there more to do here still?
9. rosebyte

   closed this as completedon Sep 3, 2026
10. github-actions

    locked and limited conversation to collaborators on Oct 4, 2026