Explanation of signal_stream concept

[PeterNSteinmetz]

@samuelgarcia It is still very unclear to me how this signal_stream concept and the former groups of channel_indexes are supposed to work / are related. The :TODO: for PR #949 is most needed here.

My prior understanding was that in the RawIO interface, groups of channel_indexes passed to methods like get_analog_signal_chunk had to have the same sampling rate (to be part of the same Segment as well), same t_start and size. It was then best if the group_id for such a set of channels was the same, so that they would be grouped appropriately.
(Frankly this was never clearly explained in the docs and was just implicit in the code.)

Is a signal_stream supposed to replace this idea? And all channels in a single signal_stream are supposed to have the same sampling rate, t_start, and length?

Can one keep the group_ids in an existing RawIO and these will still work? Or do they need to now correspond to signal_stream? What happens if the groupings by group_id and signal_stream are not the same?

[samuelgarcia]

Yes the signal tsream is more explicit version of this : group of channel with same feqtures (sample rate/size/t_sart) that can be retrieve at the same time.

Can one keep the group_ids in an existing RawIO and these will still work? Or do they need to now correspond to signal_stream?

You can keep the same 'goup_id' as before but now it is renamed 'stream_id' (a string) for channel maps.
But each stream_id have to be declared in "signal_stream".

What happens if the groupings by group_id and signal_stream are not the same?

No "signal_channel" table have to be consistent with "signal_stream" table. It is nested.

I would be happy to make a call with you for more explanation.

[PeterNSteinmetz]

The ultimate target of this design is also unclear. The requirements for RawIO and what is necessary for BaseFromRaw to work should be made explicit. Particularly if not all RawIOs need to be compatible with BaseFromRaw.

One big item here is whether a RawIO stream must be the same over all Segments ? From what I can see, a BaseRawIO is allowed to have multiple Segments. By definition, that implies that different segments could have different clocks, and therefore different sampling rates and beginning times of the clocks. Such differences would NOT be compatible with one stream spanning multiple Segments and a model where one stream is mapped to one AnalogSignal. And that appears to be an assumption made in the current code, for example

Clarification of this is critical for RawIOs where there may be multiple channels created for the same recorded channel which have different start_time and lengths, but the same sampling rate. Any RawIO dealing with the case where a recording for a channel is started and stopped, or contains gaps, such as can happen with Neuralynx, will have to deal with the situation. The way I have been dealing with this so far in other IOs (though not the neo.raw.NeuralynxRawIO itself) is to use a different group_id.

It seems like the intention here is to always have all the RawIO streams and other objects in the RawIO belong to one segment and one block. If so, it may be clearest to explicitly state that assumption and to deprecate the block_index and seg_index arguments.

Yes, a call might be good. I am on Skype now as PeterNSteinmetz. Can also use Zoom if you send an invite.

[PeterNSteinmetz]

At a higher level, the other reason this matters is in ephyviewer. Presently neoanalogsignalsources:get_analog_sources_by_suffix_from_neo_rawio uses the neorawio.get_group_signal_channel_indexes call. I guess that method is gone now?

If so, what will it be replaced by? Some sort of search on the stream_id for all the channels?

[PeterNSteinmetz]

Just created PR #970 which has comments in the code on much of this.

I do think that if all the objects in a BaseRawIO are supposed to share a common clock that it may be best to deprecate the blocks and segments arguments to these functions and instead explicitly state that as a requirement for a BaseRawIO class. Otherwise, the BaseRawIO stream really should be for only one segment, not across all segments.

[samuelgarcia]

Close by #970