Supporting `Union`

[tlambert03]

quick thought: plenty of functions accept multiple types for a given argument using typing.Union. It would be nice if we could somehow support that. Perhaps if a Union is specified, the widget itself could be "swappable" (i.e. right click on it and you can select from the various types in the Union)

[Czaki]

Please also support Optional.

Right-click is not an intuitive one. There need to be something like Combobox signal (a small button with a down arrow).

[tlambert03]

for optional, see https://github.com/napari/magicgui/issues/145

[jo-mueller]

Hi,

I just stumbled over this issue because it's what I am trying to do with various workarounds. I think this would greatly ease (in the context of napari) to work with metadata of layers in plugins while still being able to work with functions easily from code (e.g., in notebooks).

In particular, I would like to do something like this:

def some_function(feature1: Union(napari.layers.Points, np.ndarray)):
  if isinstance(feature1, napari.layers.Layer):
    feature1 = feature1.features['my_feature']
  ...

As a custom workaround I solved this by creating custom types and registering them with magicgui but I'm not sure whether it's advisable to create too many types that are specific to my problem ^^"

Edit: As for how this would look like, I think displaying an appended list would perfectly do it.

[Czaki]

@jo-mueller
I do not understand what you expect to have in the interface. The core problem with annotation np.ndarray is that it does not contain any context information.

Did you expect that Union[napari.layers.Points, np.ndarray] will be rendered same as napari.layers.Points. Or something else?

If option that I suggest then why not just

def some_function(feature1: Union[napari.layers.Points, np.ndarray]):
  if isinstance(feature1, napari.layers.Layer):
    feature1 = feature1.features['my_feature']
  ...

def some_function_mgui(feature1: napari.layers.Points):
  return some_function(feature1)

That does not require any "ignore" logic.

[jo-mueller]

I do not understand what you expect to have in the interface. The core problem with annotation np.ndarray is that it does not contain any context information.

Hi @Czaki ,

thanks for the reply. You mean splitting the function into one function that does the calculation and another one that's actually visible to magicgui? I thought about that as well but for many functions of that type it would mean a lot of boilerplate code and I'm afraid that it will become difficult to manage.

Since Python doesn't actually complain about types at the runtime of a function until it throws an error inside the function I guess I could also simply do

def some_function_mgui(feature1: napari.layers.Points):
    if isinstance(feature1, napari.layers.Layer):
      feature1 = feature1.features['my_feature']

Although I'd prefer to have functions correctly annotated.

[Czaki]

Yes, but the problem is that you expect magicgui to silently ignore some of the annotation, then verbose fail. This could be problematic for other users to understand why the output of magicgui looks different than expected.

Maybe developing some decorator that would allow declaring what should be ignored could be a good workaround. But I would also wait for @tlambert03 opinion.

[tlambert03]

yeah, I kind of see this as two distinct, if related, issues:

1. the issues of supporting Unions of already recognized magicgui types
2. the issue of dealing with unrecognized types, and how they might interact in Unions

For issue one, the challenge is whether something like this, where all type annotations are registered/known to magicgui, would successfully limit the options in the visible dropdown (e.g. here, to just Points or Images)

def func(input: Union[napari.layers.Points, napari.layers.Image]): ...

or, if we combine layers and arrays, as in your example, this could also be something like:

def func(input: Union[napari.layers.Points, napari.types.PointsData]): ...

I think that these would already be possible, but the logic would actually be on the napari side (not here in magicgui) to register, I think, all the various unions that it wants to support, and it would similarly be up to napari to handle the dropdown widgets, as well as the current value passed to gui functions. To support this, someone would need to play around with the _magicgui.py file in napari and see what's possible

For issue number 2: originally posed by @jo-mueller here

def some_function(feature1: Union[napari.layers.Points, np.ndarray]):

as @Czaki points out, now we're dealing with something that magicgui/napari knows what to do with, and something it has no idea about (np.ndarray). There, I agree pretty much with @Czaki that the correct approach here is either to create a separate function just for the purpose of the widget, or to annotate it as

def some_function(feature1: Union[napari.layers.Points, napari.types.PointsData]):

(where, recall, napari.types.PointsData is a NewType of np.ndarray). I believe that would still make your annotations correct @jo-mueller. And indeed, that general goal of creating annotated np.ndarrays with specific connotations/ontologies is that big ultimate goal we discuss a lot (sometimes referred to as "imtypes" or "image-types")

The main thing thing I would not want to support here is simply "ignoring" np.ndarray if it happens to come in a union with something that magicgui recognizes. That's too magical, and would create more problems than it's worth. There I would also suggest creating a new helper function that just defers to the more general function.

[jo-mueller]

The main thing thing I would not want to support here is simply "ignoring" np.ndarray if it happens to come in a union with something that magicgui recognizes.

Thanks for the explanations! I see that point ( I think ignoring it would have been what I had in mind) and I agree that's probably not worth the additional trouble.

Maybe a better scenario of where Union would make sense would be Union[PointsData, SurfaceData]. Functions operating on points could equally operate on Surfaces (or the vertices thereof, respectively).

[tlambert03]

Maybe a better scenario of where Union would make sense would be Union[PointsData, SurfaceData]. Functions operating on points could equally operate on Surfaces (or the vertices thereof, respectively).

yep, agreed! That's the bit that I think could already work fine, but napari would have to register something for that union. (I see that the combinatorics get hairy... and it's possible magicgui could also help here... but I think it's best to have a little input from the downstream library)