Repository navigation
Make to_image reading return tensors in CHW聽#9547
Description
馃殌 The feature
Make to_image reading return tensors in CHW
Motivation, pitch
vision/torchvision/transforms/v2/functional/_type_conversion.py
Lines 10 to 25 in da44627
| @torch.jit.unused | |
| def to_image(inpt: Union[torch.Tensor, PIL.Image.Image, np.ndarray]) -> tv_tensors.Image: | |
| """See :class:`~torchvision.transforms.v2.ToImage` for details.""" | |
| if isinstance(inpt, np.ndarray): | |
| output = torch.from_numpy(np.atleast_3d(inpt)).permute((2, 0, 1)).contiguous() | |
| elif isinstance(inpt, PIL.Image.Image): | |
| output = pil_to_tensor(inpt) | |
| elif isinstance(inpt, torch.Tensor): | |
| output = inpt | |
| else: | |
| raise TypeError( | |
| f"Input can either be a pure Tensor, a numpy array, or a PIL image, but got {type(inpt)} instead." | |
| ) | |
| return tv_tensors.Image(output) | |
import pyvips
import torchvision,torchvision.transforms.v2
img = pyvips.Image.new_from_file(r"F:\test.jpg")
tensor = torchvision.transforms.v2.functional.to_image(img.numpy())
tensor.shape
#torch.Size([3, 1830, 1370])
tensor.shape, tensor.stride(), tensor.is_contiguous(memory_format=torch.channels_last), tensor.unsqueeze(0).is_contiguous(memory_format=torch.channels_last)
# (torch.Size([3, 1830, 1370]), (2507100, 1370, 1), False, False) # Not memory_format=torch.channels_last
new_tensor = torch.from_numpy(tensor.numpy()).permute((2,0,1))
new_tensor.shape, new_tensor.stride(), new_tensor.is_contiguous(memory_format=torch.channels_last), new_tensor.unsqueeze(0).is_contiguous(memory_format=torch.channels_last)
# (torch.Size([3, 1830, 1370]), (1, 4110, 3), False, True) # This is memory_format=torch.channels_lasthttps://docs.pytorch.org/vision/main/transforms.html#range-and-dtype
...
Transforms tend to be sensitive to the input strides / memory format. Some transforms will be faster with channels-first images while others prefer channels-last. Like torch operators, most transforms will preserve the memory format of the input, but this may not always be respected due to implementation details. You may want to experiment a bit if you鈥檙e chasing the very best performance. Using torch.compile() on individual transforms may also help factoring out the memory format variable (e.g. on Normalize). Note that we鈥檙e talking about memory format, not tensor shape.Note that resize transforms like Resize and RandomResizedCrop typically prefer channels-last input and tend not to benefit from torch.compile() at this time.
...
As torchvision.transforms.v2.functional.to_image usually being the starting point of the processing pipeline, the latter process of it would always be float().div_(255) or to_dtype then resize and etc., these all prefer channels_last memory_format for better performance.
Therefore the starting point of the processing pipeline should keep its NHWC memory format. And which make the memcopy by contiguous() avoid that always happen (since image is always HWC when read in numpy)
Thanks for your reading.
I could make a PR
Activity
[-]Make to_image reading return tensors in CHW[/-][+]Make `to_image` reading return tensors in CHW[/+]