[Feature Request] add ImageFolder/DatasetFolder: incremental index caching to avoid full filesystem re-scans #9659

ImageFolder and DatasetFolder do a full recursive scan of the directory
(os.walk / os.scandir) every time they're instantiated, even if nothing on
disk has changed. For large datasets, or anything on a network mount, this
can take minutes — every single run.

This hurts most with: restarting training scripts during debugging, jobs
that get preempted and restarted on clusters, distributed training where
every rank/worker rescans the same unchanged tree, and datasets that grow
over time with no way to update the index without a full rescan.

An opt-in cache_index flag with an incremental mode that only rescans
folders that actually changed.

from torchvision.datasets import ImageFolder

dataset = ImageFolder("data/train/")

dataset = ImageFolder(
    "data/train/",
    cache_index=True,
    cache_mode="incremental",
    cache_path=None,
)

First run scans normally and writes the index to cache_path. Later runs
check folder mtimes against the cache, rescan only what changed, and merge
the result — no full rescan unless nearly everything changed.

Alternatives considered

Leaving this to user-built wrappers (already common, but inconsistent and
often buggy), caching by default (risky — could silently hide stale data),
and a heavier format like LMDB/SQLite (more than this needs).

Pure Python change in torchvision/datasets/folder.py, fully backward
compatible.