Repository navigation
FileSystemWatcher on Linux uses an excessive amount of resources #62869
Description
Activity
@neilmayhew I agree that the current implementation is not perfect.
@tmds do you have any recommendations?
We can look at improving the implementation. Maybe some simpler changes can be made related to:
To make things worse, many ASP.net applications running in a container are incorrectly configured and use more FileSystemWatchers than they need to.
@neilmayhew can you give some more information about this?
@tmds Sorry for the slow reply.
Both of the blogs I linked to in the description mention ASP.net, and refer to a solution that looks like this:
configuration = new ConfigurationBuilder() .SetBasePath(Directory.GetCurrentDirectory()) .AddJsonFile("appsettings.json", optional: true, reloadOnChange: false);
However, one of the comments on the first blog says that wasn't enough and links to a Stack Overflow question about
WebApplicationFactorythat has a recursive solution. That in turn links to another question, which doesn't seem to be about ASP.net specifically. So I don't think this is exactly a problem with ASP.net, more about the default behavior ofConfigurationBuilder, but it seems that the problem crops up more often with, or is made worse by, ASP.net.A web search for the error message in question turns up a zillion hits, so a lot of people are encountering this, but previously no-one seems to have figured out the root of the problem and everyone is just finding ways to work around it.
It should be possible to watch thousands of files and directories on Linux without much overhead, but that's not possible with the way it's implemented currently.
The solution I'm thinking of is to use class variables for a common inotify instance and background thread, instead of instance variables and an instance and thread per
FileSystemWatcherobject. The common thread would send events just like the single thread does, but it would need to keep a dictionary mapping from inotify watch descriptor toFileSystemWatcherso it can obtain the .Net event it needs to send when it reads an inotify event from the inotify instance. This would be only slightly more complex than the current implementation, and the complexity would be concerned only with mutexing and keeping the dictionary updated whenFileSystemWatcherinstances are created and destroyed.BTW, I don't think it would be appropriate to switch the implementation to use
fanotifyinstead ofinotify. Althoughfanotifyis newer, it's designed for a different use case and isn't a replacement forinotify. I'll update the description.I see that the implementation already uses a mutexed dictionary for mapping watch descriptors to absolute paths (
_wdToPathMap) so a lot of the infrastructure I discussed is already in place and could be repurposed quite easily.referenced pull request is not a proper fix, but a workaround... :(
My view on this is there is a fundamental limitation in the API of FileSystemWatcher that causes this issue: it can only watch a single path. And because of that, user's are drawn to create multiple FileSystemWatchers, to watch multiple directories. And since a single FileSystemWatcher holds a single IOCP port, or inotify queue, this is artifically inflating the number of such queues that the system needs to allocate.
In IKVM we had been using FileSystemWatcher as the backing support for our java.nio.file.WatchService support. But, we ran into this issue. WatchService in Java starts out pretty much the same: user's create an instance, and that instance owns a queue or port. However, user's then register and unregister multiple potentially unrelated paths to listen to with the service. So, in that ecosystem, user's create WatchService instance less frequently. Usually one per problem domain. But then register large directory hierarchies, etc, with that single one. But they're in control of how many ports they allocate, and how many directories they watch with each.
So, as an API suggestion that might alleviate this: support multiple Paths. Yes, FileSystemWatcher.Path is a single property. And the class takes a single value in it's ctor. But maybe there's some clean way to extend this a bit.
@wasabii Your understanding matches what I was saying in the description. However, I'm suggesting a solution that doesn't require any changes to the API. (I'm pretty sure changing the API isn't an option.)
In the solution I'm suggesting, there would be a single, global
inotifyqueue for allFileSystemWatchers (ie a class variable) instead of having a separateinotifyqueue perFileSystemWatcher(ie an instance variable) as there is at present.As I've outlined, I think this would be relatively simple to implement. I wish I had the time to do it myself but the job I'm in now doesn't use .Net and so I can't justify it.
While using .NET 9 on Fedora 41 to create a web application I keep frequently running into this issue. Here's an example of what's happening. This is while I'm using VSCode. Just wondering if there is a status update on this issue?
find /proc/*/fd/* -type l -lname 'anon_inode:inotify' -exec sh -c 'cat $(dirname {})/../cmdline; echo ""' \; 2>/dev/null /opt/google/chrome/chrome /opt/google/chrome/chrome /opt/google/chrome/chrome /opt/google/chrome/chrome /opt/google/chrome/chrome /opt/google/chrome/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --string-annotations --crashpad-handler-pid=107648 --enable-crash-reporter=, --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,1162470407443857328,3612120863378465341,262144 --disable-features=EyeDropper --variations-seed-version=20250214-063938.160000 /usr/bin/plasma-browser-integration-hostchrome-extension://cimiefiiaegbelhefglklhhakcgmhkai/ /usr/bin/dbus-broker-launch--scopeuser /usr/bin/kwalletd6--pam-login1314 /usr/bin/startplasma-wayland /usr/libexec/uresourced--user /usr/bin/wireplumber /usr/bin/wireplumber /usr/bin/wireplumber /usr/libexec/kf6/baloo_file /usr/libexec/kf6/baloo_file /usr/libexec/kf6/baloo_file /usr/libexec/xdg-desktop-portal /usr/bin/dbus-broker-launch--config-file=/usr/share/defaults/at-spi2/accessibility.conf--scopeuser /usr/bin/kded6 /usr/bin/kded6 /usr/bin/kded6 /usr/bin/kded6 /usr/bin/plasmashell--no-respawn /usr/bin/plasmashell--no-respawn /usr/bin/plasmashell--no-respawn /usr/bin/plasmashell--no-respawn /usr/bin/plasmashell--no-respawn /usr/bin/kwalletmanager5--kwalletd /usr/libexec/kactivitymanagerd /usr/bin/gmenudbusmenuproxy /usr/libexec/org_kde_powerdevil /usr/libexec/org_kde_powerdevil /usr/libexec/xdg-desktop-portal-kde /usr/libexec/xdg-desktop-portal-kde /usr/libexec/xdg-desktop-portal-kde /usr/libexec/xdg-desktop-portal-gtk /usr/bin/abrt-applet--gapplication-service /usr/libexec/DiscoverNotifier--check-delay20 /usr/libexec/DiscoverNotifier--check-delay20 /usr/libexec/flatpak-session-helper yakuake /usr/bin/akonadi_ical_resource--identifierakonadi_ical_resource_0 /usr/bin/akonadi_maildir_resource--identifierakonadi_maildir_resource_0 /usr/bin/dolphin--daemon /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csproj /usr/lib64/dotnet/dotnet/usr/lib64/dotnet/sdk/9.0.102/DotnetTools/dotnet-watch/9.0.102-servicing.24611.1/tools/net9.0/any/dotnet-watch.dll--project/home/th3oth3rjak3/Development/csharp/TheHub/TheHub.Server/TheHub.Server.csprojI need this too. I made a port of System.Web for NET Core, and it watches a lot of paths, I cannot use the current FileSystemWatcher implementation.
In the solution I'm suggesting, there would be a single, global inotify queue for all FileSystemWatchers (ie a class variable) instead of having a separate inotify queue per FileSystemWatcher (ie an instance variable) as there is at present.
This makes sense: (by default) a user may have only 128 inotify instances. The limit on the watchers is per user, so changing to a single inotify instance won't limit the amount of directories that can be watched. The inotify instance queue can hold 16k events, which should be plenty as .NET can dequeue the events continuously and then dispatch them to be handled on other threads.
I'll look into this.
- added a commit that references this issue
on Jan 26, 2026 - locked and limited conversation to collaborators
on Feb 26, 2026 - added a commit that references this issue
on Mar 2, 2026
Description
There are many reports on the web of running into this message:
The Linux implementation of
FileSystemWatchercreates a new thread with its own inotify instance for every directory that's watched. This is very expensive in kernel resources, and often hits system limits as can be seen from the message. It's particularly problematic when running in a container because the per-UID quota isn't namespaced and is therefore shared among all containers on the same host. To make things worse, many ASP.net applications running in a container are incorrectly configured and use moreFileSystemWatchers than they need to. Such a container consumes much more than its fair share of the available inotify instances, which then causes other containers in the pod (eg with k8s) to fail unexpectedly.Instead, the implementation should have a single thread with a single inotify instance that's shared by all the
FileSystemWatchers in the process. Instead of adding new instances (which are expensive) the implementation would add new watches (which are cheap). It would then be possible to accommodate even the most egregious of ASP.net apps with ease..Configuration
Any Linux system.
Regression?
No. This has been going on for a long time and there are a lot of puzzled blog posts about it.
Data
Blog posts:
Related issues:
Analysis
The inotify system was never intended to be used like this.
However, rather than changing the implementation to use a single shared thread, it might be a better use of resources to write a new implementation usingfanotifyas suggested in:Use new fanotify Linux API in FileSystemWatcher #30495