Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ same thread:
with lock_b: # RuntimeError: Deadlock detected!
pass

This would normally deadlock forever because ``lock_b`` waits for a lock that ``lock_a`` holds in the same thread.
This would deadlock forever because ``lock_b`` waits for a lock that ``lock_a`` holds in the same thread.
filelock detects this and raises ``RuntimeError`` with a message suggesting ``is_singleton=True`` as the fix.

This detection only applies to blocking acquires (``timeout < 0``) within the same thread. Non-blocking or timed
Expand All @@ -94,8 +94,8 @@ file handle can lock it. When a process dies, the OS automatically releases its

- - Pros
- Cons
- - ✓ Enforced by the kernel—foolproof.
- ✗ Exclusive only—no reader/writer distinction (use ReadWriteLock for that).
- - ✓ Enforced by the kernel, so it is foolproof.
- ✗ Exclusive only, with no reader/writer distinction (use ReadWriteLock for that).
- - ✓ Works even if your process crashes.
- ✗ Unreliable on some network filesystems.
- - ✓ No network filesystem issues.
Expand All @@ -122,8 +122,8 @@ the lock file additionally stores the process creation time to guard against PID
- - Pros
- Cons
- - ✓ Works on any filesystem, including network mounts.
- ✗ Not enforced—requires cooperation (a buggy process can ignore it).
- - ✓ Portable—same code works everywhere.
- ✗ Not enforced; it requires cooperation (a buggy process can ignore it).
- - ✓ Portable; the same code works everywhere.
- ✗ Higher overhead than OS-level locks.
- - ✓ Can detect stale locks (all platforms).
- ✗ Cross-host detection doesn't work (stale locks from other hosts require manual cleanup).
Expand All @@ -134,7 +134,7 @@ the lock file additionally stores the process creation time to guard against PID

**Windows**
Uses the :class:`WindowsFileLock <filelock.WindowsFileLock>` class, backed by ``msvcrt.locking``. This is enforced
by Windows, so all code running on the system respects it—whether it uses filelock or not.
by Windows, so all code running on the system respects it, whether it uses filelock or not.

The lock is exclusive and works reliably on local filesystems. Network filesystem (SMB) support is available but
considered less reliable.
Expand All @@ -148,7 +148,7 @@ the lock file additionally stores the process creation time to guard against PID
Uses the :class:`UnixFileLock <filelock.UnixFileLock>` class, backed by ``fcntl.flock``. This is the POSIX standard
for file locking and enforced by the kernel.

Works best on local filesystems. Network filesystems (NFS) may have issues—locking isn't always reliable on NFS even
Works best on local filesystems. Network filesystems (NFS) may have issues; locking isn't always reliable on NFS even
in POSIX-compliant systems.

Lock file cleanup: Unix and macOS delete the lock file reliably after release, even in multi-threaded scenarios.
Expand All @@ -163,7 +163,7 @@ the lock file additionally stores the process creation time to guard against PID
Which lock type should I use?
*******************************

**Start with FileLock** — the platform-aware alias that automatically chooses the best backend:
**Start with FileLock**, the platform-aware alias that automatically chooses the best backend:

.. code-block:: python

Expand Down Expand Up @@ -336,7 +336,7 @@ On older platforms without ``O_NOFOLLOW``, prefer :class:`UnixFileLock <filelock

**Locks on network filesystems**
OS-level locks (FileLock on Windows/Unix) are unreliable on network filesystems (NFS, SMB). This is a fundamental
limitation of how network filesystems work—they don't reliably support locking semantics.
limitation of how network filesystems work; they don't reliably support locking semantics.

For **exclusive locking** on NFS, use :class:`SoftFileLock <filelock.SoftFileLock>`. For **reader/writer
locking** (shared readers + exclusive writers) on NFS, use :class:`SoftReadWriteLock <filelock.SoftReadWriteLock>`,
Expand All @@ -345,11 +345,11 @@ On older platforms without ``O_NOFOLLOW``, prefer :class:`UnixFileLock <filelock

**Locks across different machines**
A lock on one machine doesn't stop another machine from accessing the resource unless they use a centralized locking
service — or a shared filesystem plus filelock's soft locks. On a multi-node slurm/HPC cluster,
service, or a shared filesystem plus filelock's soft locks. On a multi-node slurm/HPC cluster,
:class:`SoftReadWriteLock <filelock.SoftReadWriteLock>` is correct across compute nodes sharing an NFS mount.

**Read-write semantics**
FileLock is exclusive only—readers block writers and vice versa. If you need multiple readers with occasional
FileLock is exclusive only; readers block writers and vice versa. If you need multiple readers with occasional
writers, use ReadWriteLock instead.

**Atomicity without locks**
Expand All @@ -363,7 +363,7 @@ On older platforms without ``O_NOFOLLOW``, prefer :class:`UnixFileLock <filelock
# if Process B doesn't use the lock
open("data.txt").write("A")

Locks require cooperation—all code accessing a resource must use the same lock.
Locks require cooperation; all code accessing a resource must use the same lock.

*******************
Design trade-offs
Expand Down Expand Up @@ -412,9 +412,9 @@ How does SoftReadWriteLock work on NFS?
:class:`SoftReadWriteLock <filelock.SoftReadWriteLock>` fills the gap left by ``ReadWriteLock`` on network filesystems.
It stores state as a small directory tree next to the lock file:

- ``<path>.state`` — a short-held :class:`SoftFileLock <filelock.SoftFileLock>` used as the state mutex during transitions.
- ``<path>.write`` — the writer marker; its presence blocks readers and other writers.
- ``<path>.readers/<host>.<pid>.<uuid>`` — one marker file per active reader.
- ``<path>.state`` is a short-held :class:`SoftFileLock <filelock.SoftFileLock>` used as the state mutex during transitions.
- ``<path>.write`` is the writer marker; its presence blocks readers and other writers.
- ``<path>.readers/<host>.<pid>.<uuid>`` is one marker file per active reader.

Each marker stores a random 128-bit token, the holder's pid, and the holder's hostname. Every acquire uses
``O_CREAT | O_EXCL | O_NOFOLLOW`` with mode ``0o600``; the readers directory uses mode ``0o700`` and a ``lstat``
Expand All @@ -429,7 +429,7 @@ Cross-host stale detection
==========================

On a multi-node cluster, a process on ``node-42`` that crashes while holding a lock cannot be detected via
``kill(pid, 0)`` from ``node-17`` — the pid means nothing to a different kernel. ``SoftReadWriteLock`` therefore
``kill(pid, 0)`` from ``node-17``; the pid means nothing to a different kernel. ``SoftReadWriteLock`` therefore
uses a **TTL with a heartbeat** rather than ``SoftFileLock``'s PID-alive check:

- Each lock instance starts a daemon thread on acquire. The thread refreshes the marker's ``mtime`` every
Expand All @@ -443,7 +443,7 @@ uses a **TTL with a heartbeat** rather than ``SoftFileLock``'s PID-alive check:
never gets accidentally refreshed.

The trade-off: ``stale_threshold`` must be larger than any realistic pause a holder might hit (GC, syscall delay,
NFS hiccup). Pick it generously. Clock synchronization across compute nodes is assumed — every HPC cluster runs
NFS hiccup). Pick it generously. We assume clock synchronization across compute nodes; every HPC cluster runs
NTP or chrony, so this is not an additional constraint in the target environment.

Fork semantics
Expand Down
34 changes: 17 additions & 17 deletions docs/how-to.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ You can also pass ``timeout`` directly to ``acquire()``:
Use non-blocking locks
************************

Sometimes you want to attempt the lock exactly once—either you get it immediately or you don't.
Sometimes you want to try the lock once. Either you get it immediately or you don't.

Set ``blocking=False``:

Expand All @@ -60,7 +60,7 @@ Set ``blocking=False``:

When ``blocking=False``, the lock makes only one attempt and raises ``Timeout`` if it can't acquire immediately.

The ``blocking`` parameter takes precedence over ``timeout``\ —if you set both, ``blocking`` wins:
The ``blocking`` parameter takes precedence over ``timeout``. If you set both, ``blocking`` wins:

.. code-block:: python

Expand Down Expand Up @@ -121,7 +121,7 @@ For async code, use the async variants with ``async with``:

.. warning::

``with`` does not work on async locks — ``acquire`` and ``release`` are coroutines and must be awaited.
``with`` does not work on async locks. ``acquire`` and ``release`` are coroutines; await them.
Use ``async with`` as shown above.

By default, async locks run blocking I/O in a thread pool. You can customize this:
Expand All @@ -146,12 +146,12 @@ You can also pass a specific event loop:
loop = asyncio.new_event_loop()
lock = AsyncFileLock("work.lock", loop=loop)

Note that async locks default to ``thread_local=False`` (unlike sync locks which default to ``True``) because the
Async locks default to ``thread_local=False`` (unlike sync locks which default to ``True``) because the
acquiring and releasing threads may differ when using an executor.

Available async lock classes:

- :class:`AsyncFileLock <filelock.AsyncFileLock>` — platform-aware (recommended).
- :class:`AsyncFileLock <filelock.AsyncFileLock>`, platform-aware (recommended).
- :class:`AsyncSoftFileLock <filelock.AsyncSoftFileLock>`.
- :class:`AsyncUnixFileLock <filelock.AsyncUnixFileLock>`.
- :class:`AsyncWindowsFileLock <filelock.AsyncWindowsFileLock>`.
Expand Down Expand Up @@ -348,12 +348,12 @@ preemption). ``heartbeat_interval`` should be roughly ``stale_threshold / 3``; t
its ``LeaseKeepAlive``. Lower ``poll_interval`` reduces acquire latency under contention at the cost of more
NFS ``stat`` calls per waiting client.

Writer acquisition is two-phase and writer-preferring: phase one claims the writer marker (which immediately
blocks any new reader), phase two waits for existing readers to drain. This rules out writer starvation even
under a read-heavy workload like the 99/1 reader-to-writer mix typical of slurm job queues.
Writer acquisition is two-phase and writer-preferring: phase one claims the writer marker (which blocks any
new reader), phase two waits for existing readers to drain. This rules out writer starvation even under a
read-heavy workload like the 99/1 reader-to-writer mix typical of slurm job queues.

**Fork caveat.** A process that forks while holding a ``SoftReadWriteLock`` loses the lock in the child. The
inherited instance is marked fork-invalidated; ``release()`` on it becomes a no-op, and the child must call
**Fork caveat.** A process that forks while holding a ``SoftReadWriteLock`` loses the lock in the child. filelock marks the
inherited instance fork-invalidated; ``release()`` on it becomes a no-op, and the child must call
``SoftReadWriteLock(path)`` again to get a fresh instance before acquiring. Matches the semantics of
:class:`threading.Lock` and PyMongo's connection pools.

Expand Down Expand Up @@ -402,7 +402,7 @@ Low-level ``acquire_read``/``acquire_write``/``release`` methods are also availa
await rw.release()

The same reentrancy and upgrade/downgrade rules as the synchronous :class:`ReadWriteLock <filelock.ReadWriteLock>`
apply — see :ref:`how-to:Use shared read / exclusive write locks` for details.
apply. See :ref:`how-to:Use shared read / exclusive write locks` for details.

For network filesystems, use :class:`AsyncSoftReadWriteLock <filelock.AsyncSoftReadWriteLock>`, which wraps
:class:`SoftReadWriteLock <filelock.SoftReadWriteLock>` the same way:
Expand All @@ -421,9 +421,9 @@ For network filesystems, use :class:`AsyncSoftReadWriteLock <filelock.AsyncSoftR
**************************************

:class:`SoftFileLock <filelock.SoftFileLock>` stores the PID and hostname of the lock holder. It can detect when the
holding process has died and automatically break stale locks on all platforms.
holding process has died and break stale locks on all platforms.

This happens automatically—you don't need to do anything special:
This happens automatically. You don't need to do anything special:

.. code-block:: python

Expand All @@ -438,8 +438,8 @@ This happens automatically—you don't need to do anything special:

Stale lock detection only detects locks from the same host. Cross-host stale locks still require manual removal.

On Windows, the lock file additionally stores the process creation time to guard against PID recycling. Malformed lock
files (empty or corrupted) are evicted automatically after a brief safety window.
On Windows, the lock file additionally stores the process creation time to guard against PID recycling. filelock evicts
malformed lock files (empty or corrupted) after a brief safety window.

*****************************************
Inspect and manage PID locks
Expand Down Expand Up @@ -505,7 +505,7 @@ library:
Set lock lifetime
*******************

You can create locks that automatically expire after a certain time:
You can create locks that expire after a set time:

.. code-block:: python

Expand All @@ -519,7 +519,7 @@ You can create locks that automatically expire after a certain time:
pass

This is useful for distributed systems where a process might crash and leave a lock behind. After the lifetime expires,
other processes can acquire it automatically.
other processes can acquire it.

**************************
Cancel lock acquisition
Expand Down
34 changes: 17 additions & 17 deletions docs/tutorials.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
Tutorials
###########

This section guides you through the fundamentals of file locking. We'll learn by doing, starting with the basics and
building up to more advanced patterns.
This section guides you through the fundamentals of file locking, starting with the basics and building up to advanced
patterns.

*****************
Your first lock
*****************

Let's create our first lock and use it to coordinate between processes.

First, we'll import what we need and create a lock object:
We import what we need and create a lock object:

.. code-block:: python

Expand All @@ -30,14 +30,14 @@ statement):
print("I have the lock!")
# Outside this block, the lock is released

Run this code multiple times in different terminal windows at the same time. You'll see that only one process prints the
message at a time—the others wait for their turn. The lock is working correctly!
Run this code in several terminal windows at the same time. Only one process prints the message at a time; the others
wait for their turn.

************************
Protecting shared data
************************

File locks are most useful when protecting data that multiple processes access. Let's see how:
File locks are most useful when protecting data that multiple processes access:

.. code-block:: python

Expand All @@ -57,9 +57,9 @@ File locks are most useful when protecting data that multiple processes access.
with data_file.open("a") as f:
f.write("Hello from Process B\n")

The key pattern here: **Before making changes, check what's already done.** Process A checks if the file exists before
writing. Process B doesn't need to check because it's just appending. But both use the lock to ensure only one process
modifies the file at a time.
The key pattern: **before making changes, check what's already done.** Process A checks whether the file exists before
writing. Process B appends, so it skips the check. Both use the lock so that only one process modifies the file at a
time.

Run this code from two different processes. The file will contain messages from both in a consistent order.

Expand All @@ -86,8 +86,8 @@ Sometimes you need to acquire the same lock multiple times from the same process
helper_function() # Can acquire the same lock again
print("Still have the lock")

No deadlock occurs—the lock counts how many times it's been acquired and releases only when the count reaches zero. You
can inspect this counter and the lock state at any time:
No deadlock occurs. The lock counts how many times you acquire it and releases only when the count reaches zero. You can
inspect this counter and the lock state at any time:

.. code-block:: python

Expand All @@ -111,7 +111,7 @@ can inspect this counter and the lock state at any time:
print(lock.lock_counter) # 0 — fully released
print(lock.is_locked) # False

Key lesson: You can safely call functions that acquire a lock, even if you already hold the lock.
You can call functions that acquire a lock even while you already hold it.

*****************************
Multiple ways to use a lock
Expand Down Expand Up @@ -142,7 +142,7 @@ Always use a ``try/finally`` block to guarantee the lock is released, even if an

protected_operation() # Lock is acquired, function runs, lock is released

Choose whichever feels most natural for your code. The ``with`` statement is usually clearest.
Choose whichever feels most natural for your code. The ``with`` statement reads clearest.

Important: Always use the context manager
=========================================
Expand Down Expand Up @@ -171,8 +171,8 @@ Instead, always keep a reference to the lock object:
Thread-local by default
**************************

By default, each lock uses thread-local state (``thread_local=True``). This means each thread tracks its own lock
counter independently. Two threads holding the same ``FileLock`` object each get their own reentrant state.
By default, each lock uses thread-local state (``thread_local=True``): each thread tracks its own lock counter. Two
threads holding the same ``FileLock`` object each get their own reentrant state.

Async locks default to ``thread_local=False`` because they run in a thread pool where the acquiring thread may differ
from the releasing thread.
Expand Down Expand Up @@ -218,8 +218,8 @@ Key differences from ``PIDLockFile``:
Reader/writer locks on a shared NFS
*************************************

If your lock file lives on a network filesystem — a slurm-mounted home directory, a Lustre cluster scratch
space, or any NFS share — use :class:`SoftReadWriteLock <filelock.SoftReadWriteLock>` rather than
If your lock file lives on a network filesystem (a slurm-mounted home directory, a Lustre cluster scratch space, or any
NFS share), use :class:`SoftReadWriteLock <filelock.SoftReadWriteLock>` rather than
:class:`ReadWriteLock <filelock.ReadWriteLock>`. ``ReadWriteLock`` is SQLite-backed and unsafe on NFS.
``SoftReadWriteLock`` is built on :class:`SoftFileLock <filelock.SoftFileLock>` primitives and handles cross-host stale detection via a
background heartbeat thread.
Expand Down
7 changes: 3 additions & 4 deletions src/filelock/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

import sys
import warnings
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Final

from ._api import AcquireReturnProxy, BaseFileLock
from ._error import Timeout
Expand Down Expand Up @@ -44,7 +44,7 @@
from .version import version

#: version of the project as a string
__version__: str = version
__version__: Final[str] = version


if sys.platform == "win32": # pragma: win32 cover
Expand All @@ -57,8 +57,7 @@
else:
_FileLock = SoftFileLock
_AsyncFileLock = AsyncSoftFileLock
if warnings is not None:
warnings.warn("only soft file lock is available", stacklevel=2)
warnings.warn("only soft file lock is available", stacklevel=2)

if TYPE_CHECKING:
FileLock = SoftFileLock
Expand Down
Loading