Skip to content

Optimize PyCriticalSection2 to skip locking when the requested locks were acquired in the current CS #150902

@dpdani

Description

@dpdani

Feature or enhancement

Proposal:

The RecursionError-raising tests in test_copy are showing a performance bottleneck when recursively acquiring the same PyCriticalSection2 when comparing two lists or dicts. The full suite takes ~50ms on GIL-enabled builds and ~2.5s on free-threading builds.

On main with plain ./configure:

$ ./python.exe -m test -w test_copy
Using random seed: 948858056
0:00:00 load avg: 6.78 mem: 26.9 MiB Run 1 test sequentially in a single process
0:00:00 load avg: 6.78 mem: 26.9 MiB [1/1] test_copy
0:00:00 load avg: 6.78 mem: 43.4 MiB [1/1] test_copy passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 47 ms
Total tests: run=83
Total test files: run=1/1
Result: SUCCESS

On main with ./configure --disable-gil:

$ ./python.exe -m test -w test_copy
Using random seed: 739047776
0:00:00 load avg: 6.07 mem: 32.1 MiB Run 1 test sequentially in a single process
0:00:00 load avg: 6.07 mem: 32.1 MiB [1/1] test_copy
0:00:02 load avg: 6.07 mem: 48.4 MiB [1/1] test_copy passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2.7 sec
Total tests: run=83
Total test files: run=1/1
Result: SUCCESS

We should implement the same optimization for recursive critical sections which is already implemented for the single-mutex PyCriticalSection here. A quick code change showed test_copy completing in ~50ms again.

This became more evident in #149784 where test_copy takes ~1m30s.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions