score/memory/shared: Redesign proposal by crimson11 · Pull Request #1387 · eclipse-score/communication

crimson11 · 2026-10-14T16:47:35Z

Added proposal for redesign of memory/shared to
provide a much more flexible design for various
shared-memory types.

bemerybmw · 2026-10-15T06:15:56Z

+
+### The fix
+
+We set `PTHREAD_MUTEX_ROBUST` on the mutex. The kernel tracks who holds it. If that process dies, the next `pthread_mutex_lock()` returns `EOWNERDEAD` instead of deadlocking or failing.


So this is a general purpose fix to the interprocess mutex fix? Is pthread_mutex_consistent atomic? What happens if multiple processes try to recover the mutex at the same time?

bemerybmw · 2026-10-15T06:16:46Z

+
+`SharedMemoryResource` owns and manages the ControlBlock. During `SharedMemoryFactory::Create()`, it placement-news the ControlBlock at the region's base address. The usable memory for user allocations starts at `base + ControlBlock::TotalSize()`.
+
+The proxy id is not stored in the ControlBlock. It's stored by value in the `PolymorphicOffsetPtrAllocator` — same 8 bytes as an OffsetPtr, but avoids a pointer indirection on the hot path.


Need to explain what the proxy id actually is. i.e. that it's the identifier of the resource in the registry.

and it should come from (see above) MappedRegion::GetUniqueId()

bemerybmw · 2026-10-15T06:25:20Z

+
+### The problem
+
+The `InterprocessMutex` in the ControlBlock is `PTHREAD_PROCESS_SHARED` but not `PTHREAD_MUTEX_ROBUST`. If a QM process crashes while holding the mutex, the next caller gets an error from `pthread_mutex_lock()`, and our `InterprocessMutex::lock()` calls `std::terminate()`. If that next caller is ASIL-B, we just crashed a safety-critical process because some QM process died. That's a freedom-from-interference violation.


For a monotonic allocator, it can also be lock free, why do we need this? Just in case we ever add further allocation algorithms in the future?

I guess so! And I think we need dynamic memory allocation soon :) - then we need this "locking"

BTW: Funny thing - we removed the InterprocessMutex from the control block just recently afaik (and switched to lock free) - it was never an obvious issue as we only have ONE/single process, who allocates! Always the Skeleton side allocates! ... I guess we had maybe ones an issue in partial restart ... where app crashed and re-using the InterprocessMutex failed as we crashed while holding the mutex ... this is a 1 in a million case :)

bemerybmw · 2026-10-15T06:34:18Z

+
+`SharedMemoryResource` owns and manages the ControlBlock. During `SharedMemoryFactory::Create()`, it placement-news the ControlBlock at the region's base address. The usable memory for user allocations starts at `base + ControlBlock::TotalSize()`.
+
+The proxy id is not stored in the ControlBlock. It's stored by value in the `PolymorphicOffsetPtrAllocator` — same 8 bytes as an OffsetPtr, but avoids a pointer indirection on the hot path.


It's also not currently stored in the ControlBlock?

The MemoryResourceProxy is currently stored in the ControlBlock!

I think we have to re-design/split the ControlBlock! We will have parts of the mControlBlock, which are allocation-strategy specific!

bemerybmw · 2026-10-15T06:44:51Z

+
+## Region Lifecycle
+
+**Creation**: `MemoryRegionFactory::CreateNamed(path, size, perms)` does `shm_open` + `ftruncate` + `mmap`. Returns `shared_ptr<MappedRegion>`. Dedup is handled internally — same path returns the existing region.


We're missing information about what region maps are stored and where. E.g. Currently we have a map of weak_ptrs to resources in SharedMemoryFactory and a map of raw pointers to resources in the registry. How does it look in the new design?

bemerybmw · 2026-10-15T08:24:52Z

+
+The redesign splits responsibilities into three singletons:
+
+**MemoryRegionFactory** creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.


We should split this into MemoryRegionFactory and MemoryRegionRegistry. Factory has no state and the registry would store the map of opened resources.

So, MemoryRegionRegistry then becomes a singleton? MemoryRegionFactory needs to do the registration calls in MemoryRegionRegistry. So - I guess MemoryRegionRegistry needs to be a singleton ... because we don't want users to create MemoryRegionRegistry instance(s) and hand them over to create-calls of MemoryRegionFactory?

bemerybmw · 2026-10-15T08:26:15Z

+
+The redesign splits responsibilities into three singletons:
+
+**MemoryRegionFactory** creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.


Can we make the registry NOT a singleton. In mw/com, we could store it in the runtime singleton. The application has some context which may be static or not but they decide and we don't introduce here a new singleton.

See my comment above. For mw::com it would be doable. We would create the MemoryRegionRegistry once and store it within our Runtime/singleton.

But for other users? It seems "brittle". Two application parts, which are loosely coupled/not coupled at all would then have distinct MemoryRegionRegistry instances! With the effect, that they potentially end up mapping the same region twice. Bug or feature?

bemerybmw · 2026-10-15T08:48:03Z

+
+**MemoryRegionFactory** creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.
+
+**SharedMemoryFactory** wraps a region into a PMR-compatible resource. `Create()` takes a `shared_ptr<MappedRegion>`, placement-news a ControlBlock, creates a `SharedMemoryResource`, and registers it in the registry. `Open()` does the same but casts the existing ControlBlock instead of placement-new (the creator already initialized it). Both return `shared_ptr<SharedMemoryResource>`.


We would like that if a user has created a memory region and then hand it over to the SharedMemoryFactory that they don't still have access to the region directly e.g. they made a copy of the shared_ptr. If the MemoryRegionFactory for example returned a unique_ptr then this would make it explicit but this would require doing ref counting manually in the MemoryRegionFactory/Registry.

We could also have different registries (one which destroys regions based on ref semantics and the other which requires explicit cleanup)

This is hard! Because MemoryRegionRegistry (see discussion above) is most likely a singleton! Then we would need two singletons? And then you have two different MemoryRegionFactory - one backed by MemoryRegionRegistry A and one by MemoryRegionRegistry B

bemerybmw · 2026-10-15T08:49:03Z

+**MemoryRegionFactory** creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.
+
+**SharedMemoryFactory** wraps a region into a PMR-compatible resource. `Create()` takes a `shared_ptr<MappedRegion>`, placement-news a ControlBlock, creates a `SharedMemoryResource`, and registers it in the registry. `Open()` does the same but casts the existing ControlBlock instead of placement-new (the creator already initialized it). Both return `shared_ptr<SharedMemoryResource>`.
+


Does a user really need access to the internals of the MemoryRegion? E.g. the shm path. If this is just for print debugging, could we rather expose a print() function, tostring etc.

To the MappedRegion you mean?

bemerybmw · 2026-10-15T08:52:15Z

+
+**SharedMemoryFactory** wraps a region into a PMR-compatible resource. `Create()` takes a `shared_ptr<MappedRegion>`, placement-news a ControlBlock, creates a `SharedMemoryResource`, and registers it in the registry. `Open()` does the same but casts the existing ControlBlock instead of placement-new (the creator already initialized it). Both return `shared_ptr<SharedMemoryResource>`.
+
+**MemoryResourceRegistry** is a pure lookup service. It maps resource ids to `ManagedMemoryResource*` pointers and provides bounds checking for address validation. Called on the hot path (every allocation via proxy), so it's kept separate from the factories for interface segregation.


The SharedMemoryResource holds a shared_ptr to the mem region but doesn't manage any resources itself. So do we really need to hand out shared_ptrs to the resources or we can hand out copies which internally contain shared_ptrs to the same region?

I see the following issue:
If SharedMemoryFactory would hand-out instances of SharedMemoryResource byVal/copy ... then, what does SharedMemoryFactory store via MemoryResourceRegistry::RegisterResource() ?
Still a raw-pointer to a SharedMemoryResource?
Then SharedMemoryFactory would internally store a SharedMemoryResource and return copies of it from the Create calls?
And what does MemoryResourceRegistry::LookupById() return?

I guess the issue is then: When will a ManagedMemoryResource / SharedMemoryResource be "unregistered" from the MemoryResourceRegistry again?

crimson11 · 2026-10-15T10:16:15Z

+}
+```
+
+`SharedMemoryResource` owns and manages the ControlBlock. During `SharedMemoryFactory::Create()`, it placement-news the ControlBlock at the region's base address. The usable memory for user allocations starts at `base + ControlBlock::TotalSize()`.


The issue with MemoryRegionFactory handing out shared_ptr<MappedRegion> and SharedMemoryFactory::Create() taking shared_ptr<MappedRegion>:
You can easily create TWO SharedMemoryResources based on the same MappedRegion ... the 2nd overwriting/destroying the CONTROL-block?
I guess, we started to discuss this already - but this imho makes more sense:

MemoryRegionFactory::CreateXXX() returns std::unique_ptr<MappedRegion>

SharedMemoryFactory::Create() takes a std::unique_ptr<MappedRegion>

MappedRegion needs an interface GetUniqueId()

This id returned by GetUniqueId() will be used by SharedMemoryFactory::Create() to register the SharedMemoryResource in MemoryResourceRegistry::RegisterResource(). If this id already exists, SharedMemoryFactory::Create() fails.

bemerybmw · 2026-10-16T10:11:35Z

+
+The redesign splits responsibilities into three singletons:
+
+**MemoryRegionFactory** creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.


We need to make sure that whether we call Create / Open on the SharedMemoryFactory is the same as what we called on MemoryRegionFactory. Because the SharedMemoryFactory will initialize the shared memory in Create which must be called exactly once per MemoryRegion. If the region was opened via the MemoryRegionFactory but then we call Create on the SharedMemoryFactory then this will double init the shm. Since we don't want to duplicate the constuctors of the MemoryRegionFactory in SharedMemoryFactory, could we use a builder pattern to construct both at once? This would also avoid the user directly interacting with the MemoryRegion pointer returned by MemoryRegionFactory.

Added proposal for redesign of memory/shared to provide a much more flexible design for various shared-memory types.

sequence and added refs to it.

crimson11 requested review from LittleHuba, bemerybmw, castler, hoe-jo and limdor as code owners October 14, 2026 16:47

github-project-automation Bot added this to COM - Communication FT Oct 14, 2026

github-project-automation Bot moved this to Backlog in COM - Communication FT Oct 14, 2026

bemerybmw reviewed Oct 15, 2026

View reviewed changes

crimson11 commented Oct 15, 2026

View reviewed changes

bemerybmw reviewed Oct 16, 2026

View reviewed changes

crimson11 force-pushed the mf_add_lib_memory_redesign_proposal branch from ef1e149 to bb6eda4 Compare October 22, 2026 15:24

crimson11 and others added 7 commits October 22, 2026 17:32

score/memory/shared: Redesign proposal

4d076fa

Added proposal for redesign of memory/shared to provide a much more flexible design for various shared-memory types.

working

df29019

remove singletons and add shared memory factory create seq diagram

c48a73f

working

ca22b6c

working

602c4b0

Introduced PosixNamedMappedRegion creation

4683ead

sequence and added refs to it.

Rework README.md

d28ffe5

crimson11 force-pushed the mf_add_lib_memory_redesign_proposal branch from bb6eda4 to d28ffe5 Compare October 22, 2026 15:37


		### The fix

		We set `PTHREAD_MUTEX_ROBUST` on the mutex. The kernel tracks who holds it. If that process dies, the next `pthread_mutex_lock()` returns `EOWNERDEAD` instead of deadlocking or failing.


		`SharedMemoryResource` owns and manages the ControlBlock. During `SharedMemoryFactory::Create()`, it placement-news the ControlBlock at the region's base address. The usable memory for user allocations starts at `base + ControlBlock::TotalSize()`.

		The proxy id is not stored in the ControlBlock. It's stored by value in the `PolymorphicOffsetPtrAllocator` — same 8 bytes as an OffsetPtr, but avoids a pointer indirection on the hot path.


		### The problem

		The `InterprocessMutex` in the ControlBlock is `PTHREAD_PROCESS_SHARED` but not `PTHREAD_MUTEX_ROBUST`. If a QM process crashes while holding the mutex, the next caller gets an error from `pthread_mutex_lock()`, and our `InterprocessMutex::lock()` calls `std::terminate()`. If that next caller is ASIL-B, we just crashed a safety-critical process because some QM process died. That's a freedom-from-interference violation.


		## Region Lifecycle

		Creation: `MemoryRegionFactory::CreateNamed(path, size, perms)` does `shm_open` + `ftruncate` + `mmap`. Returns `shared_ptr<MappedRegion>`. Dedup is handled internally — same path returns the existing region.


		The redesign splits responsibilities into three singletons:

		MemoryRegionFactory creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.


		MemoryRegionFactory creates and deduplicates memory regions. It has no PMR dependencies — it's a standalone factory that returns `shared_ptr<MappedRegion>`. Direct users (those who place data in SHM without an allocator) call this directly and use the region as-is. PMR users call this first, then pass the region to SharedMemoryFactory.

		SharedMemoryFactory wraps a region into a PMR-compatible resource. `Create()` takes a `shared_ptr<MappedRegion>`, placement-news a ControlBlock, creates a `SharedMemoryResource`, and registers it in the registry. `Open()` does the same but casts the existing ControlBlock instead of placement-new (the creator already initialized it). Both return `shared_ptr<SharedMemoryResource>`.


		SharedMemoryFactory wraps a region into a PMR-compatible resource. `Create()` takes a `shared_ptr<MappedRegion>`, placement-news a ControlBlock, creates a `SharedMemoryResource`, and registers it in the registry. `Open()` does the same but casts the existing ControlBlock instead of placement-new (the creator already initialized it). Both return `shared_ptr<SharedMemoryResource>`.

		MemoryResourceRegistry is a pure lookup service. It maps resource ids to `ManagedMemoryResource*` pointers and provides bounds checking for address validation. Called on the hot path (every allocation via proxy), so it's kept separate from the factories for interface segregation.

Conversation

crimson11 commented Oct 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

crimson11 Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

crimson11 Oct 15, 2026 •

edited

Loading