Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ToolsAPI: add SHM clean up on session end #404

Open
AnarManafov opened this issue Nov 19, 2021 · 2 comments
Open

ToolsAPI: add SHM clean up on session end #404

AnarManafov opened this issue Nov 19, 2021 · 2 comments
Assignees
Milestone

Comments

@AnarManafov
Copy link
Contributor

Extend Tools API with a possibility to register shared memory names. Those SHM segments must be removed, if exists, on DDS session shutdown, i.e. on the agent's clean up.

@AnarManafov AnarManafov self-assigned this Nov 19, 2021
@AnarManafov AnarManafov added this to the 3.6 milestone Nov 19, 2021
@rbx
Copy link
Member

rbx commented Dec 5, 2021

In case of FairMQ it is not as simple as giving a list of names. We use a number of different boost shared memory objects of different types.

Current list is:

name type
fmq_<shmId>_m_<segmentId> boost::interprocess::shared_memory_object
fmq_<shmId>_mng boost::interprocess::shared_memory_object
fmq_<shmId>_mtx boost::interprocess::named_mutex
fmq_<shmId>_cv boost::interprocess::named_condition
fmq_<shmId>_rg_<index> boost::interprocess::shared_memory_object OR boost::interprocess::file_mapping (user configured)
fmq_<shmId>_rgq_<index> boost::interprocess::message_queue
fmq_<shmId>_ms boost::interprocess::named_mutex

(The shmId is derived from the current FairMQ session id.)

Each of these types have their own removal methods according to boost docs, for example shared_memory_object is removed via shared_memory_object::remove(), but named_mutex is removed named_mutex::remove(). It may well be that the underlying implementation is the same for all of them, but I would prefer to use correct methods, to be robust against changes in future boost versions.

I tested if all of the above can be removed simply via boost::interprocess::shared_memory_object::remove(). And it works for all except the mutexes, possibly simply because boost prefixes the mutex files with sem. - sem.fmq_<shmId>_mtx.

Alternatively, do not call any boost API to remove these objects at all (unless you know which type is which somehow...), and simply rm -f them. However, here special knowledge would be necessary to know that boost prefixes some of these files, like in mutex case.

Another point is - where exactly should this registration happen? FairMQ core has no dependency on DDS Tools API, and this registration certainly should not always be done, but only in cases where it is desired. One option would be to make it a configurable setting in the ODC plugin. FairMQ core shmem could provide a static list of names, ODC plugin would read this list and give it to DDS and DDS would then call rm for each.

In my opinion the cleanest way would be to call fairmq-shmmonitor --cleanup --session <sessionId>, which knows what methods to call for which objects, instead of providing a list of names or unknown type. To remain robust, you can treat it as a user task and force kill it if it misbehaves.

Whatever the solution will be, it must be flexible enough that a new FairMQ version can change the names, types and the number of these files if it needs to, without requiring update anywhere else.

@rbx
Copy link
Member

rbx commented Dec 5, 2021

Alternatively, do not call any boost API to remove these objects at all (unless you know which type is which somehow...), and simply rm -f them. However, here special knowledge would be necessary to know that boost prefixes some of these files, like in mutex case.

Actually a static list will not work, because the sessionid/shmid us dynamic, but also the number of segments and regions and their IDs will vary depending on runtime configuration.

A regex/wildcards would do the trick, something like rm -f /dev/shm/*fmq_<shmid>*.

@AnarManafov AnarManafov modified the milestones: 3.6, 3.8 Jan 11, 2022
@AnarManafov AnarManafov modified the milestones: 3.8, 3.9 Jan 12, 2024
@AnarManafov AnarManafov modified the milestones: 3.9, 3.10, 3.11 Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants