Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FFM-12192 Use client Singleton / Lock eval & target map clone operation #48

Merged
merged 3 commits into from
Nov 12, 2024

Conversation

erdirowlands
Copy link
Contributor

@erdirowlands erdirowlands commented Nov 12, 2024

What

  • We are seeing segment faults that are closely linked to the evaluation metrics map.
  • Customer shared logs to confirm the evaluation metrics map has stored corrupt data.

Theory

Theory is when we clone the evaluation metrics map, the evaluation thread can and will modify the original map concurrently, resulting in data corruption. Theory extends to the Segment fault being caused by this access too.

Also, the cf_client uses a class variable which is not thread safe, which could be causing issues.

Fix

Lock the cloning operation with a mutex
Replace the client class variable with a Singleton

Testing

  • Unit tests for singleton access
  • Testgrid

@erdirowlands erdirowlands merged commit a1a708a into main Nov 12, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants