Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start Interning instances of SlsVersionMatcher #942

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

iamdanfox
Copy link
Contributor

@iamdanfox iamdanfox commented Apr 19, 2024

Before this PR

I was looking at heap histograms of apollo-catalog and noticed we're storing ~286 megabytes of this one SlsVersionMatcher class.

  12:       7159066      286362640  com.palantir.sls.versions.ImmutableSlsVersionMatcher

Given that I think the vast majority of SlsVersionMatchers come from a small number of constants (e.g. 1.x.x, 2.x.x, 3.x.x) I figured we could eliminate pretty much all of this memory usage by just interning them.

I started looking at the @value.immutables feature of weak interning https://immutables.github.io/immutable.html#instance-interning, but the generated code just used a ConcurrentHashMap under the hood, which made me worry about throughput. (Maybe this is because we have the jdkOnly = true flag on our ImmutablesStyle?)

It did however have a neat optimization for the equals method, where it just checks reference equality (i.e. ==) and uses a separate .isEqualTo method which does the real equals check just for the interner.

After this PR

==COMMIT_MSG==
start Interning instances of SlsVersionMatcher
==COMMIT_MSG==

HOWEVER I've added do not merge because I am a bit worried about the implications of these leading zeros in the numbers. This seems to be arising because we've marked user input string as @Value.Auxiliary (so it doesn't contribute to the equals check) and we're just relying on the other fields to determine equality.

    @Value.Auxiliary
    @Value.Parameter
    public abstract String getValue();

I think we have a few options:

  1. detect these leading zeros and just ban constructing objects like this
  2. write a function to detect these leading zeros and run a migration to check they don't already exist inside apollo-catalog, THEN ban creating them
  3. remove the @Value.Auxiliary annotation from the String getValue() and put it on all the other fields, so that "01.0.0" is genuinely considered not equal to "1.0.0". I think this leads to weird implications because the .compare method would return 0.

Possible downsides?

@changelog-app
Copy link

changelog-app bot commented Apr 19, 2024

Generate changelog in changelog/@unreleased

What do the change types mean?
  • feature: A new feature of the service.
  • improvement: An incremental improvement in the functionality or operation of the service.
  • fix: Remedies the incorrect behaviour of a component of the service in a backwards-compatible way.
  • break: Has the potential to break consumers of this service's API, inclusive of both Palantir services
    and external consumers of the service's API (e.g. customer-written software or integrations).
  • deprecation: Advertises the intention to remove service functionality without any change to the
    operation of the service itself.
  • manualTask: Requires the possibility of manual intervention (running a script, eyeballing configuration,
    performing database surgery, ...) at the time of upgrade for it to succeed.
  • migration: A fully automatic upgrade migration task with no engineer input required.

Note: only one type should be chosen.

How are new versions calculated?
  • ❗The break and manual task changelog types will result in a major release!
  • 🐛 The fix changelog type will result in a minor release in most cases, and a patch release version for patch branches. This behaviour is configurable in autorelease.
  • ✨ All others will result in a minor version release.

Type

  • Feature
  • Improvement
  • Fix
  • Break
  • Deprecation
  • Manual task
  • Migration

Description

start Interning instances of SlsVersionMatcher

Check the box to generate changelog(s)

  • Generate changelog entry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant