Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rewrite shared core to allow for concurrency of wasm core #165

Closed
wants to merge 1 commit into from

Conversation

dschaller
Copy link

With the merging of extism/go-sdk#81 it's now possible to have more than one instance of the wasm module. This change modifies the behavior of the shared core to take advantage of this by creating a compiled plugin and spawning off instances of the core when necessary.

It's important to note that this will only help if clients perform the full lifecycle (new client, {action}, release) in parallel as each instance of the wasm module is isolated which means that the credentials cannot be shared across these various instances for different actions.

This change also remains backwards compatible in that if clients only have a single client multiple actions called they will still operate as they currently do in a serial manner, locking to be sure there is only one action accessing the shared wasm core at once.

Testing the resolve function call as the action on a client that resolves 10 references improved performance by ~50% from ~6s to ~3 seconds

Copy link

⚠️ This PR contains unsigned commits. To get your PR merged, please sign those commits (git rebase --exec 'git commit -S --amend --no-edit -n' @{upstream}) and force push them to this branch (git push --force-with-lease).

If you're new to commit signing, there are different ways to set it up:

Sign commits with gpg

Follow the steps below to set up commit signing with gpg:

  1. Generate a GPG key
  2. Add the GPG key to your GitHub account
  3. Configure git to use your GPG key for commit signing
Sign commits with ssh-agent

Follow the steps below to set up commit signing with ssh-agent:

  1. Generate an SSH key and add it to ssh-agent
  2. Add the SSH key to your GitHub account
  3. Configure git to use your SSH key for commit signing
Sign commits with 1Password

You can also sign commits using 1Password, which lets you sign commits with biometrics without the signing key leaving the local 1Password process.

Learn how to use 1Password to sign your commits.

Watch the demo

@dschaller dschaller force-pushed the dschaller/concurrency branch from a615796 to fdca7dc Compare December 10, 2024 19:04
@dschaller dschaller force-pushed the dschaller/concurrency branch from fdca7dc to 936f793 Compare December 10, 2024 19:07
Comment on lines +120 to +121
c.lock.Lock()
defer c.lock.Unlock()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from a practical standpoint this is probably unnecessary and I can drop since callWithCtx will always spawn a new instance. Kept it here for some degree of safety for unknown unknowns but if we are ok dropping it would be my preference

@AndyTitu
Copy link
Contributor

@dschaller As discussed with the team, here's a (beta) release that allows you to resolve multiple secret references at once: https://github.com/1Password/onepassword-sdk-go/releases/tag/v0.1.7-beta.1. This feature might address your scalability issues. Let us know how this works for you.

@dschaller dschaller closed this Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants