Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking concurrency limits OR step/function confusion #179

Open
01programs opened this issue Oct 23, 2024 · 1 comment
Open

Breaking concurrency limits OR step/function confusion #179

01programs opened this issue Oct 23, 2024 · 1 comment

Comments

@01programs
Copy link

I was following the "Step parallelism in Python" in this doc and debugged for hours to find out why function that has a concurrency limit of 2 has 8+ instances running at a time.
Turn out I made two mistakes in one place. I called a function as a step and I wrote "step.invoke" instead of "step.run". This should either not be possible or respect the function parameters, no?

offending piece of code:

    async def enqueue_jobs(emails, parent_job):
        jobs: list[function] = []
        for email, domain in emails:
            logging.debug(f"Email: {email}")
            jobs.append(
                lambda domain=domain: step.invoke("process_csv_row",
                                                  function=process_csv_row,
                                                  data={
                                                      "domain": domain,
                                                      "parent_job": parent_job
                                                  }))
        await step.parallel(jobs)
@amh4r
Copy link
Collaborator

amh4r commented Oct 30, 2024

Could you reply with both full Inngest functions? The problem is a little unclear with this code snippet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants