Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom base images for functions #1297

Merged
merged 14 commits into from
May 7, 2024
Merged

Conversation

IceKhan13
Copy link
Member

@IceKhan13 IceKhan13 commented Apr 25, 2024

Summary

Allows usage of custom images for function execution

Details

User will need to do following steps:

  • implement function template file
  • build docker image with template file and push
  • run function

Function template

# runner.py
class Runner:
    def run(self, arguments: dict) -> dict:
        # this is just an example
        # your function can call for other modules, function, etc.
        return {
            **arguments,
            **{
                "answer": 42
            }
        }

Dockerfile

FROM icr.io/quantum-public/quantum-serverless-ray-node:0.9.0-py39

# install all necessary dependencies for your custom image

# copy our function implementation in `/runner.py` of the docker image
RUN mkdir /runner
COPY ./runner.py /runner/runner.py

Upload funciton with custom image

serverless = ServerlessClient(...)

function_with_custom_image = QiskitFunction(
    title="custom-image-function",
    image="my-custom-function-image:latest"
)

serverless.upload(function_with_custom_image)

@IceKhan13 IceKhan13 marked this pull request as ready for review May 2, 2024 16:04
@@ -0,0 +1,253 @@
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the rest of the procedural notebooks are in docs/getting_started/experimental, whereas the other ones here (VQE / QAOA) are more application-like ... which begs the question of whether we need to think about how to organize our documentation (i.e. should we start making sections for end users, for function providers, etc.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the pending move to the qiskit org probably provides a good cover for a docs reorg ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with you :)
I also was trying to figure out where to put those. I was trying to decide between examples or deployment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally moved it to deployment as rst document.

@@ -0,0 +1,253 @@
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we should also add this to the e2e tests too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created separate issue for that #1316

Not sure yet how to make e2e, since it will not work in local mode and in docker compose as well.

@@ -0,0 +1,7 @@
FROM icr.io/quantum-public/quantum-serverless-ray-node:0.9.0-py39
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth creating a template repo in the Qiskit org to store these files so folks can make use of it when writing their own functions. Also would give us a little insight into some of the third party functions out there in the wild

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion! I like it! Templates in github are very powerful

gateway/api/ray.py Outdated Show resolved Hide resolved
docs/examples/custom_function/Dockerfile Outdated Show resolved Hide resolved
docs/examples/03_building_custom_image.ipynb Outdated Show resolved Hide resolved
Co-authored-by: Paul Schweigert <paul@paulschweigert.com>
Copy link
Member

@Tansito Tansito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much, @IceKhan13 I left a couple of comments but reviewing my comments + @psschwei comments is almost ready to merge 🙏

gateway/api/ray.py Show resolved Hide resolved
gateway/api/ray.py Show resolved Hide resolved
gateway/api/ray.py Outdated Show resolved Hide resolved
gateway/api/serializers.py Show resolved Hide resolved
@Tansito Tansito self-requested a review May 3, 2024 15:05
Copy link
Member

@Tansito Tansito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless @psschwei comments from my part LGTM! 🥳

Thanks @IceKhan13 !!

data={
"title": program.title,
"image": program.image,
"arguments": json.dumps({}),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated question. Is the arguments for the program used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably you are right and it is not used anymore. I will look it up and try to fix it in following PRs

f"{artifact_file_path} is {int(size_in_mb)} Mb, "
f"which is greater than {MAX_ARTIFACT_FILE_SIZE_MB} allowed. "
f"Try to reduce size of `working_dir`."
elif program.entrypoint is not None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it good to put a waring message when both image and entrypoint are set?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to move all validation to QiskitFunction itself to reduce boilderplate code. Created an iissue for that. #1315

Thanks for suggestion!

Copy link
Collaborator

@akihikokuroda akihikokuroda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Thanks!

@IceKhan13 IceKhan13 merged commit 510e1d4 into Qiskit:main May 7, 2024
15 checks passed
@IceKhan13 IceKhan13 deleted the chore/custom-images branch May 7, 2024 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants