[Alpha pipeline] Add image retrieval component #110

NielsRogge · 2023-05-10T08:20:20Z

This PR adds the image retrieval component, which retrieves image URLs using clip-retrieval from LAION-5B.

Fixes #95

NielsRogge · 2023-05-10T13:05:21Z

examples/pipelines/controlnet-interior-design/components/generate_prompts/src/main.py

+        # TODO remove, just use a tiny df for testing purposes
+        data = {
+            "prompts_text": [
+                "comfortable bathroom, art deco interior design",
+                "comfortable bathroom, bauhaus interior design",
+            ]
+        }
+        pandas_df = pd.DataFrame.from_dict(data)
+        df = dd.from_pandas(pandas_df, npartitions=1)
+        # end of TODO


Would keep this as long as we're building the pipeline. Would remove once the pipeline is finished

PhilippeMoussalli · 2023-05-10T14:04:14Z

examples/pipelines/controlnet-interior-design/components/generate_prompts/requirements.txt

@@ -1,2 +1,2 @@
-git+https://github.com/ml6team/fondant.git
+git+https://github.com/ml6team/fondant.git@2abeabe07412266b78e3b2a4055e0b10d62168cb


I'm guilty of this myself but don't forget to remove this

PhilippeMoussalli

Thanks Niels :) Looks good! I left a few comments.
How did you manage to resolve the issues with slow retrieval eventually?

PhilippeMoussalli · 2023-05-10T14:06:06Z

fondant/data_io.py

@@ -140,7 +141,9 @@ def write_index(self, df: dd.DataFrame):
        )

        # Write index
-        dd.compute(upload_index_task)
+        with ProgressBar():


Nice, something similar to tqdm with an ETA? does it show up nicely in the kfp logging?

It looks like this:

It's not ideal as it seems to replicate the progress bars across cores? Not sure. Do we know how many cores the machines on Kubeflow have?

Will depend on the machine defined for the component, but often quite a few I think. And also multiple machines in the future.

PhilippeMoussalli · 2023-05-10T14:07:31Z

examples/pipelines/controlnet-interior-design/components/retrieve_images/requirements.txt

@@ -0,0 +1,2 @@
+git+https://github.com/ml6team/fondant.git@2abeabe07412266b78e3b2a4055e0b10d62168cb


examples/pipelines/controlnet-interior-design/components/retrieve_images/fondant_component.yaml

examples/pipelines/controlnet-interior-design/components/retrieve_images/src/main.py

examples/pipelines/controlnet-interior-design/config/components_config.py

examples/pipelines/controlnet-interior-design/components/retrieve_images/src/main.py

RobbeSneyders

Thanks @NielsRogge!

examples/pipelines/controlnet-interior-design/components/retrieve_images/Dockerfile

examples/pipelines/controlnet-interior-design/components/retrieve_images/src/clip_client.py

fondant/data_io.py

examples/pipelines/controlnet-interior-design/components/retrieve_images/src/main.py

examples/pipelines/controlnet-interior-design/components/retrieve_images/fondant_component.yaml

RobbeSneyders · 2023-05-10T16:08:03Z

fondant/data_io.py

@@ -140,7 +141,9 @@ def write_index(self, df: dd.DataFrame):
        )

        # Write index
-        dd.compute(upload_index_task)
+        with ProgressBar():


Will depend on the machine defined for the component, but often quite a few I think. And also multiple machines in the future.

RobbeSneyders · 2023-05-10T16:11:34Z

As discussed before, I would make these reusable components available in a components directory directly under the root directory. We can merge this one as is though, as we will have to provide a mechanism to easily load the reusable components. We can implement this using this component as an example in a separate PR.

RobbeSneyders · 2023-05-11T10:06:58Z

examples/pipelines/controlnet-interior-design/components/generate_prompts/src/main.py

+        # end of TODO
+
+        # add id and source columns
+        df["id"] = df.assign(id=1).id.cumsum()


We should use the LAION ids here.

Can I leave this for a follow-up PR? Would like to first get all components up and running.

There's no real benefit of having the LAION id's over integer indices that go from 0 to length of the dataset for this pipeline

This PR adds the image retrieval component, which retrieves image URLs using clip-retrieval from LAION-5B. Fixes #95 --------- Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

Niels Rogge added 7 commits May 10, 2023 10:17

First draft

f2e5f0a

More improvements

a774af1

More improvements

a88f47f

More fixes

7110601

Use tiny df for now

2abeabe

More improvements

9658756

Improve code

51923d9

NielsRogge requested review from RobbeSneyders and PhilippeMoussalli May 10, 2023 13:04

NielsRogge commented May 10, 2023

View reviewed changes

PhilippeMoussalli reviewed May 10, 2023

View reviewed changes

NielsRogge requested a review from j-branigan May 10, 2023 14:46

PhilippeMoussalli reviewed May 10, 2023

View reviewed changes

j-branigan reviewed May 10, 2023

View reviewed changes

examples/pipelines/controlnet-interior-design/components/retrieve_images/src/main.py Outdated Show resolved Hide resolved

Renmae to laion_retrieval

c37584b

RobbeSneyders reviewed May 10, 2023

View reviewed changes

Niels Rogge added 6 commits May 11, 2023 10:27

Address comments

cc8c897

Clarify aesthetic embedding

f5b2bd9

Remove map_paritions

4b7f53f

Update source

8c1443d

More improvements

d77d580

Add LAION ids

00e7006

RobbeSneyders reviewed May 11, 2023

View reviewed changes

More improvements

745f703

PhilippeMoussalli approved these changes May 11, 2023

View reviewed changes

NielsRogge merged commit 0b3c267 into main May 11, 2023

RobbeSneyders deleted the add_image_retrieval branch May 15, 2023 16:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Alpha pipeline] Add image retrieval component #110

[Alpha pipeline] Add image retrieval component #110

NielsRogge commented May 10, 2023 •

edited

Loading

NielsRogge May 10, 2023

PhilippeMoussalli May 10, 2023

PhilippeMoussalli left a comment

PhilippeMoussalli May 10, 2023

NielsRogge May 10, 2023

RobbeSneyders May 10, 2023

PhilippeMoussalli May 10, 2023

RobbeSneyders left a comment

RobbeSneyders May 10, 2023

RobbeSneyders commented May 10, 2023

RobbeSneyders May 11, 2023

NielsRogge May 11, 2023 •

edited

Loading

		@@ -1,2 +1,2 @@
		git+https://github.com/ml6team/fondant.git
		git+https://github.com/ml6team/fondant.git@2abeabe07412266b78e3b2a4055e0b10d62168cb

		@@ -0,0 +1,2 @@
		git+https://github.com/ml6team/fondant.git@2abeabe07412266b78e3b2a4055e0b10d62168cb

[Alpha pipeline] Add image retrieval component #110

[Alpha pipeline] Add image retrieval component #110

Conversation

NielsRogge commented May 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhilippeMoussalli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RobbeSneyders left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RobbeSneyders commented May 10, 2023

Choose a reason for hiding this comment

NielsRogge May 11, 2023 • edited Loading

Choose a reason for hiding this comment

NielsRogge commented May 10, 2023 •

edited

Loading

NielsRogge May 11, 2023 •

edited

Loading