Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FlyteClient][FlyteDeck] Get Downloaded Artifact Signed URL via Data Proxy #2777

Merged

Conversation

Future-Outlier
Copy link
Member

@Future-Outlier Future-Outlier commented Sep 30, 2024

Why are the changes needed?

This is a new API for Flyte to get the signed URL for the deck artifact.

What changes were proposed in this pull request?

add a method called get_download_deck_signed_url using the IDL CreateDownloadLinkRequest and CreateDownloadLinkResponse.

How was this patch tested?

  1. flyte sandbox
from flytekit.clients.friendly import SynchronousFlyteClient as _SynchronousFlyteClient
from flytekit.configuration import PlatformConfig, AuthType


client = _SynchronousFlyteClient(PlatformConfig.for_endpoint("localhost:30080", True))
signed_url = client.get_download_deck_signed_url(
    node_id="n1",
    project="flytesnacks",
    domain="development",
    execution_id="awzmsq9jbjfz8p6jksdg",
)
print(signed_url.signed_url)

Screenshots

  • sandbox
image image

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
@Future-Outlier Future-Outlier changed the title [FlyteDeck] add get_download_deck_signed_url for FlyteClient [FlyteClient][FlyteDeck] add get_download_deck_signed_url Sep 30, 2024
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] add get_download_deck_signed_url [FlyteClient][FlyteDeck] get_download_deck_signed_url Sep 30, 2024
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] get_download_deck_signed_url [FlyteClient][FlyteDeck] get_download_deck_signed_url by interaciting with Data Proxy Sep 30, 2024
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] get_download_deck_signed_url by interaciting with Data Proxy [FlyteClient][FlyteDeck] Get Downloaded Deck Sign URL by interaciting with Data Proxy Sep 30, 2024
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] Get Downloaded Deck Sign URL by interaciting with Data Proxy [FlyteClient][FlyteDeck] Get Downloaded Deck Sign URL by accessing via Data Proxy Sep 30, 2024
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] Get Downloaded Deck Sign URL by accessing via Data Proxy [FlyteClient][FlyteDeck] Get Downloaded Deck Signed URL by accessing via Data Proxy Sep 30, 2024
@Future-Outlier Future-Outlier enabled auto-merge (squash) September 30, 2024 14:33
@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] Get Downloaded Deck Signed URL by accessing via Data Proxy [FlyteClient][FlyteDeck] Get Downloaded Deck Signed URL via Data Proxy Sep 30, 2024
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
@Future-Outlier Future-Outlier enabled auto-merge (squash) October 1, 2024 07:40
flytekit/clients/friendly.py Outdated Show resolved Hide resolved
flytekit/clients/friendly.py Outdated Show resolved Hide resolved
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Co-authored-by: pingsutw  <pingsutw@apache.org>
@@ -1046,3 +1048,33 @@ def get_data(self, flyte_uri: str) -> _data_proxy_pb2.GetDataResponse:

resp = self._dataproxy_stub.GetData(req, metadata=self._metadata)
return resp

def get_download_deck_signed_url(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we need to change the function name to get_download_signed_url and add ArtifactType to the input args?
cc @wild-endeavor @eapolinario

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. What if we set the default artifact_type to ARTIFACT_TYPE_DECK?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I can do that

Comment on lines 1053 to 1062
self,
node_id: str,
project: str,
domain: str,
name: str,
expires_in: datetime.timedelta = None,
) -> _data_proxy_pb2.CreateDownloadLinkResponse:
"""
This is a new API for flyte and union cluster to get the signed url for the deck artifact.
"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self,
node_id: str,
project: str,
domain: str,
name: str,
expires_in: datetime.timedelta = None,
) -> _data_proxy_pb2.CreateDownloadLinkResponse:
"""
This is a new API for flyte and union cluster to get the signed url for the deck artifact.
"""
self,
node_id: str,
project: str,
domain: str,
name: str,
artifact_type: _data_proxy_pb2.ArtifactType = ARTIFACT_TYPE_DECK,
expires_in: datetime.timedelta = None,
) -> _data_proxy_pb2.CreateDownloadLinkResponse:
"""
Get a signed url for an artifact.
:param node_id: Node id associated with artifact
:param project: Name of the project the resource belongs to
:param domain: Name of the domain the resource belongs to
:param name: User or system provided value for the resource
:param artifact_type: ArtifactType of the artifact requested
:param expires_in: If provided this defines a requested expiration duration for the generated url
:rtype: flyteidl.service.dataproxy_pb2.CreateDownloadLinkResponse
"""

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Co-authored-by: Eduardo Apolinario  <eapolinario@users.noreply.github.com>
@Future-Outlier
Copy link
Member Author

Update, please review it for the final check, thank you all!

cc @pingsutw @eapolinario

@Future-Outlier Future-Outlier changed the title [FlyteClient][FlyteDeck] Get Downloaded Deck Signed URL via Data Proxy [FlyteClient][FlyteDeck] Get Downloaded Artifact Signed URL via Data Proxy Oct 3, 2024
@Future-Outlier Future-Outlier merged commit f923f77 into master Oct 4, 2024
104 checks passed
otarabai pushed a commit to otarabai/flytekit that referenced this pull request Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants