You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rather than download the redwood urls we should generate signed URLs and allow the pipeline to download them. This is necessary for autoscaling dockers.
To do this, I plan on modifying DockstoreRunner.download_and_transform_json to look at an argument along the lines of --generate_signed_urls. If that argument is passed, we can map the redwood url to an s3 url and use boto's generate_url function to create our signed url.
The text was updated successfully, but these errors were encountered:
Rather than download the redwood urls we should generate signed URLs and allow the pipeline to download them. This is necessary for autoscaling dockers.
To do this, I plan on modifying
DockstoreRunner.download_and_transform_json
to look at an argument along the lines of--generate_signed_urls
. If that argument is passed, we can map the redwood url to an s3 url and use boto'sgenerate_url
function to create our signed url.The text was updated successfully, but these errors were encountered: