How to retrieve S3 URLs

Introduction

BaseSpace Sequence Hub (BSSH) files are stored in Amazon S3.

It is possible to retrieve the HTTPS S3 URL of each file. This includes a signature that is valid for 7 days.

We describe how it can be done:

• in any web browser
• using BaseMount

Retrieving S3 links to BSSH files via a web browser

The details are slightly different for Internet Explorer, Chrome and Firefox.

1. You need to be logged into your BaseSpace account.
2. Navigate to the BAM you want to view (likely My Data → Projects → whatever project → whatever analysis → Output Files → whatever folder → whatever BAM), then click Download.

• On Internet Explorer, this spins up a new browser tab and asks if you want to proceed with download. The URL in the address bar of the new tab is the S3 link you want, copy this somewhere then click "no."
• On Chrome, a box containing a progress bar appears at the bottom left of the browser window. Immediately click on the chevron at the right of this box then choose "cancel" to stop the download. Now either do CTRL-J or click the three dots at the top right then choose "Downloads." You'll see a cancelled download (if there's more than one you'll want the topmost) containing the S3 link - right click on it and click "copy link address," then paste it somewhere
• On FireFox, a window pops up on which you select "save file" then "OK." Immediately click on the download icon (down arrow near top right of window) then click "X" to cancel download. Then right click on the file name and click "copy download link."

Retrieving S3 links to BSSH files via BaseMount

• Install BaseMount
• Mount your BSSH files onto your Linux filesystem, as described in BaseMount documentation. Let's assume that your mount point is /basespace/$USER (this is our automount location on Illumina clusters)
• Suppose you want the link for /basespace/$USER/Projects/MY_PROJECT/AppResults/MY_APP_RESULT/Files/MY_FILE.bam, then cat /basespace/$USER/Projects/MY_PROJECT/AppResults/MY_APP_RESULT/Files.metadata/MY_FILE.bam/Content/Url will reveal the S3 link. Note that if you look in the Files.metadata directory it seems empty, but it's not ... trust the Force ...

Here's a function you can paste into your .bashrc file to make life a bit easier - type bs2s3 file_list and it will return the S3 links for the files in file_list.

function bs2s3()
{
    for arg in "$@"
    do
        if [ ! -e "$arg" ]; then
            echo "$arg: file not found"
        else
            p1=$(dirname "$arg")
            p2=$(basename "$arg")
            s3path="${p1}/../Files.metadata/${p2}/Content/Url"
            if [ ! -e "$s3path" ]; then
                echo "$arg: found, but could not obtain s3 link (is this a BaseMount file?)"
            else
                echo "$arg:"
                cat "$s3path"
            fi
        fi
    done
}