Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added imread, imsave and synchronize functionality #2

Open
wants to merge 85 commits into
base: main
Choose a base branch
from
Open

Conversation

thawn
Copy link

@thawn thawn commented Aug 8, 2022

You can now just use imread(filename) and the library uses a local file on the cluster if it finds a matching one, or it downloads the file to a temporary location from the fileserver and accesses that one.

Similarly, imsave(filename, data), saves the file if the path is accessible. Otherwise, it saves to a temp file and then copies that to the final location (on the fileserver or project space) using the datamover tools.

The best way to test is to install the test singularity image for biapol-taurus:

git clone https://gitlab.mn.tu-dresden.de/bia-pol/singularity-devbio-napari.git
cd singularity-devbio-napari
./install.sh biapol_taurus

and then open a notebook with the devbio-napari_biapol_taurus kernel.

inside the notebook run:

from biapol_taurus import ProjectFileTransfer
from skimage.io import imshow

# Define the source directory. /grp/g_biapol/ is the mount point at the export node. HPCsupport can tell us this:
source_dir = "/grp/g_biapol/"

# Define the target directory. /projects/p_bioimage/ is the project space. HPCsupport told us that when we applied for the project:
target_dir = "/projects/p_bioimage/"

# We setup a connection:
pft = ProjectFileTransfer(source_dir, target_dir)

image = pft.imread('data/test/blobs19.tif')
# show image
imshow(image)
pft.imsave('/projects/p_bioimage/till/blobs_saved.tif', image)

use either fileserver or project space as source.
no more syncing between project space and fileserver
avoid unnecessary file transfers from the fileserver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants