diff --git a/README.rst b/README.rst index 67417e59..51d9939e 100644 --- a/README.rst +++ b/README.rst @@ -13,16 +13,3 @@ Build/Test: Test Coverage: .. image:: ../../coverage.svg - -#### Docker - -The Dockerfile is currently using,\ -- bioformats2raw version: 0.7.0. -- imod version: 4.11.24 - -In order to build the docker image, use `--platform linux/amd64` option. -Explanation can be found [here](https://teams.microsoft.com/l/entity/com.microsoft.teamspace.tab.wiki/tab::5f55363b-bb53-4e5b-9564-8bed5289fdd5?context=%7B%22subEntityId%22%3A%22%7B%5C%22pageId%5C%22%3A15%2C%5C%22sectionId%5C%22%3A17%2C%5C%22origin%5C%22%3A2%7D%22%2C%22channelId%22%3A%2219%3A869be6677ee54848bc13f2066d847cc0%40thread.skype%22%7D&tenantId=14b77578-9773-42d5-8507-251ca2dc2b06) - -```bash -$ docker build . -t hedwig_pipelines --platform linux/amd64 -``` diff --git a/docs/source/development.rst b/docs/source/development.rst index bd731224..b3b2ec6b 100644 --- a/docs/source/development.rst +++ b/docs/source/development.rst @@ -50,15 +50,21 @@ pre-commit format compliance at commit time. It is configured via a top-level ``.pre-commit-config.yaml`` file that runs ``black``, ``Flake8`` and other checks. +Docker +====== + +`Docker `_ is a platform that provides OS-level virtualization so that softwares are developed and packaged in containers. This helps consistent development in different environments. + ******* Testing ******* -There are currently four ``pytest`` files in the `test` directory: +All ``pytest`` files reside in the `test` directory: - ``test_brt``: end-to-end test of batchruntomo pipeline - ``test_dm``: 2D end-to-end pipeline test - ``test_sem``: end-to-end test of FIBSEM pipeline +- ``test_lrg_2d``: Large 2d pipeline test - ``test_utils``: unit tests of utils/utils.py module There is test data for `test_dm` in the Git repo, but not for the others. These files need to be @@ -125,6 +131,32 @@ not have binaries built for the M1. Need to install using an x86_64 version of P are issues installing `biofomats2raw` as there is no OpenCV package for arm64 chip. There currently is no fix or workaround for this issue. +Docker +====== + +You can also choose to use Docker for local development and testing. + +In order to build the docker image, use `--platform linux/amd64` option. +Explanation can be found `here `_ + +The basic usage for testing would look like such: + +.. code-block:: sh + + # To build the image + docker build . -t hedwig_pipelines --platform linux/amd64 + + +In order to boot up a container with pipeline image, you can run the command below. The command assumes that you are running the container from the project directory. + +.. code-block:: sh + + # To run the container of that image + docker run -v "$(pwd):/image_portal_workflows" -e USER=root -it --rm hedwig_pipelines:latest + + +Once you are in the container, you can run the commands you want to. For example: `pytest`. + HPC Set up ========== diff --git a/test/copy_test_data.sh b/test/copy_test_data.sh index 6b0a584c..c9b15893 100755 --- a/test/copy_test_data.sh +++ b/test/copy_test_data.sh @@ -7,12 +7,12 @@ # - cd to top-level "image_portal_workflows" dir of your repo # test_brt - one 970MB file -mkdir -p test/input_files/brt_inputs -rsync --progress --stats -ave ssh ai-rmlsbatch1.niaid.nih.gov:/mnt/ai-fas12/RMLEMHedwigQA/test/input_files/brt_inputs/Projects/2013-1220-dA30_5-BSC-1_10.mrc test/input_files/brt_inputs/ +mkdir -p test/input_files/brt_inputs/Projects +rsync --progress --stats -ave ssh ai-rmlsbatch1.niaid.nih.gov:/mnt/ai-fas12/RMLEMHedwigQA/test/input_files/brt_inputs/Projects/2013-1220-dA30_5-BSC-1_10.mrc test/input_files/brt_inputs/Projects # test_sem - directory of smallish files, but 1.1GB total -mkdir -p test/input_files/sem_inputs -rsync --progress --stats -ave ssh ai-rmlsbatch1.niaid.nih.gov:/mnt/ai-fas12/RMLEMHedwigDev/Projects/BCBB_TEST/test/input_files/sem_inputs/ test/input_files/sem_inputs/ +mkdir -p test/input_files/sem_inputs/Projects +rsync --progress --stats -ave ssh ai-rmlsbatch1.niaid.nih.gov:/mnt/ai-fas12/RMLEMHedwigDev/Projects/BCBB_TEST/test/input_files/sem_inputs/1/ test/input_files/sem_inputs/Projects/ # test_dm - These were in the repo as of March 2023, but uncomment to copy #rsync -ave ssh ai-rmlsbatch1.niaid.nih.gov:/mnt/ai-fas12/RMLEMHedwigQA/test/input_files/dm_inputs/Projects/Lab/PI/ test/input_files/dm_inputs/ diff --git a/test/test_utils.py b/test/test_utils.py index 9a217b61..b5dfefd3 100644 --- a/test/test_utils.py +++ b/test/test_utils.py @@ -237,7 +237,6 @@ def test_mrc_to_movie(mock_nfs_mount): input_dir = "test/input_files/sem_inputs/Projects/mrc_movie_test" input_path = Path(os.path.join(proj_dir, input_dir)) # FIXME input directory `sem_inputs` in test/input_files is missing - print(input_path) assert os.path.exists(input_path) # FIXME adjusted.mrc is missing image_path = Path(os.path.join(proj_dir, input_dir, "adjusted.mrc"))