software and containers page updates

[vortexing]

URL of existing content (a rendered, sciwiki.fredhutch.org link)

https://sciwiki.fredhutch.org/scicomputing/compute_environments/

Specific information requests or concerns, or specific region on page of content to refresh.

@ GitHub username of the listed primary reviewer of the page if known

[tefirman]

@vortexing @sitapriyamoorthi -- Thoughts on these proposed updates?

SciWiki Docker Content Refresh

Current State

Core Content:

• “Computing Environments and Containers” (SciComp)
  • Comprehensive article about environments at FH, potentially too long…
  • Environment Modules, Docker Containers, Virtual Environments with Python/R
• “Using Apptainer Containers” (SciComp)
  • Instructions specific to Apptainer, important for cluster usage
• “Building Software Containers” (HDC)
  • Step by step building instructions
• “Using Docker at Fred Hutch”
  • Limited info, pretty much a shell
• “Why is your computing environment important?”
  • Summary article from Ted

Mentioned in:

• “Workflow Background”
  • Nextflow overview from Sam Minot
• “Cromwell Workflow Manager and WDL Workflows”
  • WDL overview from Amy
• “Running Workflows on Gizmo”
  • Nitty-gritty of running Nextflow, includes super helpful example config
• “Using AWS Batch”
• “Nextflow - an Emerging Service for Workflow Management”
• “PROOF Troubleshooting”

Proposed State

Priority:

• Split “Computing Environments and Containers” into three separate articles
  • Environment Modules
  • Docker Container Background
  • Virtual Environments with Python/R
• Delete/absorb “Using Docker at Fred Hutch” into "Building Software Containers"
  • Add more command line details and point to WILDS Docker Library
• Leave Ted’s article as-is, good overview of why each one is important
• Leave “Using Apptainer” article as-is, important distinction
• Update links wherever Docker is mentioned

Optional:

• Consolidate “Workflow Background” and “Nextflow” articles into one?
• Update Cromwell/WDL article to include miniWDL, Sprocket, example config

Notes

Search seems to be indexing deleted pages in “datademos” and “_datademos”, produces 404 errors, which will probably deter readers…

[vortexing]

FYI @tefirman I don't believe search works in teh preview site b/c it renders against what's in main. So that's a red herring.

[vortexing]

Seems like a big chunk of work that we should do together in March instead of with this massive reorg.

[tefirman]

Seems like a big chunk of work that we should do together in March instead of with this massive reorg.

Totally agree, smarter to finish the reorg and we can address this after.