Batch Shipyard can accommodate most containerized Batch and HPC workloads. This area contains recipes and sample container workloads that may relate to your own anticipated scenario. Batch Shipyard is not limited to these programs or types of workloads represented by these recipes, but can run arbitrary containers to execute your Batch and HPC scenario.
Please note that all recipes have sample configurations that you can use to
quickly get started, however, some of the config files cannot be used as-is
as they may need to be modified for specific values (such as the
credentials.yaml file which needs to be populated with your Batch and
Storage account credentials). Please review each config file you are planning
on using and modify accordingly. As these recipe configuration are intended
to show example usage, you will need to modify and configure them for your
actual workloads.
If you would like to contribute your recipe, please take a look at this guide before submitting a pull request.
Use the following links to quickly navigate to recipe collections:
- Benchmarks
- Computational Fluid Dynamics (CFD)
- Deep Learning
- Genomics and Bioinformatics
- Molecular Dynamics (MD)
- RemoteFS
- Slurm on Batch
- Video Processing
- Windows
This HPCG-Infiniband-IntelMPI recipe contains information on how to containerize HPCG across Infiniband/RDMA Azure VMs with Intel MPI.
This HPLinpack-Infiniband-IntelMPI recipe contains information on how to containerize HPLinpack (HPL) across Infiniband/RDMA Azure VMs with Intel MPI.
This mpiBench-Infinibnad-MPICH recipe contains information on how to containerize mpiBench across Infiniband/RDMA Azure VMs with MPICH.
This mpiBench-Infinibnad-MPICH recipe contains information on how to containerize mpiBench across Infiniband/RDMA Azure VMs with MPICH.
This mpiBench-IntelMPI recipe contains information on how to containerize mpiBench across Azure VMs with Intel MPI.
This mpiBench-MPICH recipe contains information on how to containerize mpiBench across Azure VMs with MPICH.
This mpiBench-OpenMPI recipe contains information on how to containerize mpiBench across Azure VMs with Open MPI.
This OSUMicroBenchmarks-Infiniband-MVAPICH recipe contains information on how to containers the OSU Micro-Benchmarks across Infiniband/RDMA Azure VMs with MVAPICH.
This OpenFOAM-Infiniband-IntelMPI recipe contains information on how to containerize distributed OpenFOAM across Infiniband/RDMA Azure VMs with Intel MPI.
This OpenFOAM-TCP-OpenMPI recipe contains information on how to containerize distributed OpenFOAM across multiple Azure Batch compute nodes.
This Caffe-CPU recipe contains information on how to containerize Caffe for use on Azure Batch compute nodes.
This Caffe-GPU recipe contains information on how to containerize Caffe on GPUs for use with N-Series Azure VMs.
This Caffe2-CPU recipe contains information on how to containerize Caffe2 for use on Azure Batch compute nodes.
This Caffe2-GPU recipe contains information on how to containerize Caffe2 on GPUs for use with N-Series Azure VMs.
This Chainer-CPU recipe contains information on how to containerize Chainer for use on Azure Batch compute nodes.
This Chainer-GPU recipe contains information on how to containerize Chainer on GPUs for use with N-Series Azure VMs.
This CNTK-CPU-Infiniband-IntelMPI recipe contains information on how to containerize CNTK for CPUs, including execution across multiple Infiniband/RDMA Azure VMs with multi-instance tasks.
This CNTK-CPU-OpenMPI recipe contains information on how to containerize CNTK for CPUs, including execution across multiple compute nodes with multi-instance tasks.
This CNTK-GPU-Infiniband-IntelMPI recipe contains information on how to containerize CNTK on GPUs for use with N-Series Azure VMs, including execution across multiple Infiniband/RDMA Azure VMs with multi-instance tasks.
This CNTK-GPU-OpenMPI recipe contains information on how to containerize CNTK on GPUs for use with N-Series Azure VMs, including execution across multiple compute nodes and multiple GPUs with multi-instance tasks.
This recipe contains information on how to containerize the Microsoft High Performance ML Algorithms (HPMLA) for use across multiple compute nodes.
This Keras+Theano-CPU recipe contains information on how to containerize Keras with the Theano backend for use on Azure Batch compute nodes.
This Keras+Theano-GPU recipe contains information on how to containerize Keras with the Theano backend for use with N-Series Azure VMs.
This MXNet-CPU recipe contains information on how to containerize MXNet for use on Azure Batch compute nodes, including execution across multiple compute nodes with multi-instance tasks.
This MXNet-GPU recipe contains information on how to containerize MXNet on GPUs for use with N-Series Azure VMs, including execution across multiple compute nodes and multiple GPUs with multi-instance tasks.
This PyTorch-CPU recipe contains information on how to containerize PyTorch for use on Azure Batch compute nodes.
This PyTorch-GPU recipe contains information on how to containerize PyTorch on GPUs for use with N-series Azure VMs.
This TensorFlow-CPU recipe contains information on how to containerize TensorFlow for use on Azure Batch compute nodes.
This TensorFlow-Distributed recipe contains information on how to containerize TensorFlow on GPUs for use with N-series Azure VMs or across multiple CPU nodes.
This TensorFlow-GPU recipe contains information on how to containerize TensorFlow on GPUs for use with N-series Azure VMs.
This Torch-CPU recipe contains information on how to containerize Torch for use on Azure Batch compute nodes.
This Torch-GPU recipe contains information on how to containerize Torch on GPUs for use with N-series Azure VMs.
This BLAST-CPU recipes contains information on how to containerize a BLAST pipeline with parallelization using Azure Batch compute nodes.
This RNASeq-CPU recipes contains information on how to containerize a proof-of-concept RNA-Seq pipeline using Azure Batch compute nodes.
This NAMD-GPU recipe contains information on how to containerize NAMD on GPUs for use with N-Series Azure VMs.
This NAMD-Infiniband-IntelMPI recipe contains information on how to containerize distributed NAMD across Infiniband/RDMA Azure VMs with Intel MPI.
This NAMD-TCP recipe contains information on how to containerize distributed NAMD across multiple Azure Batch compute nodes using TCP.
This RemoteFS-GlusterFS recipe contains information on how to provision a sample multi-VM GlusterFS storage cluster.
This RemoteFS-GlusterFS+BatchPool recipe contains information on how to provision a Batch pool and automatically link it against a provisioned GlusterFS storage cluster.
This RemoteFS-NFS recipe contains information on how to provision a sample single VM NFS server.
This recipe contains information on how to orchestrate a Slurm cluster with an NFS shared file system.
This recipe contains information on how to containerize FFmpeg on GPUs for use with the N-series Azure VMs.
This recipe contains information on how to run a containerized version of the DiskSpd tool with Windows Server Containers on Azure VMs.
This recipe contains information on how to run a sample DotNet application with Windows Server Containers on Azure VMs.