KIDS24-team3

Optimizing High-Performance Computing Resource Allocation Through Statistical Analysis

Welcome to our GitHub repository for the KIDS24 BioHackathon challenge! 🎉 This repository will serve as the central hub for all our code, data, and documentation.

Table of Contents

• Introduction
• Dataset
• Mission
• Setup Instructions
• Resources
• Progress
• Notes

Introduction

Our team is tackling the challenge of optimizing high-performance computing (HPC) resource allocation through statistical analysis. By improving HPC utilization, we aim to help researchers run their experiments faster and more efficiently, thereby accelerating scientific discoveries.

Dataset

The dataset includes three types of data:

• Time series: e.g., host load, job load
• Events: e.g., job status changes
• Configurations: job submission info, cluster configuration

more details of the dataset will be available before the challenge starts.

Mission

Our mission is to leverage our collective expertise to improve the understanding of HPC statistics and find measures to enhance HPC utilization.

Setup Instructions

1. Clone the repository:

   git clone https://github.com/stjude-biohackathon/KIDS24-team3.git

Resources

Offical Schedule

• BioHackathon Opening Reception Night. 5:30 PM Sept 3, ARC Lobby

HPC metrics dataset

TODO

Progress

Overview progress

+-------------------+       +-------------------+       +-------------------+
|                   |       |                   |       |                   |
|  Data Collection  +------>+  Data Processing  +------>+  Data Cleaning    |
|                   |       |                   |       |                   |
+-------------------+       +-------------------+       +-------------------+
        ^                                                       |
        |                                                       v
+-------------------+       +-------------------+       +-------------------+
|                   |       |                   |       |                   |
|  HPC Cluster      |       |  Optimization     |       |  Statistical      |
|  Metrics          |       |  Techniques       +<------+  Analysis         |
|                   |       |                   |       |   (You are here)  |
+-------------------+       +-------------------+       +-------------------+

Notes

Please be sure not to upload personal/sensitive information to this github repo given it's publicly accessible.