A responsible path to generative AI in healthcare: Unleash the power of Med-PaLM 2 to revolutionize medical knowledge, answer complex questions, and enhance healthcare experiences with accuracy, safety, and equitable practices.
pip install MedPalm
import torch
from medpalm.model import MedPalm
#usage
img = torch.randn(1, 3, 256, 256)
text = torch.randint(0, 20000, (1, 4096))
model = MedPalm()
output = model(img, text)
print(output.shape)📝 Note: Modify the examples to suit your data and project needs.
Here is a comprehensive markdown table containing metadata and details for all the datasets mentioned in the MED PALM paper
| Dataset | Modality | Description | # Training examples | # Test examples | Tasks |
|---|---|---|---|---|---|
| MultiMedBench | Multimodal | Benchmark for biomedical AI | 14 biomedical tasks | ||
| MedQA | Text | US licensing exam questions | 10,178 | 1,273 | Question answering |
| MedMCQA | Text | Indian medical exam questions | 182,822 | 4,183 | Question answering |
| PubMedQA | Text | Biomedical literature questions | 0 | 500 | Question answering |
| MIMIC-III | Radiology reports | Radiology reports for ICU patients | 58,405 reports | 13,057 reports | Report summarization |
| VQA-RAD | Radiology images | QA pairs on radiology images | 1,797 QA pairs | 451 QA pairs | Visual question answering |
| Slake-VQA | Radiology images | English-Chinese QA pairs | 9,849 samples | 2,070 samples | Visual question answering |
| Path-VQA | Pathology images | QA pairs on pathology images | 19,755 QA pairs | 6,761 QA pairs | Visual question answering |
| MIMIC-CXR | Chest X-ray | Images and reports | 353,542 | 4,834 | Report generation, classification |
| PAD-UFES-20 | Dermatology images | Skin lesion smartphone images | 1,838 images | 460 images | Image classification |
| CBIS-DDSM (mass) | Mammography | Mammogram mass patches | 1,318 images | 378 images | Image classification |
| CBIS-DDSM (calcification) | Mammography | Mammogram calcification patches | 1,544 images | 326 images | Image classification |
| VinDr-Mammo | Mammography | Mammogram studies | 16,000 images | 4,000 images | Image classification |
| PrecisionFDA (training) | Genomics | Genomic variant images | 197,038 images | Image classification | |
| PrecisionFDA (evaluation) | Genomics | Genomic variant images | 13,030 images | Image classification | |
| Montgomery County | Chest X-ray | Chest X-rays | 0 | 138 images | TB detection evaluation |
| MIMIC-CXR (human evaluation) | Chest X-ray | Chest X-ray images and reports | 246 cases | Human evaluation |
Med Palm has thousands of potential use cases the 3 below are simple, for more detailed applications check out my new blog article on MedPalm's use in the real world. Click here to learn more
-
Clinical Diagnostics: Combining medical imaging, patient tales 📖, and genes, we're aiming for top-notch diagnostic solutions.
-
Healthcare Research: Dive deep into diverse datasets and discover something new with Med-PaLM by your side! 🤿
-
Telemedicine: Quick, reliable, and remote! 🌍 Med-PaLM's here to revolutionize telehealth.
Help with the todo list!
Med-PaLM's is under the MIT license. Check out the details here.
@misc{2307.14334,
Author = {Tao Tu and Shekoofeh Azizi and Danny Driess and Mike Schaekermann and Mohamed Amin and Pi-Chuan Chang and Andrew Carroll and Chuck Lau and Ryutaro Tanno and Ira Ktena and Basil Mustafa and Aakanksha Chowdhery and Yun Liu and Simon Kornblith and David Fleet and Philip Mansfield and Sushant Prakash and Renee Wong and Sunny Virmani and Christopher Semturs and S Sara Mahdavi and Bradley Green and Ewa Dominowska and Blaise Aguera y Arcas and Joelle Barral and Dale Webster and Greg S. Corrado and Yossi Matias and Karan Singhal and Pete Florence and Alan Karthikesalingam and Vivek Natarajan},
Title = {Towards Generalist Biomedical AI},
Year = {2023},
Eprint = {arXiv:2307.14334},
}- Implement a training script using Huggingface's accelerate and deepspeed