final-proj

CS 543 Final Project: Focused object detection for tetraplegic people using Mask R-CNN segmentation and gaze estimation

Requirements to run

pip install git+https://github.com/facebookresearch/segment-anything.git. If you want to use SAM2, use: pip install git+https://github.com/facebookresearch/sam2.git. SAM2 requires a different environment than SAM (python 3.10+).

For SAM2
- Ensure python version >= 3.10
- sudo git clone https://github.com/facebookresearch/sam2.git && cd sam2
- pip install -e .

pip install -r requirements.txt
Download for vit_b here: https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
Download for yolo here: https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8s.pt (To run with yolo, use main_yolo.ipynb notebook)
Download for SAM2 here: https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_tiny.pt
Save downloads under ckpt folder to run. Make sure they match with the file nams in the ipynb notebook desired.

Name		Name	Last commit message	Last commit date
Latest commit History 34 Commits
ckpt		ckpt
data		data
example		example
modules		modules
sam2_configs		sam2_configs
stereoCalibrate		stereoCalibrate
utils		utils
.gitignore		.gitignore
README.md		README.md
config.py		config.py
frame_handler.ipynb		frame_handler.ipynb
image.png		image.png
left_chessboard.npy		left_chessboard.npy
main.ipynb		main.ipynb
main.py		main.py
main_yolo.ipynb		main_yolo.ipynb
main_yolo_sam2.ipynb		main_yolo_sam2.ipynb
model.py		model.py
normalization_example.py		normalization_example.py
pts_left.npy		pts_left.npy
pts_right.npy		pts_right.npy
requirements.txt		requirements.txt
right_chessboard.npy		right_chessboard.npy
segmenter.ipynb		segmenter.ipynb
trainer.py		trainer.py
utils.py		utils.py
video_processor.ipynb		video_processor.ipynb

Provide feedback