Big-picture discussion

[eddogola]

👋 Welcome!

This can be the main discussion channel for the research project. From reading it, one should get a chronological idea of what I am working on, what's working, what's not and the way forward. It helps to have my end goal for grounding(at least for myself), which is at the bottom of this note.

Research Contribution Goal: Make segmentation models, that haven't been made efficient in the before, efficient and accurate by fitting them in the browser without experience issues(latency, crashing etc.).

[eddogola]

As of the first week of November

I have been doing preliminary experiments to find the best set of models to work with, to focus on making more efficient, and the boundaries of performance, i.e. the highest image dimensions before accuracy and efficiency dwindles. This series of experiments focused on the U-Net family of models especially U-Net, squeeze U-Net, and Mask-RCNN.

Going forward

• I will experiment with the SAM family of models, running the same set of experiments after finding a way to make it work in the browser without any user experience difficulties.
• Experiment using webGPU with either types of models
• Reimagine the Piximi experience: how else could the user experience of Piximi change while maintaining a high standard of accuracy and efficiency

[eddogola]

• Big change: decided to focus on YOLACT, implementing it with an additional set of regression layers, then porting it to TF.js
• After spending a few weeks attempting to recapitulate the YOLACT architecture, we decided to build on top of Yolact_Minimal