Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP API for Yolact, python package #323

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
1 change: 1 addition & 0 deletions data/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -819,6 +819,7 @@ def set_cfg(config_name:str):

if cfg.name is None:
cfg.name = config_name.split('_config')[0]
return cfg

def set_dataset(dataset_name:str):
""" Sets the dataset of the current config. """
Expand Down
35 changes: 26 additions & 9 deletions yolact.py
Original file line number Diff line number Diff line change
Expand Up @@ -396,9 +396,20 @@ class Yolact(nn.Module):
- pred_aspect_ratios: A list of lists of aspect ratios with len(selected_layers) (see PredictionModule)
"""

def __init__(self):
def __init__(self,
config_name="yolact_base_config",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, you wouldn't have to modify config.py to use YOLACT as a pip package. Thus, I think this should accept either the name of the config or a config object (i.e., Union[str, Config]). This would require a similar change in set_cfg too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accept either the name of the config or a config object (i.e., Union[str, Config]).

isn't that what's there now? Name of the config. I can change to the (str, Config) but that seems the same to me. I was thinking about ommiting the set_cfg() and expecting the user to import/create the config they want and just pass it here directly. So from data.config import yolact_base; net = Yolact(yolact_base) ?

):
"""
@param config_name: string name of used config, choose from ./data/config.py, default "yolact_base"
"""
super().__init__()

#set (custom) config
from data.config import set_cfg
breznak marked this conversation as resolved.
Show resolved Hide resolved
cfg = set_cfg(str(config_name))
self.cfg = cfg


self.backbone = construct_backbone(cfg.backbone)

if cfg.freeze_bn:
Expand Down Expand Up @@ -470,6 +481,16 @@ def __init__(self):
self.detect = Detect(cfg.num_classes, bkg_label=0, top_k=cfg.nms_top_k,
conf_thresh=cfg.nms_conf_thresh, nms_thresh=cfg.nms_thresh)


# set default backbone weights
self.init_weights(backbone_path='weights/' + cfg.backbone.path)

# GPU
#TODO try half: net = net.half()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Off topic, but has anyone tried running the models as half/fp16? Generally improves performance,mem footprint.

self.cuda()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably either pass in the device as an argument or let the user handle this externally. Just calling .cuda() has pytorch pick a GPU, and it restricts future CPU or TPU support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and to add arbitrary device support (basically change all the .cuda() and fix all the tensor initializations).

I'll need some guidance on this. Do you mean just add Yolact(device="gpu/cpu/tpu") and handle these accordingly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this and prepared support for the feature is in 236352d ,
But there's a lot of .cuda() calls in the code so far, even in external DCNv2. I'd prefer to finish this feature in a separate PR later.
Btw, the alternative to .cuda() is .to(device="gpu") ..if so, that's what we'd use.

torch.set_default_tensor_type('torch.cuda.FloatTensor')
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this didn't break anything, then it's fine, but this should probably go at the start of init not here.



def save_weights(self, path):
""" Saves the model's weights using compression because the file sizes were getting too big. """
torch.save(self.state_dict(), path)
Expand Down Expand Up @@ -683,19 +704,15 @@ def forward(self, x):
from utils.functions import init_console
init_console()

# initialize yolact
net = Yolact()

# Use the first argument to set the config if you want
import sys
if len(sys.argv) > 1:
from data.config import set_cfg
set_cfg(sys.argv[1])
net = Yolact(config_name=sys.argv[1])

net = Yolact()
net.train()
net.init_weights(backbone_path='weights/' + cfg.backbone.path)

# GPU
net = net.cuda()
torch.set_default_tensor_type('torch.cuda.FloatTensor')

x = torch.zeros((1, 3, cfg.max_size, cfg.max_size))
y = net(x)
Expand Down