Put feature map visualizations in the same "exp" directory

[adrianholovaty]

I was excited to check out the new feature map functionality! One thing came up in my testing, which seems like an opportunity for improvement.

At the moment, each distinct feature map image gets put in a new (incremented) exp directory — exp2, exp3, exp4, etc. It looks as though the feature map image files will always have distinct names within the context of a single run, so it would be safe (and more intuitive) to put them in a single directory together.

So, instead of this resulting directory structure:

exp/
    stage_1_Conv_features.png
exp2/
    stage_2_C3_features.png
exp3/
    stage_3_Conv_features.png
exp4/
    stage_4_C3_features.png
exp5/
    stage_5_Conv_features.png
exp6/
    stage_6_C3_features.png
exp7/
    stage_13_C3_features.png
exp8/
    stage_14_Conv_features.png
exp9/
    stage_17_C3_features.png
exp10/
    stage_18_Conv_features.png
exp11/
    stage_20_C3_features.png

...it would be:

exp/
    stage_1_Conv_features.png
    stage_2_C3_features.png
    stage_3_Conv_features.png
    stage_4_C3_features.png
    stage_5_Conv_features.png
    stage_6_C3_features.png
    stage_13_C3_features.png
    stage_14_Conv_features.png
    stage_17_C3_features.png
    stage_18_Conv_features.png
    stage_20_C3_features.png

The downside of the current approach is: When you run inference, the system will generate several exp directories in thefeatures directory — e.g., features/exp2, features/exp3, features/exp4 — while only using a single detect/exp directory for the inference results. This makes it difficult to cross-reference which features/* directories go with which detect/* directories, because the numbers are out of sync.

To see what I mean, change forward_once() in models/yolo.py to something like this:

  if feature_vis and m.type in {'models.common.C3', 'models.common.SPP', 'models.common.Conv'}:
      feature_visualization(x, m.type, m.i)

I believe this is what @Zigars was referring to in this comment, but I'm not 100% sure.

The root of this issue is that the new feature_visualization() function always increments a new exp directory, for every image it creates. A solution could be:

• Change feature_visualization() to take an output_dir parameter and use that, instead of calling increment_path().
• Change all calls to feature_visualization() to pass the output_dir.
• From there, I'm unsure of the best approach. It seems reasonable to create a global "run number" at the start of training/inference, and pass that to the various places appropriately (the X to be used in detect/expX and features/expX), but I'm not comfortable enough with the YOLOv5 code to know where to put this.

[glenn-jocher]

@adrianholovaty ya maybe that's a good idea, though the model of course has no concept of what it's doing really beyond being in .train() mode or .eval() mode, so it doesn't know if it's being called by train, test, detect, or as a hub model, or even less what directories those processes may log to.

I'll go take a look.

[adrianholovaty]

@glenn-jocher Yeah, it smells like some tight coupling for sure. :-/

Is there an object that contains state for the current train/test/detect run? Like, runtime params. That would be a good place to put this data, rather than on the model (which, yeah, seems inappropriate).

[glenn-jocher]

@adrianholovaty can you try this variant of feature_visualization() (in utils/plots.py) and submit a PR if this works better?

def feature_visualization(x, module_type, stage, n=64):
    """
    x:              Features to be visualized
    module_type:    Module type
    stage:          Module stage within model
    n:              Maximum number of feature maps to plot
    """
    batch, channels, height, width = x.shape  # batch, channels, height, width
    if height > 1 and width > 1:
        f = f"stage_{stage}_{module_type.split('.')[-1]}_features.png"  # filename
        project, name = Path('runs/features'), 'exp'
        save_dir = increment_path(project / name, exist_ok=not (project / name / f).exists())  # increment run
        save_dir.mkdir(parents=True, exist_ok=True)  # make dir

        plt.figure(tight_layout=True)
        blocks = torch.chunk(x, channels, dim=1)  # block by channel dimension
        n = min(n, len(blocks))
        for i in range(n):
            feature = transforms.ToPILImage()(blocks[i].squeeze())
            ax = plt.subplot(int(math.sqrt(n)), int(math.sqrt(n)), i + 1)
            ax.axis('off')
            plt.imshow(feature)  # cmap='gray'

        print(f'Saving {save_dir / f}...')
        plt.savefig(save_dir / f, dpi=300)

[glenn-jocher]

@adrianholovaty good news 😃! Your original issue may now be fixed ✅ in PR #3920. This is a complete revamp of feature visualization with many fixes and improvements.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload with model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

[adrianholovaty]

@glenn-jocher Looks good — that fixes the issue I raised here. Nice one!

[chelsea456]

@adrianholovaty good news 😃! Your original issue may now be fixed ✅ in PR #3920. This is a complete revamp of feature visualization with many fixes and improvements.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload with model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

@adrianholovaty good news 😃! Your original issue may now be fixed ✅ in PR #3920. This is a complete revamp of feature visualization with many fixes and improvements.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload with model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

Hi sir, i have a question in your picture. In picture hava 32 images, what mean in it's image ? tks sir !!!