Image preprocessing suggestions

[fsuk]

I was having problems reading barcodes with poor print quality, eventually I have worked out how to preprocess the image to make it more readable by Quagga2.

This would be good as a documented example or even as a feature.

My method uses OpenCV.js however I'm sure there would be a way to do this without it.
In my final solution I try to decode first without preprocessing and then with.
It does have a small performance impact, in my testing I found it took between 150-250ms which is still fast enough for my needs.

import { imread, Mat, cvtColor, GaussianBlur, adaptiveThreshold, COLOR_RGBA2GRAY, THRESH_BINARY, ADAPTIVE_THRESH_MEAN_C, imshow, Size } from '@techstark/opencv-js'

async preProcessImage (htmlImage:HTMLImageElement) {

    // Assumptions:
    // The image is 800x800 px
    // The barcode is roughly of the correct orientation 
    // The barcode is located within the central 50% of the image vertically

    // Load the image as an OpenCV.js Mat
    const mat = imread(htmlImage)

    // Crop the image vertically to only use the center of the image
    // Similar to the 'area' property in the quagga input stream
    const readArea = {x: 0, y: 0.25, width: 1, height: 0.5}

    const cropped = mat.roi({
      x: Math.floor(mat.cols * readArea.x),
      y: Math.floor(mat.rows * readArea.y),
      width: Math.floor(mat.cols * readArea.width),
      height: Math.floor(mat.rows * readArea.height)
    })

    // Tidy up the orignal
    mat.delete()

    // Convert the image to greyscale
    // Quagga2 does this already but we need to do it before the rest of the preprocessing.
    const gray = new Mat()
    cvtColor(cropped, gray, COLOR_RGBA2GRAY)
    cropped.delete()

    // Smooth the image to reduce noice
    // This uses a patch size which is taller than it is wide as we are assuming the barcode is more or less the correct orientation, we therefore want to smooth the bars of the barcode vertically.
    // Patch size may need to be adjusted if the original image is not 800x800
    const smoothed = new Mat()
    GaussianBlur(gray, smoothed, new Size(3, 9), 0)
    gray.delete()

    // Convert the image to binary black and white
    // Block size may need to be adjusted if the original image is not 800x800
    const binary = new Mat()
    adaptiveThreshold(smoothed, binary, 255, ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 17, 2)
    smoothed.delete()

    // Tidy up and return the image as a data url.
    const filteredMat = binary
    const canvas = document.createElement('canvas')
    canvas.width = filteredMat.cols
    canvas.height = filteredMat.rows
    imshow(canvas, filteredMat)
    const filteredImage = canvas.toDataURL()
    filteredMat.delete()
    return filteredImage
  }