A barcode reading benchmark tool based on Quasar + Vue. It can create barcode reading benchmark projects which save the test results and images in indexedDB. The benchmark can run purely in the browser.
The following SDKs are included for testing:
Extra SDKs are supported via HTTP using Barcode-Reader-Aggregator.
- Run batch barcode decoding on an image dataset
- Calculate metrics like reading rate, precision and average time
- Compare the performance of different barcode reading libraries
- Compare the performance in categories
- Export and import of benchmark projects
- Reading rate:
correctly detected barcodes / total barcodes
- Precision:
incorrectly detected barcodes / all detected barcodes
- Average time:
total time elapsed / total files
- Check if there is a ground truth barcode which overlaps with the detected barcode using overlapped percent. (the area of the intersection / the area of the ground truth or the detected result which has a smaller area).
- Check if the barcode text is correct. If the barcode content is in raw bytes, compare the binaries. Since 2D barcodes use Reed–Solomon error correction, it is okay if some of the binaries do not match with the ground truth's (based on edit distance).
-
Barcode Detection Result
export interface DetectionResult { elapsedTime:number; results:BarcodeResult[]; } export interface BarcodeResult { barcodeFormat: string; barcodeText: string; barcodeBytes: string; confidence?: number; x1:number; x2:number; x3:number; x4:number; y1:number; y2:number; y3:number; y4:number; }
Barcode detection result is saved in the above format in JSON with the
.json
extension. -
Ground Truth
export interface GroundTruth { text:string; x1:number; x2:number; x3:number; x4:number; y1:number; y2:number; y3:number; y4:number; attrib:Attrib; value_attrib:ValueAttrib; } export interface Attrib { Type:string; } export interface ValueAttrib { Mode?:string; }
Ground truth is saved in the above format in JSON with the
.txt
extension. -
Remote Projects
Put the files like the following example to the dataset folder as remote projects. Users can download them to check them out locally. The files can be retrieved using the export feature.
│ projects.json │ │─ Project detection_result_filenames.json project_manifest.json results.zip [P]ISBN_18_0010.jpg [P]ISBN_18_0077.jpg
yarn
# or
npm install
quasar dev
quasar build