feat: Ability to scan both normal codes and inverted codes #1215

RafaRuiz · 2024-10-12T00:53:15Z

Started on #1214 to improve #1071 but with a few enhancements:

No usage of OpenCV for Android, MLKit would do the inversion.
Set intervalInvertImage instead of choosing whether to invert the image constantly or not. This way we can read both types, for example:

Both types of codes (example)

Kept to latest version 6.0.1 instead from the first one which was taken from 5.0.1

navaronbracke

Overall, this looks okay, but I have feedback for the implementation.

Will you be looking into color inversion for the Vision API (for MacOS) as well?

lib/src/objects/start_options.dart

lib/src/mobile_scanner_platform_interface.dart

lib/src/mobile_scanner_controller.dart

android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScanner.kt

ios/Classes/MobileScanner.swift

navaronbracke · 2024-10-14T14:40:49Z

ios/Classes/MobileScanner.swift

+
+    func invertImage(image: UIImage) -> UIImage {
+        let ciImage = CIImage(image: image)
+        let filter = CIFilter(name: "CIColorInvert")


I would not use a magic string here, you should be using https://developer.apple.com/documentation/coreimage/cifilter/3228292-colorinvert

@navaronbracke can you give me a hand here?

I'm trying to make it API available, as I've seen there are a few statements for iOS 13 somewhere in the project, but despite setting my project deployment target to 15, I keep seeing this:

I've done just a little Swift in my life, it's been more Android

Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

…canner.kt Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

… from helpers

# Conflicts: # android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScannerHandler.kt

navaronbracke

@juliansteenbakker This entire implementation would omit MacOS, even though that should be supported there also, considering the API's we use.

Since this is a draft, can we close this PR and reopen this on the dev branch instead?

navaronbracke · 2025-01-16T08:37:32Z

lib/src/mobile_scanner_platform_interface.dart

@@ -116,6 +116,11 @@ abstract class MobileScannerPlatform extends PlatformInterface {
    throw UnimplementedError('updateScanWindow() has not been implemented.');
  }

+  /// Set inverting image colors in intervals (for negative Data Matrices).


Suggested change

/// Set inverting image colors in intervals (for negative Data Matrices).

/// Enable or disable the inverting of image colors.

///

/// This is useful when working with negative-color Data Matrices.

/// See also: https://en.wikipedia.org/wiki/Negative_(photography)

navaronbracke · 2025-01-16T08:38:53Z

lib/src/mobile_scanner_controller.dart

+  ///
+  /// Defaults to false and is only supported on Android and iOS.
+  ///
+  /// Defaults to false.


The second Defaults to false. can be removed.

navaronbracke · 2025-01-16T08:39:22Z

lib/src/method_channel/mobile_scanner_method_channel.dart

+  Future<void> setShouldConsiderInvertedImages(bool shouldConsiderInvertedImages) async {
+    await methodChannel.invokeMethod<void>(
+      'setShouldConsiderInvertedImages',
+      {'shouldConsiderInvertedImages': shouldConsiderInvertedImages},


Can we just pass in a boolean flag instead of a Map here? The native implementations will need to be updated accordingly.

navaronbracke · 2025-01-16T08:40:53Z

android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScanner.kt

@@ -79,7 +83,17 @@ class MobileScanner(
    @ExperimentalGetImage
    val captureOutput = ImageAnalysis.Analyzer { imageProxy -> // YUV_420_888 format
        val mediaImage = imageProxy.image ?: return@Analyzer
-        val inputImage = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
+
+        // Invert every other frame.


Suggested change

// Invert every other frame.

// Invert every other frame, by flipping the flag on every input frame.

navaronbracke · 2025-01-16T08:41:04Z

android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScanner.kt

+
+        // Invert every other frame.
+        if (shouldConsiderInvertedImages) {
+            invertCurrentImage = !invertCurrentImage // so we jump from one normal to one inverted and viceversa


Suggested change

invertCurrentImage = !invertCurrentImage // so we jump from one normal to one inverted and viceversa

invertCurrentImage = !invertCurrentImage

navaronbracke · 2025-01-16T08:43:36Z

android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScanner.kt

+        // Convert YUV_420_888 image to NV21 format
+        // based on our util helper
+        val bitmap = Bitmap.createBitmap(image.width, image.height, Bitmap.Config.ARGB_8888)
+        YuvToRgbConverter(activity).yuvToRgb(image, bitmap)


This adds another usage of the deprecated RenderScript API's. Can we avoid this?

See #1142

navaronbracke · 2025-01-16T08:45:26Z

ios/Classes/MobileScannerPlugin.swift

+    /// Sets the zoomScale.
+    private func setShouldConsiderInvertedImages(_ call: FlutterMethodCall, _ result: @escaping FlutterResult) {
+        let shouldConsiderInvertedImages = call.arguments as? Bool
+        if (shouldConsiderInvertedImages == nil) {


Can we guard mobileScanner.setShouldConsiderInvertedImages(shouldConsiderInvertedImages!) to not do anything if the argument is null ? Otherwise this does not match Android.

navaronbracke · 2025-01-16T08:48:03Z

ios/Classes/MobileScanner.swift

@@ -48,6 +48,12 @@ public class MobileScanner: NSObject, AVCaptureVideoDataOutputSampleBufferDelega

    var detectionSpeed: DetectionSpeed = DetectionSpeed.noDuplicates

+    var shouldConsiderInvertedImages: Bool = false
+    // local variable to invert this image only this time,


I think this can be a one line comment.

Suggested change

// local variable to invert this image only this time,

// This flag is used to determine if the current frame should be color-inverted.

juliansteenbakker · 2025-01-16T08:56:55Z

Sounds good to me @navaronbracke. The android implementation can be a lot faster, i will work on it. I'll first merge the changes from master into develop, before fixing the merge conflicts on this PR.

# Conflicts: # android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScanner.kt # android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScannerHandler.kt # ios/Classes/MobileScanner.swift # ios/Classes/MobileScannerPlugin.swift # lib/src/mobile_scanner_controller.dart # lib/src/objects/start_options.dart

juliansteenbakker · 2025-01-17T17:19:37Z

@navaronbracke do you see any usecase for the setShouldConsiderInvertedImages function in the interface? I don't think we would want to set options via functions, it would be best to just let the MobileScannerController handle this with shouldConsiderInvertedImages parameter.

navaronbracke · 2025-01-17T17:32:13Z

@juliansteenbakker I agree, you'd probably want to set it up on the controller itself. That follows the same pattern for the other options.

RafaRuiz added 3 commits October 12, 2024 02:29

Feature: Read inverted data matrix

dff1935

Feature: Read inverted data matrix

740b720

Clean up

ebfe698

RafaRuiz requested a review from juliansteenbakker as a code owner October 12, 2024 00:53

github-actions bot assigned RafaRuiz Oct 12, 2024

RafaRuiz mentioned this pull request Oct 12, 2024

feat: read inverted datamatrix (improve #1071) #1214

Closed

RafaRuiz changed the title ~~feature: Ability to scan both normal codes and inverted codes~~ feat: Ability to scan both normal codes and inverted codes Oct 12, 2024

navaronbracke requested changes Oct 14, 2024

View reviewed changes

RafaRuiz and others added 15 commits October 14, 2024 20:23

Update lib/src/objects/start_options.dart

b1bb926

Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

Update lib/src/mobile_scanner_controller.dart

25e52d7

Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

Update android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileS…

88b7b67

…canner.kt Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

Update ios/Classes/MobileScanner.swift

89d0f20

Co-authored-by: Navaron Bracke <brackenavaron@gmail.com>

Rename feature

5ceb867

Rename methods and variables

989f441

Wording

71d1e83

Refactor the way to invert the image, applying the conversion to NV21…

1e76615

… from helpers

Refactor convertCIImageToCGImage

c8f48b2

redundant temp variable

96becf0

rename var

d8512f4

let

62a728d

private

638c94b

CIFilter magic string

b7080a6

Merge branch 'master' into master

47f8155

RafaRuiz marked this pull request as draft October 18, 2024 09:40

RafaRuiz added 2 commits October 18, 2024 12:03

let uiimage

c1313ef

make CIFilter API available

ebae80d

juliansteenbakker mentioned this pull request Jan 12, 2025

feat: read inverted datamatrix #1071

Closed

Merge branch 'master' into master-cv

a386275

# Conflicts: # android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScannerHandler.kt

navaronbracke requested changes Jan 16, 2025

View reviewed changes

juliansteenbakker changed the base branch from master to develop January 16, 2025 08:53

juliansteenbakker and others added 4 commits January 17, 2025 13:44

fix: remove merge conflicts

0b81241

style: format

46850cd

imp: improve scan speed for inverted images

0277f9b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Ability to scan both normal codes and inverted codes #1215

feat: Ability to scan both normal codes and inverted codes #1215

RafaRuiz commented Oct 12, 2024

navaronbracke left a comment

navaronbracke Oct 14, 2024

RafaRuiz Oct 18, 2024

navaronbracke left a comment

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025 •

edited

Loading

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025

navaronbracke Jan 16, 2025

juliansteenbakker commented Jan 16, 2025

juliansteenbakker commented Jan 17, 2025

navaronbracke commented Jan 17, 2025

-  /// Set inverting image colors in intervals (for negative Data Matrices).
+  /// Enable or disable the inverting of image colors.
+  ///
+  /// This is useful when working with negative-color Data Matrices.
+  /// See also: https://en.wikipedia.org/wiki/Negative_(photography)

	// Invert every other frame.
	// Invert every other frame, by flipping the flag on every input frame.

	invertCurrentImage = !invertCurrentImage // so we jump from one normal to one inverted and viceversa
	invertCurrentImage = !invertCurrentImage

	// local variable to invert this image only this time,
	// This flag is used to determine if the current frame should be color-inverted.

feat: Ability to scan both normal codes and inverted codes #1215

Are you sure you want to change the base?

feat: Ability to scan both normal codes and inverted codes #1215

Conversation

RafaRuiz commented Oct 12, 2024

navaronbracke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

navaronbracke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

navaronbracke Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juliansteenbakker commented Jan 16, 2025

juliansteenbakker commented Jan 17, 2025

navaronbracke commented Jan 17, 2025

navaronbracke Jan 16, 2025 •

edited

Loading