Add Copy text from image to image context menu (Windows)

[bsclifton]

Description

Exact same feature / capability as #27513 (macOS) - but on Windows and Linux

Unlike macOS, there isn't a clear API to use. That'll be the challenging part of this task

[luquinhaspmw]

is it working on windows?

[bsclifton]

@luquinhaspmw no, this feature is not available on Windows (yet). There are some leads we have about something which might work. I'll add those to the ticket soon here

[simonhong]

Investigations on Windows

It seems chromium already has the TextDetection::Detect interface(textdetection.mojom) via ShapeDetectionService service (shape_detection_service.mojom).
Maybe this service is for W3C Accelerated Text Detection in Images spec and it seems not enabled yet testing with code pen.

To test that service, we need to launch ShapeDetectionService and bind TextDetection.
After that, we could call TextDetection::Detect(skia.mojom.BitmapN32 bitmap_data) to get text from bitmap.

shape_detection::mojom::ShapeDetectionService* GetShapeDetectionService() {
  static base::NoDestructor<
      mojo::Remote<shape_detection::mojom::ShapeDetectionService>>
      remote;
  if (!*remote) {
#if BUILDFLAG(GOOGLE_CHROME_BRANDING) && BUILDFLAG(IS_CHROMEOS)
    ServiceProcessHost::Launch<shape_detection::mojom::ShapeDetectionService>(
        remote->BindNewPipeAndPassReceiver(),
        ServiceProcessHost::Options()
            .WithDisplayName("Shape Detection Service")
            .Pass());
#else
    auto* gpu = GpuProcessHost::Get();
    if (gpu)
      gpu->RunService(remote->BindNewPipeAndPassReceiver());
#endif
    remote->reset_on_disconnect();
  }

  return remote->get();
}

void BindTextDetection(
    mojo::PendingReceiver<shape_detection::mojom::TextDetection> receiver) {
  GetShapeDetectionService()->BindTextDetection(std::move(receiver));
}

As we can see in GetShapeDetectionService(), there are two ways to launch ShapeDetectionService.
First one is launching in separated utility service and the other is in gpu process.
There is constaint that this service should be run in gpu process as has_shape_detection_utility is only defined at (services\shape_detection\public\mojom\BUILD.gn) for chromeos.

[EnableIf=has_shape_detection_utility]
const sandbox.mojom.Sandbox kShapeDetectionSandbox = sandbox.mojom.Sandbox.kUtility;
[EnableIfNot=has_shape_detection_utility]
const sandbox.mojom.Sandbox kShapeDetectionSandbox = sandbox.mojom.Sandbox.kGpu;

const sandbox.mojom.Sandbox kShapeDetectionSandbox = sandbox.mojom.Sandbox.kGpu;

There is no easy way to use GpuProcessHost::Get(); outside of content layer.
Also only en text is detected by shape detection service.
So, trying to use the impls of TextDetectionImplWin directly in browser process.

Tested with Windows::Media::Ocr APIs and there is a difference with macOS VNRecognizeTextRequest API.
With macOS, we can set target languages and it tries to check all that target languages from image.
However, we need to set specific target language when creating ocr engine or can set user profile's language implicitely.
If image doesn't have that target language, ocr engine doesn't detect anymore.
Maybe we have to check all our target languages and need to compare its results.
and the result with the largest number of detected test could be the appropriate target language.

[MadhaviSeelam]

QA passing this issue as #30203 is filed as a follow up.