What would be the proper way to access raw canvas bitmap data for a "flood fill" (or "paint bucket") tool?

[Offlein]

Thanks for your recent help on an unrelated question! I'm trying to build a flood fill tool (aka a "paint bucket"), such that you click a point in the drawing area and its bounds are identified and filled in with the pen color.

There is a JS algorithm called q-floodfill that looks highly performant and I thought might work well for this. (Although it has not been updated in some time -- which maybe is fine for a low-level algorithm implementation?)

Anyway, that particular algorithm takes an ImageData object, accessible from an HTML Canvas element.

I am not sure what the best way would be to generate ImageData (assuming this is a reasonable method) from the js-draw editor. I happened upon this, however:

const dataArray = editor.display
      .getDryInkRenderer()
      .getCanvasToScreenTransform()
      .toArray()

I'm not fully confident in my understanding of the DryInkRenderer vs WetInk, but I think that "WetInk" is used while the user is interacting with the drawing area (e.g. holding down a click and dragging to make a path), and as soon as that process concludes (i.e. they release the click) that "wet" ink is changed to "dry" ink. Is that right?

So I was thinking I would extend the base PenTool.onPointerUp() method to check the pixel the point the cursor was on, generate an array of data from the canvas, convert that to ImageData, feed it to q-floodfill, and somehow replace the canvas with that new data. But I'm not sure if I'm barking up the wrong tree here.

Any insight would be appreciated!

[personalizedrefrigerator]

but I think that "WetInk" is used while the user is interacting with the drawing area (e.g. holding down a click and dragging to make a path), and as soon as that process concludes (i.e. they release the click) that "wet" ink is changed to "dry" ink. Is that right?

That's correct! Unless the user is currently drawing something, the dry ink canvas contains what's currently visible on the screen.

Anyway, that particular algorithm takes an ImageData object, accessible from an HTML Canvas element.

I would suggest rendering editor.image to a new HTMLCanvasElement with CanvasRenderer and editor.viewport, though using the dry ink canvas on pointer up may also work!

For example, to render the visible content of the editor onto a canvas:

Example

import { Editor, CanvasRenderer } from 'js-draw';


const editor = new Editor(document.body);

// Adds the defualt toolbar
const toolbar = editor.addToolbar();

// Loads from SVG data
await editor.loadFromSVG(`
<svg
	viewBox="0 0 500 500"
	width="500" height="500"
	version="1.1" baseProfile="full" xmlns="http://www.w3.org/2000/svg"
>
	<path
		d="M500,500L500,0L0,0L0,500L500,500"
		fill="#aaa"
		class="js-draw-image-background"
	/>
	<text
		style="transform: matrix(1, 0, 0, 1, 57, 192); font-family: serif; font-size: 32px; fill: #111;"
	>Testing...</text>
</svg>
`);


toolbar.addSaveButton(() => {
  // Given some editor.
  // Set up the canvas to be drawn onto.
  const canvas = document.createElement('canvas');
  const ctx = canvas.getContext('2d');
  
  // Ensure that the canvas can fit the entire rendering
  const viewport = editor.viewport;
  canvas.width = viewport.getScreenRectSize().x;
  canvas.height = viewport.getScreenRectSize().y;
  
  // Render editor.image onto the renderer
  const renderer = new CanvasRenderer(ctx, viewport);
  editor.image.render(renderer, viewport);
  
  // Add the rendered canvas to the document.
  document.body.appendChild(canvas);
  
  // To make the canvas easier to see, hide the editor.
  editor.remove();
});

[Offlein]

Thank you, Henry!

Then the only part that I'd be a little unclear on is getting the filled content drawn back into the actual "image" (that is, js-draw's view of the image). It looks like with strokes that happens by doing EditorImage.addElement(...). But I'm a little unclear on what occurs then.

I see that there is a listener on AbstractComponent objects, and (Strokes appear to use that directly), which I guess is merely for self-modification. The way the, uh, pixels actually get "put on the screen" is still a bit of a mystery to me, though. It looks like it actually gets applied to "the image" (again I'm not 100% sure I understand what "the image" means, but i guess it's the Dry Ink?) is via these ImageNode things that get added via addLeaf.

But I don't think I've seen js-draw yet have any sort of handling for a structured representation of image components. That is, once something is drawn, it's on the canvas and that's it -- there doesn't seem to be a remembered z-index or a way to reorder or move around layers of image components, which is what I think I'm expecting seeing this tree-based representation.

Assuming that's correct (once the ink is dried, it's just a canvas of pixels) then I'm thinking maybe I can take the output of q-floodfill and just somehow draw it onto the root ImageNode? But I'm not clear on that. Maybe I should just overwrite t he dryink directly?

[personalizedrefrigerator]

But I don't think I've seen js-draw yet have any sort of handling for a structured representation of image components. That is, once something is drawn, it's on the canvas and that's it -- there doesn't seem to be a remembered z-index or a way to reorder or move around layers of image components, which is what I think I'm expecting seeing this tree-based representation.

I've updated the editor.image documentation to better describe how to access/modify items in the image.

Internally, editor.image is a tree (it stores components and not pixels). To make rendering faster for huge images, components are grouped by location and size:

flowchart TD
    subgraph EditorImage
        Root

        Root-->ImageNode1
        Root-->ImageNode2
        Root-->ImageNode3
        ImageNode1-->Component1
        ImageNode2-->ImageNode4
        ImageNode2-->ImageNode5
        ImageNode3-->...
        ImageNode5-->Component2
        ImageNode4-->Component3
    end

Loading

Above each of the Component1, ... Component3s would be AbstractComponents (e.g. Strokes). The ImageNodes are internal objects that wrap either other ImageNodes or Components. They're present to make rendering large images faster — if an ImageNode isn't in the viewport, then none of the Components it contains are (and they don't have to be rendered).

See the editor.image documentation for how to render this tree to a canvas and add new components to the editor.

The way the, uh, pixels actually get "put on the screen" is still a bit of a mystery to me, though. It looks like it actually gets applied to "the image" (again I'm not 100% sure I understand what "the image" means, but i guess it's the Dry Ink?) is via these ImageNode things that get added via addLeaf.

Certain operations trigger a re-render. For example, when a component is added or removed from the image. This then causes the editor to clear the dry/wet ink canvases and re-render from editor.image.

For example,

1. User creates a command that adds a component to the editor (e.g. with editor.addImage).
2. The user dispatches the command with editor.dispatch.
3. The command runs:
   • First, it modifies the internal EditorImage data structure to add the component from step 1.
   • Next (in most cases), it notifies the editor that it should re-render.
4. At some point in the near future, the editor rerenders because it was told to in step 3:
   • The editor determines which components could be visible.
   • The editor renders those components (possibly with caching if part of the image hasn't changed recently).

Assuming that's correct (once the ink is dried, it's just a canvas of pixels) then I'm thinking maybe I can take the output of q-floodfill and just somehow draw it onto the root ImageNode? But I'm not clear on that. Maybe I should just overwrite t he dryink directly?

I would suggest either:

• Creating a new Stroke with the result of the floodfill.
  • If q-floodfill outputs pixels and not paths, it may be possible to trace its output with something like imagetracerjs.
• Creating an ImageComponent that contains the pixel result of q-floodfill. This second option isn't ideal — ImageComponents currently don't work well with partial stroke erasers (and will likely be larger in size than Strokes).

[Offlein]

Thank you so much for the updates to the documentation and for the explanation. I feel I understand this pretty fully now.

The q-floodfill library does output pixels, unfortunately. I took some time before responding this to do a little research. A tool like imagetracerjs might work, but q-floodfill uses a scanline algorithm for calculating this, and seems like it is, honestly, reasonably well-suited to [optionally] build an actual 2D Path as it goes.

For my own purposes, I'm building something that will not use js-draw's exceptional handling for zoom and large canvas sizes, and I suspect generating ImageComponents would be more than suitable for my needs.

...However, I'm curious if the project has any need for an included Flood Fill tool. Because if so I could look into also modifying the q-floodfill library (or some other library... Or forking it.) and using it to generate an actual Stroke, but of course that introduces dependencies to a possibly-dead module in your code. (FWIW, his entire implementation seems to be a single 178 line file and you both use the MIT license, so it could also be copied over wholecloth.

[personalizedrefrigerator]

...However, I'm curious if the project has any need for an included Flood Fill tool. Because if so I could look into also modifying the q-floodfill library (or some other library... Or forking it.) and using it to generate an actual Stroke, but of course that introduces dependencies to a possibly-dead module in your code. (FWIW, his entire implementation seems to be a single 178 line file and you both use the MIT license, so it could also be copied over wholecloth.

I think it would be useful to have floodfill built-in. However, I also would like:

• Not to add new dependencies (if possible).
• To have an algorithm that creates Strokes, rather than pixels. This would probably mean not using q-floodfill (or modifying it) and instead following some other approach.
  • This would allow, for example, floodfilled regions to work better with other parts of the library. For example, ImageComponents are currently embedded as base64 URLs when saving to SVG. Adding a floodfilled region can thus significantly increase the size of an otherwise small image.