-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mutable ImageData like JavaScript's ImageData #2870
Comments
Just realized I could pass I assume there's nothing to be gained from calling I'm still wondering why the only way to access |
Can you clarify whether you got something like this to work, so you could modify the data? I don't follow your explanation from Apr 21 about what you realized you could do, or what settled your problem. It seems like you're saying you can't do this from Rust via ctx.put_image_data(). I've been trying similar things and so far my image data is effectively read-only. |
Sure. I forgot what I ended up doing exactly, but what I meant by that was I realized it was possible to pass a I learned that there was no trickery to be had with the double buffering and that you had to copy the image data over, but it's been working okay for me so far. |
Thanks for that. I've ended up for the moment with a different approach, where I'm repeatedly modifying a stored Vec<[u8]>, and then doing |
@alexcrichton @s1gtrap @peter9477 Hello, could you help me out with a vaguely related issue please? If not I can open a new question. I have an HtmlCanvasElement, associated CanvasRenderingContext2d and ImageData. All created in rust. I need to do some calculations with the pixel values after using the context to draw. The problem is, calling methods such as .fill or .fill_rect using the context does not seem to modify my ImageData. It seems get_image_data is returning a 'snapshot', how do I maintain a reference to the underlying vector? I have also attempted storing a Vec<u8> (by calling .data().to_vec()) but had the same issue. My current workaround is to keep replacing my ImageData with a fresh copy by calling context.get_image_data after drawing, but this is very costly (in a loop, part of a rendering engine I'm working on). TLDR: how do I avoid having to constantly call get_image_data? Here's a very simplified example with a 1x1 canvas: let testCanvas = create_canvas();
resize_canvas(&testCanvas, 1, 1);
let testCtx = get_context(&testCanvas);
let testData: ImageData = testCtx.get_image_data(0.0, 0.0, 1.0, 1.0).unwrap();
console::log_1(&format!("Test data before : {:?}", &testData.data().to_vec()).into());
testCtx.set_fill_style(&JsValue::from("#fff"));
testCtx.fill_rect(0.0, 0.0, 1.0, 1.0);
// Shouldn't have to do this, I expect testCtx.fill_rect to have modified testData
// FIXME: how to avoid calling calling get_image_data?
testData = testCtx.get_image_data(0.0, 0.0, 1.0, 1.0).unwrap();
console::log_1(&format!("Test data after : {:?}", &testData.data().to_vec()).into());
fn create_canvas() -> HtmlCanvasElement {
let window = web_sys::window().expect("no global `window` exists");
let document = window.document().expect("should have a document on window");
document
.create_element("canvas")
.unwrap()
.dyn_into::<HtmlCanvasElement>()
.unwrap()
}
fn resize_canvas(canvas: &HtmlCanvasElement, w: u32, h: u32) {
canvas.set_width(w);
canvas.set_height(h);
}
fn get_context(canvas: &HtmlCanvasElement) -> CanvasRenderingContext2d {
let opts = js_sys::Object::new();
js_sys::Reflect::set(&opts, &"willReadFrequently".into(), &true.into()).unwrap(); // not sure if this works either
canvas
.get_context_with_context_options("2d", &opts)
.unwrap()
.unwrap()
.dyn_into::<web_sys::CanvasRenderingContext2d>()
.unwrap()
} Ouput:
Is this because I need to be storing a #[wasm_bindgen()]
pub struct Engine<'a> { // error: structs with #[wasm_bindgen] cannot have lifetime or type parameters
...
testData: &'a ImageData,
} Apologies if I'm missing something obvious, I'm new to rust. |
If I understood it correctly you intend to edit what is on screen by directly editing the I'm afraid it wouldn't work like that as the Do you need to use the
|
Yea I highly doubt you'd get away with that for the time being (interesting proposal over at whatwg/html#5173). It's not even a rust thing, just a matter of limited memory access for obvious reasons. Looks like you need to reconsider your need for canvas rendering along with rust data access as the two seemingly don't play nice together. |
I see, that's unfortunate.
Any good options other than implementing my own fill_rect? 😕 |
Afraid not.. At least not from me at the moment. In the end I opted for copying from rust with |
I think not all your steps are needed. Here is some code snippets that I put together using these ideas. It worked on my machine (TM). JS only codeconst draw = () => {
let imageData = canvas.getContext('2d').getImageData(0, 0, canvas.width, canvas.height);
let pixelData = imageData.data;
// Iterate through the pixels of the RGBA image.
for (let i = 0; i < pixelData.length; i += 4) {
// No need to draw on all
if ((i / 4) % 10 === 0) {
pixelData[i] = 0; // Red component
pixelData[i + 1] = 255; // Green component
pixelData[i + 2] = 0; // Blue component
// The alpha component (pixelData[i + 3]) remains unchanged
}
}
canvas.getContext('2d').putImageData(imageData, 0, 0);
}; Rust + WASMSame as pure JS, but the import init, * as wasm from './pkg/my_package.js';
async function drawWasm() {
// Instantiate the WebAssembly module
await init("./pkg/net_bg.wasm");
let imageData = canvas.getContext('2d').getImageData(0, 0, canvas.width, canvas.height);
let pixelData = imageData.data;
wasm.draw(pixelData);
canvas.getContext('2d').putImageData(imageData, 0, 0);
} The Rust part with your #[wasm_bindgen]
pub fn draw(pixels: &mut [u8]) {
let count = pixels.len();
for i in (0..count).step_by(4) {
if (i / 4) % 10 == 0 {
pixels[i] = 255; // Red component
pixels[i + 1] = 255; // Green component
pixels[i + 2] = 0; // Blue component
}
}
} However, it feels to me that the documentation of Clam suggests that the input type of the rust function should not be All in all, thanks for your input. I can now finally progress with my project. |
Summary
I'm really confused about the
ImageData
struct. Is it meant to be read-only?An example from MDN ought to also be possible with WASM, right?
But obviously, since
ImageData::data(&self)
produces aClamped<Vec<u8>>
the data is copied to a freshVec
from the underlyingUint8ClampedArray
and not writable like in the JS example above, so unless I misunderstood something entirely?Additional Details
I'm hoping to make calls to WASM for somewhat efficient rendering but having to allocate a new
ImageData
every frame sort of defeats the purpose. Knowing howImageData
worked on the JS side of things I was expecting to be able to modify one allocated beforehand like sowith something like
So is it possible to modify the underlying byte array? I know the
data
field is marked as read-only but modifying said data isn't.(I see a reference to
Clamped<&mut [u8]>
on the page forClamped
but as far as I can tell it's not possible to get from anImageData
)The text was updated successfully, but these errors were encountered: