Problems with installing node-webcam

[YaraAmin]

I'm trying to follow your example for the demo but I'm facing problems with installing node-webcam with a React app using require("node-webcam") and it erroring out. also, I got errors with var OS = require( "os" ); and some other require functions.
any tips for that?

[vladmandic]

ReactJS and/or NextJS are very specific setups, search previously closed issued, there are several ways to approach this.
But what are you trying to accomplish? React is a front-end framework and node-webcam.js is a NodeJS demo?

[YaraAmin]

I'm trying to use Human on stream, so I can get the best usage of caching using multiple frames per second instead of just a screenshot every second from the camera stream
I will check the previous discussions

[YaraAmin]

is there any way to get the needed input for Human using React-webcam? If you have any idea it will be more than appreciated, thanks

[vladmandic]

never used react-webcam, but took a quick look at the sources - it sets its public ref property to this.video which is an actual HTMLVideoElement - i dont see why not use ref as input for human.detect

[YaraAmin]

yeah, that's what I use right now, please correct me if I'm wrong, you're using a buffer from the stream then convert it to tensor then passing it to Human, right?
I mean: will caching be working if I'm running human. detect every second on one frame? and also the configurations of skipping frames, will it still be working in this case?

[vladmandic]

no, i tried that but decoding stream in JS is too slow. Fastest way depends on many thing, so there is no single answer

assuming modern browser which does accelerated stream decoding (far faster than any JS could do) and config.filter=enabled (default) and config.backend is webgl or humangl (default) then its:

• freeze input into temp canvas using small GL program, this is near-instant as it happens inside GPU
• copy canvas data into tensor using another GL program, also inside GPU so its near-instant
• process resize, reshape and normalize tensor, also inside GPU
  all this happens in <1ms

but can slow down a lot - for example:

• if filter is disabled we need to use canvas draw,
• if backend is wasm we need to use imageData read

regarding caching, sklpFrame doesnt care how much time passed between frames. nor it cares what is the input type at all. you can use caching on set of images if you want.

but it does check if input is similar enough by resizing it to a tiny image and running checksum on it - if checksum difference is below cacheSensitivity then skipFrame may happen. so setting cacheSensitivity=0 means nothing will pass the check and cacheSensitivty=1 means everything will pass the check

what does each model do with its skipFrame is up to the model. For example, for emotion it means just return cached value, but for face is means recalculate face box based on last face position and try to analyze that that without running face detector

where time between frames matters is results interpolation - if you're using human.next (strongly recommended), then it converges from previous result to current result at the speed depending how much time has passed (more time passed and interpolation will be closer to current result)

[YaraAmin]

Thank you so much for your detailed reply!
Regarding the backend, Do you recommend a specific one? thanks

[vladmandic]

See https://github.com/vladmandic/human/wiki/Backends

[YaraAmin]

Thanks!