Neat stuff. I wonder if one could use a neural network to try to do prediction as well.
Human movements are super stereotyped in the face and you could likely predict with decent accuracy the next frame given the previous frames.
What is the SotA OSS for face tracking these days? I've never been able to make face tracking work without having alignment issues during movement.
The artistic possibilities of this are really fascinating. Looking forward to what people come up with! Projection mapping interactive art is always mesmerizing
Pretty cool. I've had this idea of converting face data into MIDI CC data to use with DAWs or synths, this could be a useful algorithm for it.
Is the source code available?
Would love to know how to build such a system but with a normal camera and projector if possible for creative art+tech projects!
Can't mediapipe detect faces at 60fps?
Is good but when you need that extra lens shift camera, it goes from mass market to super niche.
> to create immersive experiences that have been used in the makeup and entertainment industries.
Or for fake propaganda videos. Dangerous time for these kinds of progress.
Ooooh, this is so cool. I didn’t realize I was seeing projected ‘makeup’ on faces until they showed some lower latency prior work. Once I saw it, I could see occasional misalignment. But, this is just an awesome and fun thing. I can imagine it being just amazing for live theater costuming and story telling.
That said, it looks like the tech currently works at 1-2m range. I’d guess this is because that’s an easy range for a projector to work at and provide high res, in which case there’s no reason you couldn’t get a zoom setup going.
Perhaps some enterprising director of the next Disney theater production will put the money in to get this working reliably on stage.