Generate Normal Maps For Any Shot With ComfyUI and Nuke

Normal maps are stage crew. Nobody in the audience sees them... but the moment they fail, everyone notices the magic is gone. Alex just showed us how to build that invisible layer from practically nothing... and more importantly, how to make it hold still long enough to be useful.

The Problem Nobody Talks About

Every VFX compositing|VFX compositor knows the ache. You need to relight a plate. The director wants the light coming from the left instead of the right. Or a holographic glow bouncing off someone's cheek. Or a subtle fill that makes a face pop against a dark environment.

You reach for your tools... Luma to Normals, aeRelight... and the results are fine. Functional. But "fine" and "functional" don't survive a client review on a 40-foot screen.

What if you could generate a proper normal map for any plate? Any footage. Any face. Any surface.

That's what Alex walks through using a ComfyUI relighting workflow... and the results are genuinely stunning. Wrinkles, pores, the fine strands of hair catching directional light data. At 2048 resolution, the detail is almost absurd for something that required zero manual effort to create.

But here's where the pit opens up.

Beautiful Per Frame. Useless in Motion.

The AI-generated normals look incredible on any single frame. Freeze it. Admire it. Screenshot it for your reel.

Now hit play.

The whole thing chatters. Wrinkles dance. Eyes shimmer where they shouldn't. The beard looks alive in ways no beard should look alive. Frame-to-frame, the machine learning model is making independent decisions with zero memory of what it did one frame ago. There's no temporal stability|temporal coherence... just 25 brilliant strangers per second pretending to be a sequence.

This is a familiar pain for anyone working with AI-assisted VFX. The per-frame quality ceiling has gone through the roof. But consistency? That's where the craft lives.

And craft is what separates a trick from a tool.

Enter CopyCat... Two Frames and a Prayer

Here's where it gets good.

Nuke's CopyCat node is a neural network trainer built right into your compositing environment. You feed it input-output pairs... "here's what the plate looks like, here's what the normal map should look like"... and it learns the relationship.

The wild part? Alex demonstrates that you can train CopyCat on as few as two or three key frames. Pick your hero frames. Feed it the ComfyUI normals for just those moments. Let it learn.

Then run it across your entire sequence.

BAM... temporally stable normal maps. The chatter vanishes. The quality holds. The detail from the AI generation gets preserved, but now it flows frame to frame like it was always supposed to be there.

Two tools. Neither perfect alone. Together... something genuinely new.

The Practical Wisdom in the Workflow

Alex drops several workflow gems that deserve attention:

Test small, commit big. Start at 512 resolution where generation takes a second or two. Validate your setup. Then move to 1024 for speed tests (about 8 seconds per frame). Only push to 2048 or higher when you're ready to commit. At 2048, a 25-frame sequence takes just over two minutes on a NVIDIA RTX 4090|4090. The math scales from there.

Generate both matted and unmatted versions. The ComfyUI workflow includes background extraction by default... it tries to isolate the subject before generating normals. Smart in theory. But the automatic matting clips edges. Hair detail gets eaten. Generate with the matte for clean work, without it for edge preservation. Having both gives you maximum flexibility when you're deep in the compositing trenches.

The WAS Node Suite is your friend. The default ComfyUI load node handles single images. For sequences, you need the Load Image Batch node from the WAS Node Suite. Same goes for saving... the WAS Image Save node gives you proper naming conventions, frame padding, and file format control. Small details that become enormous when you're processing hundreds of frames.

Reset your increment counter. When re-running batches, the index counter keeps climbing from its last value. Forget to reset it to zero and wonder why nothing is rendering. Ask me how I know this kind of thing matters... actually, don't. Just reset it. 😅

Why This Pipeline Matters

The traditional path to normal maps for arbitrary footage was... limited. Luma to Normals conversions are fast but imprecise. Dedicated tools like aeRelight offer more control but still work from assumptions about surface geometry.

This pipeline... ComfyUI generation plus CopyCat stabilization... represents a fundamentally different approach. The AI generates normals from learned understanding of how light interacts with surfaces. The neural network in CopyCat learns temporal consistency from your specific footage. Neither is guessing based on luminance values.

The relighting applications Alex demonstrates in Nuke using PMatte and PMask with these normals show the practical payoff. Light direction, intensity, falloff... all driven by normal data that actually represents the geometry of what was filmed.

For compositors who've spent years faking relight with careful grade work and roto... this is a real shift. Not a replacement for artistic judgment. A foundation for it.

The Quiet Work

Normal maps will never trend on social media. Nobody watches a film and says, "Wow, those normals were exquisite." The work is invisible by design.

But when the light falls correctly across a face that was shot in completely different conditions... when a CG element sits in a plate and something in your gut says that's real... that's the normal map doing its job.

Quietly working. Behind the scene. Making the magic happen so someone else's performance gets to shine.

Sounds like a pretty good gig to me. ✨

If you're a compositor who's been wrestling with relighting challenges, this pipeline is worth your afternoon. Grab the ComfyUI workflow from the GitHub link Alex provides, install the WAS Node Suite, and test it at 512 before you get ambitious. Train a CopyCat session on two or three key frames. Watch the chatter disappear. Then go make something that nobody will ever know you touched... because that's exactly how it's supposed to look. 💪

--- Source: https://www.youtube.com/watch?v=CwhQ4Dl7Fn8