---
name: materia-nuke-node
description: >
  Build Nuke VFX plugin nodes for Materia (Biohazard VFX). Use when creating
  ML inference nodes, gizmos, or Cattery plugins for Foundry Nuke. Covers
  TorchScript model wrapping, .cat file compilation, gizmo design, knob
  definitions, and plugin distribution. Triggers: "nuke node", "nuke plugin",
  "nuke gizmo", "materia node", "cattery plugin", "inference node",
  "torchscript for nuke", "depth estimation node", "ML node for nuke",
  "wrap model for nuke", "VFX plugin", "compositing node".
---

# Materia Nuke Node Builder

Build ML inference nodes for Foundry Nuke as part of the Materia VFX tools
suite by Biohazard VFX.

## Architecture Decision

Always use Cattery + TorchScript + Inference node. NOT nuke-ML-server, NOT
pure Python pixel processing, NOT custom NDK from scratch.

- Zero deps for end users (drop .gizmo + .cat into Cattery/)
- Nuke's Inference node handles GPU/CPU, precision, batch automatically
- TorchScript models are portable across platforms
- Official Foundry-supported pattern

## Workflow

1. Read the model's source code thoroughly BEFORE writing wrapper code
2. List state_dict keys from a pretrained checkpoint to understand module hierarchy
3. Design TorchScript wrapper matching exact module names for weight loading
4. Write compilation script (download weights, wrap, trace, save .pt)
5. Build gizmo (node UI + internal processing pipeline)
6. Convert .pt to .cat via Nuke's CatFileCreator
7. Test in Nuke

## TorchScript Constraints (critical)

- ONLY: torch ops, python builtins, math module
- NO: numpy, opencv, einops, xformers, addict, PIL
- All class attributes initialized in __init__
- No dynamic typing (variables can't change type)
- Use int not bool
- F.interpolate: size=(h, w) not size=shape[2:]
- Replace xformers -> torch.nn.functional.scaled_dot_product_attention
- Replace einops rearrange -> native torch reshape/permute/contiguous
- Replace addict.Dict -> plain dicts or baked constants

## Common Failure Modes

- Module hierarchy mismatch: wrapper names don't match checkpoint keys, strict=False silently loads nothing
- Missing LayerScale: modern ViTs use learnable gamma*x after attn/mlp -- skip it and weights won't load
- Wrong feature extraction layers: each model variant has different intermediate layer indices, read the yaml config
- Wrong activation: exp vs sigmoid vs relu -- read the model config, don't guess
- Gizmo enum expressions: use numeric index (0, 1, 2) not string comparison
- sRGB conversion: use Nuke Colorspace node, not ColorMatrix (sRGB EOTF is nonlinear)

## Reference Implementation Locations

- DA2 plugin (working): /mnt/work/dev/materia/Depth-Anything-for-Nuke/
- DA3 plugin (in progress): /mnt/work/dev/materia/Materia-DepthAnythingV3/
- Foundry research report: /mnt/work/dev/materia/foundry-nuke-research-report.md

## Detailed References

- [Cattery pattern and gizmo design](references/cattery-pattern.md)
- [TorchScript wrapper patterns](references/torchscript-patterns.md)

## Repo Template

```
MateriaNodeName/
├── README.md
├── SPEC.md
├── LICENSE (Apache 2.0)
├── .gitignore
├── nuke/Cattery/NodeName/
│   ├── NodeName.gizmo
│   ├── NodeName.cat
│   ├── cat.json
│   └── NodeName.png
├── compile/
│   ├── wrapper.py
│   ├── nuke_compile.py
│   ├── requirements.txt
│   └── docker/Dockerfile
├── models/.gitkeep
├── tests/
└── docs/
```
