Retarget Free Animation Clips onto AI-Rigged Custom Characters (Blender)

---
name: retarget-clips-to-custom-rigs
description: Retarget free animation clips (e.g. CC0 KayKit, 76 clips) onto AI-rigged custom characters (UniRig bone_N skeletons) in Blender bpy - automatic bone-role mapping from geometry, T-pose to A-pose calibration, rigid held props, dress/cape fixes, per-body-part clip mixing, prop sockets, web export. Use when custom game characters need idle/walk/attack/work/death clips without hand-animating.
---

# Retarget stock clips onto custom AI-rigged characters

Tested Oct 2026: 8 custom RTS units (villagers, swordsman, archer, spearman, two-handed knight, caped king)
rigged by UniRig (28–52 joints named `bone_0..N`), with motion from the CC0 KayKit Adventurers library
(41-joint T-pose rig, 76 clips). Blender 5.2 background mode, CPU, about 5 s per unit. Outputs validate with 0
glTF errors. This is the "motion" stage after rigging; UniRig gives you skin weights but no actions.

## Pipeline
1. Rig the static mesh (UniRig skeleton → skin → merge, GPU). Keep the rigged GLB as the master.
2. `retarget.py rigged.glb source_clips.glb role_clips.json ROLE out.glb [flags]` in `blender -b --python`.
3. Review sheet: render each clip at a mid frame from front and side (Workbench, textured) and **look at it**.
4. Web export: `gltf-transform resize --width 1024 --height 1024` → `webp` → `meshopt` (not `optimize`; see traps),
   then fix materials (metallicFactor 0, drop occlusionTexture) and validate.
5. Manifest: sha256, joints, clip durations, sockets, forward vector per unit, for the engine team.
Keep settings data-driven: `units.json` (unit → rig, role, flags) and `role_clips.json` (role → output clip → source clip).

## Map bone roles from geometry, never from bone numbers
UniRig numbering changes per model and per seed. Derive roles from topology and positions:
- **hips** = the root. **Legs** = root children going down, followed through single-child links
  (upper, lower, foot, toe). **Spine** = the root child going up, followed by highest tail.
  **Chest** = the first spine bone with ≥3 children. **Head** = the end of the spine chain.
- **Arms** = chest children not on the spine. Walk single-child links. **Hand = the first bone that branches**
  (fingers), with lower/upper arm the two bones before it. If the chain never branches (one finger chain, as on our
  king), use the **two longest consecutive bones** as upper/lower arm, and the next bone is the hand.
- **Facing** = the normal of the hip line (left-hip → right-hip × up), sign chosen so the toes point forward. Foot
  direction alone is skewed by splayed feet. **Left/right is decided at the shoulder/hip end**, never the hand:
  clasped hands cross the midline.
Bugs we hit before these rules: assuming 5 fingers (a 2-finger hand walked to a fingertip), side from hand position
(both arms classified "right"), and walking past a finger-less wrist into the fingers (arms barely moved).

## Motion transfer: world-space deltas with rest calibration
For each mapped pair, swing the source rest bone direction onto the target rest direction (min-arc). This lines up a
T-pose source with an A-pose or arms-down target:
```python
cal[s] = (yaw @ rest_dir(src, s)).rotation_difference(rest_dir(tgt, t))
# per frame, for target bones in hierarchy order:
delta = yaw @ (R_src_pose @ R_src_rest.inverted() @ yaw.inverted()) @ cal[s].inverted()
want_world = delta @ R_tgt_rest                                  # target world rotation
inherited  = parent_pose @ parent_rest.inverted() @ bone_rest     # armature space
basis      = inherited.inverted() @ desired                       # -> pose bone rotation_quaternion
```
`yaw` rotates the source facing onto the target facing. Unmapped target bones (neck, clavicles, fingers) keep their
rest pose relative to the parent. Hips translation = source hips displacement × (target leg length / source leg
length). Bake every frame of each clip into a new action; export `export_animation_mode='ACTIONS'`,
`export_force_sampling=True`. Hold the last frame of death clips in the engine.

## Held props are fused into the mesh: pin them rigid
Image-to-3D characters come with sword, shield, bow, spear and axe baked into the one mesh. UniRig spreads those
weights across forearm, hand and even legs, so blades bend like rubber.
1. **Island pass:** weld coincident vertices first (glTF splits at UV seams, so raw islands are UV charts), then find
   connected pieces. Any piece with ≥60% of its weight on one arm becomes 100% the hand bone (shields, separate bows).
2. **Fused pass:** vertices with ≥50% arm weight that are **past the wrist** along the elbow→fingertip line and
   **outside a capsule** (radius ≈ 0.35 × forearm length) are prop geometry: pin them to the hand. The past-wrist test
   matters; without it, bulky sleeves near the elbow get pinned into rigid slabs.
3. **Skip the fused pass when the hands are clasped** (hand gap < 2 upper-arm lengths, e.g. a knight holding a sword
   point-down): cloth between the hands is indistinguishable from the prop.
4. **Explicit overrides** for the rest (`props.json`: pin islands whose centroid is in a box to a role bone), e.g. an
   axe head resting behind the neck that UniRig weighted to the head.

## Clip mixing per body part
`"Walking_A|arms=REST"` walks the legs and spine while the arms keep the character's own rest pose (relative to the
chest). `"Walking_A|arms=2H_Melee_Idle"` takes the arms from another clip. This rescued:
- **Two-handed grip with cloth weighted to the hands** (knight surcoat): any clip that separates the hands tears the
  cloth, so locomotion/hit/death use `arms=REST` and the attack uses a two-handed stab.
- **Long fused props** (spear, axe across the shoulders): rest arms on locomotion; attacks become body lunges.
- **Capes weighted to the arms** (king): rest arms except on cheer/attack; drop clips that still tear (spell-cast).
Drop a clip rather than ship one that visibly tears.

## Dresses: blend thigh weights (`--skirt`)
Between ankle and hip, replace per-leg weights with hips + a lateral blend of both thighs (hips share grows toward
the waist). On our farmer this re-weighted 9,405 vertices and stopped the dress splitting between the legs on
strides. The hem still stretches on runs, which is acceptable at RTS distance.

## Sockets for tools and loads
Add empties parented to bones (`parent_type='BONE'`): `socket_hand_r/l` at the palm with +Z along the knuckle line
(index − pinky, out of the thumb side), and `socket_carry` in front of the lower chest. Give each socket the scale
`mesh_height / nominal_height_m` so meter-sized props attach with an identity transform. Verify with an attach test
render (axe in Chop, logs in CarryWalk, pickaxe in Mine).

## Traps
- **Blender 5 actions are layered:** `action.fcurves` no longer exists. Don't post-process curves; export with forced
  sampling instead.
- **`gltf-transform optimize` strips empty nodes even with `--prune false`.** Sockets vanish, so run
  resize/webp/meshopt as separate commands and assert the socket names survive.
- glTF import can add an `Icosphere` bone-shape helper; ignore it when measuring weights or bounds.
- A rigged model's rest pose may be yawed 15–22° (generated meshes aren't axis-aligned). Report the forward vector
  so the engine wrapper can correct it.
- Skeleton height ≠ mesh height (bones miss the top of the head and the soles). Normalize from mesh bounds.
- **Output clip named like its source clip = frozen pose (found 2026-10-21).** `bpy.data.actions.new('Idle')` collides
  with the loaded KayKit `Idle`, becomes `Idle.001`, the "remove actions not in clips" cleanup keeps the SOURCE and
  deletes the bake, and the exporter samples that source against the target armature (no matching bones) → a constant
  rest pose for the clip's duration. It silently froze Idle on all 8 units plus Block/PickUp/Cheer; Walk/Run/Attack were
  fine because their names differed. Bake as `__bake__<name>`, rename after the sources are deleted, and ALWAYS audit
  the export: count channels whose values change (>1° or >2 mm vs the first key) per clip; 0 moving channels = bug,
  not "subtle".

## Clip sets that worked (KayKit names)
villager: Idle, Walking_A, Running_A, CarryWalk=`Walking_A|arms=2H_Melee_Idle`, Chop=1H_Melee_Attack_Chop,
Build=1H_Melee_Attack_Slice_Diagonal, Mine=2H_Melee_Attack_Chop, Farm=1H_Melee_Attack_Slice_Horizontal,
Gather=Interact, PickUp, Hit_A, Death_A · swordsman: + Block, 1H slice/chop · archer: 2H_Ranged_Aiming/Shoot
(crossbow pose borrowed for a longbow) · knight (clasped 2H): `X|arms=REST` locomotion, 2H stab attack.
These are combat swings standing in for work animations: readable at RTS distance, not bespoke.

## Honest limits
No foot IK (slight sliding). Fused props need mesh separation in Blender for real weapon swings. One female
villager rig; male variants need an empty-handed A-pose model. Always ship the review sheets with the GLBs,
plus a per-clip moving-channel audit (see Traps) so a frozen clip cannot hide behind a static review frame.