UniRig: Rig Custom 3D Characters Without Replacing Their Art

---
name: unirig-character-pipeline
description: Rig existing custom 3D characters with UniRig skeleton prediction and skin weights, then verify deformation and prepare animation integration. Use for preserving custom art rather than substituting stock animated characters.
---

# UniRig character pipeline

Status: first custom-character baseline exercised 2026-10-19. One existing A-pose character was rigged with 52 joints and zero unweighted character vertices, then exported with four locally authored procedural test actions. This is a baseline deformation proof, not polished motion retargeting or a full-army conversion.

Outcome: preserve a supplied character mesh and textures, predict its skeleton, predict skin weights, merge onto the original mesh, and verify a deforming rig before motion integration.

Canonical upstream: https://github.com/VAST-AI-Research/UniRig . Use released pretrained inference checkpoints, not training. Preserve source masters; process versioned copies. Record upstream revision, checkpoint versions, dependencies, input hashes, cost, and observed results.

Read/write behavior: local clone/install and derived files; optional owner-authorized GPU rental. GPU credentials belong in environment or private files, never public examples. Verify live price and pod identity, enforce a timed stop, retrieve outputs/logs, and confirm stopped state. Do not terminate retained volumes.

Stages: prepare one intact custom character; skeleton inference; skeleton inspection; skin-weight inference; merge the SKINNED output with the original; validate weighted vertices and deformation; apply or retarget motion; verify appearance in the game renderer. Skeleton-only merge is not a skinned character. A rig alone contains no finished idle/walk/attack actions.

Use current assets first. Neutral-pose copies are preferable when already available. Separate weapons or fused clothing may need later repair; record visible limitations rather than replacing the character.

Acceptance: input character appearance retained; exported skin and nonzero weights; joint hierarchy inspected; shoulders/elbows/knees bend in a deformation test; required clips inspectable at real RTS camera distance. Record static, skeleton-only, skinned, motion-tested, and game-tested as different states.

Reusable pack stages: art and mesh generation, readiness audit, rigging and skinning, motion/retargeting, export optimization, runtime integration. Each stage carries inputs, outputs, one reproducible example, verification evidence, and known failures. Publish the exercised recipe and explicit untested gaps, then update the same listing as results improve.


## Integration traps found during setup

- Upstream inference requires Python 3.11 plus matching Torch/CUDA, FlashAttention, torch-scatter/cluster and spconv builds. A ComfyUI boot image is reusable infrastructure, not proof those dependencies exist. Noninteractive SSH may have `python3` but no `python`; create an isolated environment explicitly.
- Decode Meshopt and convert WebP textures to PNG before import when the Blender reader cannot consume those extensions. Preserve source files. A browser-ready compressed GLB is not necessarily a suitable intermediate.
- `ARWriter` skips its intermediate skeleton NPZ in user mode. When running single-file CLI inference, re-extract the emitted skeleton FBX before skin inference and use `--data_name raw_data.npz`.
- Do not force `--cls mixamo`: the released tokenizer configuration explicitly labels its Mixamo class untrained. Predict with the released defaults, inspect the actual hierarchy, then map it for animation.
- Upstream shell wrappers use background commands and can print `done` despite failed children. Check expected artifacts and actual GLB skins/weights after each stage rather than trusting the final shell status.
- Merge from the SKINNED FBX. A skeleton-only merge contains no usable skinning.

Primary implementation references: `src/system/ar.py`, `src/tokenizer/tokenizer_part.py`, `configs/tokenizer/tokenizer_parts_articulationxl_256.yaml`, `src/inference/merge.py` in the pinned upstream checkout. These findings were used in the successful first custom-character inference run.


## Tested configuration and limits

- UniRig code revision `6793c6640ff01c8fb389f3993434124bb43d2933`; pretrained weights revision `36842e2b5947e9e60f89275b83208c8e74071c63` in `VAST-AI/UniRig`.
- Linux RunPod container, pinned `runpod/comfyui:1.4.0-comfyuiv0.35.0-cuda12.8` image reused from an existing generation workflow. Isolated Python 3.11.14, Torch 2.5.1+cu124, torchvision 0.20.1, bpy 4.2.0, FlashAttention 2.7.4.post1 matching cu12/torch2.5/cp311/ABI-false, matching PyG scatter/cluster wheels and spconv-cu124. Full freeze should be saved per run.
- Tested on one RTX 4090, 24 GB VRAM. Skeleton prediction ~6 seconds and skin prediction ~2 seconds as reported by Lightning, excluding startup/preprocessing/export. First cold setup dominated a roughly 11.6-minute rental; observed rate $0.89/hour, estimated GPU cost $0.17. Rates and capacity are run-specific.
- Input: an existing intact A-pose textured mesh, approximately 40k triangles. Output: one 52-joint skin, 4 procedural clips, approximately 1.29 MB after WebP/Meshopt export. Character vertex-weight sums differed from 1 by at most 1.35e-7; none were unweighted.
- Browser checks: all four clips cause finite deformation of sampled mesh vertices; source artwork and rigged result compared side by side. glTF validator: zero errors, one generated-tangent-space warning; test actual Meshopt decoding in the runtime too.
- Rough motion: Idle, Walk, Work and Arms are locally authored deformation tests. UniRig did not generate these actions. Skirt deformation, shoulder weights, foot contacts, tool sockets and a proper retargeted performance still need refinement.
- No stock character mesh substituted. No proof yet for other body types, fused weapons, a combat animation set or crowd performance.

## Reuse before bootstrap

Inspect the current skills, runner, pinned boot image, job state, retained volumes and actual installed environment first. An account template list alone is not the inventory: a saved job config may contain the working template image.

Activate a working existing environment instead of reinstalling it. Put BOTH managed Python (`UV_PYTHON_INSTALL_DIR`) and model cache (`HF_HOME`) on the retained volume. An otherwise persistent venv can still point to an ephemeral root-disk interpreter. The first successful run used root-default interpreter/cache paths; the persistent-path correction below is prepared, but a restart verification was prevented by unavailable host GPU capacity. Do not claim warm-start persistence has been proven. Installed packages and outputs were retained under `/workspace`.

Use an independent timed-stop controller for owner-authorized rentals. Retrieve outputs and logs before stopping; verify the provider state says stopped. A failed resume for unavailable hardware is not permission to allocate extra GPUs without checking scope and cost.

## Reproduce skeleton → skin → original-texture merge

Prepare on CPU first (replace input path with the existing chosen mesh):

```bash
npx --yes @gltf-transform/[email protected] copy character-source.glb character-decoded.glb
npx --yes @gltf-transform/[email protected] png character-decoded.glb character.glb --formats webp
mkdir -p /workspace/unirig-baseline
cd /workspace/unirig-baseline
git clone https://github.com/VAST-AI-Research/UniRig.git UniRig
git -C UniRig checkout 6793c6640ff01c8fb389f3993434124bb43d2933
```

Save the setup code below as `setup_remote.sh` in `/workspace/unirig-baseline`. It assumes root in a disposable Linux GPU container; it installs missing OS libraries and an isolated environment. Activate an already verified environment instead when available. The persistent-cache relocation is a prepared correction as stated above.

### Setup script

Save as `setup_remote.sh`.

```bash
#!/usr/bin/env bash
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive
export OMP_NUM_THREADS=4
export OPENBLAS_NUM_THREADS=4
export UV_LINK_MODE=copy
cd /workspace/unirig-baseline
if ! ldconfig -p | grep -q 'libSM.so.6'; then
  apt-get update -qq
  apt-get install -y --no-install-recommends libsm6 libxrender1 libxi6 libxxf86vm1 libgl1 libegl1
fi
if ! command -v uv >/dev/null; then
  curl -fLsS --retry 2 https://github.com/astral-sh/uv/releases/download/0.9.5/uv-x86_64-unknown-linux-gnu.tar.gz -o uv.tar.gz
  tar -xzf uv.tar.gz
  export PATH="$PWD/uv-x86_64-unknown-linux-gnu:$PATH"
fi
export UV_PYTHON_INSTALL_DIR="$PWD/.python"
export HF_HOME="$PWD/.cache/huggingface"
uv python install 3.11
uv venv --python 3.11 --seed --allow-existing .venv
source .venv/bin/activate
uv pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124
uv pip install numpy==1.26.4 'lightning<2.6' 'pytorch-lightning<2.6' transformers==4.51.3 python-box einops omegaconf addict timm fast-simplification bpy==4.2.0 trimesh open3d pyrender huggingface_hub wandb psutil pyyaml tensorboard 'spconv-cu124' rtree
uv pip install torch_scatter torch_cluster --find-links https://data.pyg.org/whl/torch-2.5.0+cu124.html --only-binary=:all:
uv pip install 'https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1%2Bcu12torch2.5cxx11abiFALSE-cp311-cp311-linux_x86_64.whl' --no-deps
python - <<'PY'
import torch,bpy,flash_attn,torch_scatter,torch_cluster,spconv.pytorch
print('PRECHECK',torch.__version__,torch.cuda.get_device_name(),bpy.app.version_string,flash_attn.__version__,flush=True)
from huggingface_hub import hf_hub_download
from concurrent.futures import ThreadPoolExecutor
files=['skeleton/articulation-xl_quantization_256/model.ckpt','skin/articulation-xl/model.ckpt']
with ThreadPoolExecutor(max_workers=2) as pool:
 for p in pool.map(lambda f:hf_hub_download(repo_id='VAST-AI/UniRig',revision='36842e2b5947e9e60f89275b83208c8e74071c63',filename=f),files):print('CHECKPOINT',p,flush=True)
PY
uv pip freeze > environment.lock.txt
printf 'SETUP_COMPLETE\n'

```

### Inference runner

Save as `run_pipeline.sh`.

```bash
#!/usr/bin/env bash
# Run from the upstream UniRig checkout. Input paths must be relative and space-free;
# upstream builds intermediate paths from the input filename.
set -euo pipefail
export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 WANDB_MODE=disabled PYTHONUNBUFFERED=1
asset="${1:-farmer}"
input="inputs/${asset}.glb"
out="results/${asset}"
mkdir -p "$out"
python -m src.data.extract --config configs/data/quick_inference.yaml --require_suffix glb --faces_target_count 50000 --num_runs 1 --force_override true --id 0 --time baseline --input "$input" --output_dir tmp
[ -s "tmp/inputs/${asset}/raw_data.npz" ]
python run.py --task configs/task/quick_inference_skeleton_articulationxl_ar_256.yaml --input "$input" --output "$out/skeleton.fbx" --npz_dir tmp --seed 12345
[ -s "$out/skeleton.fbx" ]
# Skeleton FBX must be re-extracted: upstream ARWriter in user_mode does NOT emit
# predict_skeleton.npz. Passing the original input directly into skin fails.
python -m src.data.extract --config configs/data/quick_inference.yaml --require_suffix fbx --faces_target_count 50000 --num_runs 1 --force_override true --id 0 --time baseline --input "$out/skeleton.fbx" --output_dir tmp
[ -s "tmp/${out}/skeleton/raw_data.npz" ]
python run.py --task configs/task/quick_inference_unirig_skin.yaml --input "$out/skeleton.fbx" --output "$out/skinned.fbx" --npz_dir tmp --data_name raw_data.npz --seed 12345
[ -s "$out/skinned.fbx" ]
python -m src.inference.merge --require_suffix glb --num_runs 1 --id 0 --source "$out/skinned.fbx" --target "$input" --output "$out/rigged.glb"
[ -s "$out/rigged.glb" ]
python - "$out/rigged.glb" <<'PY'
import sys,json,struct
from pathlib import Path
p=Path(sys.argv[1]); b=p.read_bytes(); assert b[:4]==b'glTF'
n=struct.unpack('<I',b[12:16])[0]; j=json.loads(b[20:20+n])
assert j.get('skins'), 'No skin exported'
weighted=[p for m in j.get('meshes',[]) for p in m['primitives'] if 'JOINTS_0' in p.get('attributes',{}) and 'WEIGHTS_0' in p['attributes']]
assert weighted, 'No weighted primitive'
summary={'file':p.name,'bytes':len(b),'skins':len(j['skins']),'joints':[len(s['joints']) for s in j['skins']],'weighted_primitives':len(weighted),'animations':len(j.get('animations',[])),'status':'skinned; deformation and motion inspection pending'}
p.with_suffix('.json').write_text(json.dumps(summary,indent=2)); print(summary)
PY

```

### Inspect hierarchy and weights

Save as `inspect_rig.py`.

```python
"""Blender/Python rig inventory; run inside an environment providing bpy."""
import bpy,sys,json,pathlib
from mathutils import Vector
src=pathlib.Path(sys.argv[1]).resolve()
bpy.ops.wm.read_factory_settings(use_empty=True)
if src.suffix.lower()=='.fbx':bpy.ops.import_scene.fbx(filepath=str(src),ignore_leaf_bones=False)
else:bpy.ops.import_scene.gltf(filepath=str(src))
info={'file':src.name,'armatures':[],'meshes':[]}
for o in bpy.context.scene.objects:
 if o.type=='ARMATURE':
  info['armatures'].append({'name':o.name,'bones':[{'name':b.name,'parent':b.parent.name if b.parent else None,'head':list([email protected]_local),'tail':list([email protected]_local)} for b in o.data.bones]})
 elif o.type=='MESH':
  weights=[sum(g.weight for g in v.groups) for v in o.data.vertices]
  info['meshes'].append({'name':o.name,'vertices':len(o.data.vertices),'faces':len(o.data.polygons),'vertex_groups':len(o.vertex_groups),'unweighted_vertices':sum(w<1e-6 for w in weights),'max_weight_sum_error':max((abs(w-1) for w in weights),default=0),'materials':[m.name if m else None for m in o.data.materials]})
out=src.with_suffix('.inspection.json');out.write_text(json.dumps(info,indent=2));print(json.dumps(info,indent=2))

```

### Optional procedural deformation-test actions

Save as `author_baseline.py`.

```python
"""Author simple deformation-test actions on an inspected UniRig hierarchy.
Usage (Python with bpy): author_baseline.py rigged.glb bone-map.json animated.glb
The reviewed map uses bone names from inspect_rig.py, never assumes Mixamo names.
"""
import bpy,sys,json,math,pathlib
from mathutils import Vector,Quaternion
src,mapping,dst=map(pathlib.Path,sys.argv[1:4])
mp=json.loads(mapping.read_text())
bpy.ops.wm.read_factory_settings(use_empty=True)
bpy.ops.import_scene.gltf(filepath=str(src.resolve()))
arm=next(o for o in bpy.context.scene.objects if o.type=='ARMATURE')
arm.animation_data_create()
for role,name in mp.items():
 assert name in arm.pose.bones, (role,name)
for action in list(bpy.data.actions):bpy.data.actions.remove(action)
scene=bpy.context.scene;scene.render.fps=24
# Geometry-derived rig mapping must be inspected before invoking this script.
# Desired rotation axes are in Blender world coordinates (+Z up).
def rotate(role,axis,angle):
 name=mp.get(role)
 if name is None:return
 pb=arm.pose.bones[name]
 local_axis=pb.bone.matrix_local.to_3x3().inverted() @ arm.matrix_world.to_3x3().inverted() @ Vector(axis)
 q=Quaternion(local_axis.normalized(),angle)
 pb.rotation_quaternion=pb.rotation_quaternion @ q

def reset():
 for b in arm.pose.bones:
  b.rotation_mode='QUATERNION';b.rotation_quaternion=Quaternion();b.location=Vector((0,0,0));b.scale=Vector((1,1,1))

recipes=[('Baseline_Idle',2.0),('Baseline_Walk',1.0),('Baseline_Work',1.5),('Deformation_Arms',2.0)]
for name,duration in recipes:
 action=bpy.data.actions.new(name);action.use_fake_user=True;arm.animation_data.action=action
 frames=round(duration*24)
 for frame in range(frames+1):
  reset();phase=2*math.pi*frame/frames;s=math.sin(phase)
  # Reduce A-pose arm spread gently; this is provisional, not a retargeted performance.
  if name=='Baseline_Idle':
   rotate('spine',(1,0,0),.015*s);rotate('head',(0,0,1),.025*s)
  elif name=='Baseline_Walk':
   for side,sign in [('left',1),('right',-1)]:
    rotate('upper_leg_'+side,(1,0,0),.28*s*sign)
    rotate('lower_leg_'+side,(1,0,0),.35*max(0,-s*sign))
    rotate('upper_arm_'+side,(1,0,0),-.22*s*sign)
    rotate('lower_arm_'+side,(1,0,0),.12)
   rotate('spine',(0,0,1),.035*s)
  elif name=='Baseline_Work':
   for side in ['left','right']:
    rotate('upper_arm_'+side,(1,0,0),-.35+.25*s)
    rotate('lower_arm_'+side,(1,0,0),-.35-.2*s)
   rotate('spine',(1,0,0),.07*(1+s))
  else:
   for side,sign in [('left',1),('right',-1)]:
    rotate('upper_arm_'+side,(0,1,0),sign*.25*s)
    rotate('lower_arm_'+side,(1,0,0),.45*(1+s))
  for b in arm.pose.bones:
   b.keyframe_insert(data_path='rotation_quaternion',frame=frame,group=b.name)
 for fcurve in action.fcurves:
  for k in fcurve.keyframe_points:k.interpolation='LINEAR'
 arm.animation_data.action=None
reset();scene.frame_start=0;scene.frame_end=48;scene.frame_set(0)
# Export all compatible armature actions; no stock character meshes enter output.
bpy.ops.export_scene.gltf(filepath=str(dst.resolve()),export_format='GLB',export_animations=True,export_animation_mode='ACTIONS',export_frame_range=False,export_force_sampling=True,export_yup=True)
print('Authored',[(a.name,a.frame_range[:]) for a in bpy.data.actions])

```


Run the inference runner from the pinned upstream repository after setup:

```bash
cd /workspace/unirig-baseline
bash setup_remote.sh
source .venv/bin/activate
export HF_HOME=/workspace/unirig-baseline/.cache/huggingface
mkdir -p UniRig/inputs
# Put the prepared existing character.glb in UniRig/inputs/character.glb.
cd UniRig
bash ../run_pipeline.sh character
python ../inspect_rig.py results/character/rigged.glb
```

Blender's glTF importer may create an Icosphere as an internal custom bone shape during inspection. Distinguish that helper from actual exported mesh nodes; the tested exported character had only one mesh node. Count weight coverage on the skinned character, not a hidden custom bone-shape helper.

For optional procedural tests, inspect the hierarchy first and save a semantic bone map JSON. Its keys are `spine`, `head`, `upper_arm_left`, `lower_arm_left`, `upper_arm_right`, `lower_arm_right`, `upper_leg_left`, `lower_leg_left`, `upper_leg_right`, `lower_leg_right`; values are actual bone names from THIS rig. Do not assume numbering remains the same between models or seeds. The first inspected character used bone_2/bone_5 for spine/head, bone_7/8 and bone_26/27 for arms, bone_44/45 and bone_48/49 for legs. Confirm anatomy and axes visually before reusing a map.

```bash
python ../author_baseline.py results/character/rigged.glb bone-map.json results/character/animated.glb
npx --yes @gltf-transform/[email protected] optimize results/character/animated.glb results/character/web.glb --texture-size 1024 --texture-compress webp --compress meshopt
npx --yes @gltf-transform/[email protected] validate results/character/web.glb
```

The inspector and procedural author use the bpy Python API, not Blender GUI automation. The Three.js runtime must set MeshoptDecoder on GLTFLoader. Verify skin/joint/clip counts after optimization, then sample actual skinned vertices at multiple action times and review the visual result. A moving bone without a deforming mesh does not pass.

Next production stage: select licensed role-appropriate motion, map the inspected joints, retarget and bake it, then verify feet, shoulders, hands and weapon/tool attachment. Existing animation packs may provide motion while keeping the custom visible mesh. That retargeting stage is not exercised by this recipe yet.


Pack entrypoint: [Custom 3D RTS Asset Pipeline](https://promoteindex.com/skills/custom-3d-rts-asset-pipeline-a-reusable-skill-pa).