format-motion-transfer
Motion transfer / actor swap: take only the MOTION of a source clip (choreography, camera move, comedic timing) and re-perform it with OUR cast, OUR set and OUR audio, either as a trend-gag hook spliced onto a paying Dawn body or as a new-cast fatigue refresh of our own winner.
Lane: motion-transfer/recast role (NOT BUILT: one-clip proof first) feeding cartoon-h3; photoreal people (target images, spoken lines, talking-head refreshes) = ugc-omni with its G0-G4 gates · not built yet: 6 (see below) · source: /Users/ayden/.openclaw/workspace/skills/format-motion-transfer
Motion transfer — sources
davidaistar (paraphrased; his model names are dated, the method stands)
- Rf1uHxSWpjU — 80-min AI content guide, motion-control segment (
analysis/gemini_Rf1uHxSWpjU.md, txt/Rf1uHxSWpjU.txt). Input = one avatar image + a motion video taken from a clip that already went viral (his example: a TikTok Shop clothing creator who only films herself holding the camera, mouthing a song, wearing the item). Output = his avatar doing that motion in a new scene; "cheap", no number given. His rights rule: change the character AND the background; people who keep the creator's background are the ones who get copyright trouble. Keep: motion-only, change people + set. Change: his demo kept the creator's song audio and her lip-sync performance; we never reuse audio and never pick lip-sync sources. He sells clothing on it (clothes kept identical when the clothes ARE the product); our product never comes from a source.
- Rqim7SlMtZg — TikTok Shop motion swap (
analysis/ugc_seedance.md, analysis/gemini_Rqim7SlMtZg.md). Steps: find a GMV-proven clip → download clean → pick the clearest full-body frame of the dominant pose → swap his AI actress into that frame (face/hair) → Kling Motion Swap (Pro, or Standard at half the credits) with source video + new frame, no prompt → optional voice change → optional lip-sync to a new script. His own warning: swapping only face/hair keeps the creator's identity and has drawn creator complaints. Claimed: 11 s source ≈ $10k GMV. Keep: frame-1 anchoring (pose match kills first-second warp), Standard tier first, no prompt needed. Change: we never i2i into a third-party frame; we build the target from our sheets and replace the set too.
- 5Aj7S-Z4ct8 — Seedance 2.0 omni-reference motion matching (
txt/5Aj7S-Z4ct8.txt). A famous scene with a person running at the camera; he asks for a different character; the model adapts the run to the new body instead of copying raw pose. Keep: omni motion-ref as a second candidate for hooks where the new body differs (kid vs adult, cartoon proportions). Change: no famous film scenes (someone else's footage and characters).
- BHmPSaROT7c — "swap the actor" ad clone (
txt/BHmPSaROT7c.txt). Feeds a competitor's sleep-gummy ad segment and asks to replace the product, change the couple's race and make the voice "sound" a given ethnicity, keeping script and location. Don't adopt: competitor footage, script kept verbatim, demographic ventriloquism (format-ad-clone "Don't adopt"). Only the one-variable-per-pass mechanic survives.
- 0jcFpsnuJh0 / I0QSg8vOByc — multi-input mixing + style transfer (
txt/0jcFpsnuJh0.txt, txt/I0QSg8vOByc.txt). Several reference videos/instructions in one call; a few seconds of a film's art style restyles a new subject ("reference video 1's animation and drawing style to make a video of …"). Keep: style-from-our-own-clip is a restyle (own footage only). Change: never a copyrighted film's style clip as input.
- lIu6HLlKDvE — "Kling motion on Starpop": no transcript and no video file in the research dir; nothing extracted.
- OvAyKctgNE4 omni-ref modes (
analysis/ugc_seedance.md): camera/motion match, multi-input mixing with @image/@video tags, video styling, extension, editing (behaves like full replacement). Its restriction bypasses (face blur, name-keyword dodge, translate-to-Chinese) are not adopted (analysis/diff_ai_ugc.md).
- Diffs:
analysis/diff_cloning.md (P1 "cheap video-to-video swap lane", unbuilt) and analysis/diff_ai_ugc.md (grey-hat product-swap-only clones: don't adopt).
Our receipts
- Charswap lane (8-19 → 9-01), Seedance 2.5 video-edit on the Higgsfield unlimited entitlement (ended 9-09).
scripts/charswap_overnight.py: serial, never-bill guard, @video_1 was a competitor rip; 5 of 7 visual-QA fails were the band rendering as a thick tracker cuff with a module, fixed with worn + detail refs of our real band. scripts/charswap_source_refs.py: 24 of 25 live ads edited ONE source video, 4 of 28 got delivery; references = third-party TikTok talking heads. scripts/charswap_batch2.py: per-scene swap rules (people shots swap person + setting; no-people shots swap background + hands). charswap_g video added to the DWB scale test 10-05 (memory project_dwb_scale_test_2026-10-05), unread. Lessons kept: provenance gate, spread sources, real product refs. Lesson changed: third-party footage edits are out.
- Higgsfield video refs:
/fnf/video upload route (memory reference_higgsfield_cli_gotchas.md 9-01); scripts/higgsfield_unlim.py --video-ref / --mode video_edit|video_extension; account is shared, record job ids at submit.
kling_motion in scripts/scene_parser.py and scripts/test_pipeline_fixes.py is a per-scene motion TEXT hint for i2v prompts. Not motion transfer.
Model schemas verified 2026-10-06 (role "Motion transfer / recast"; all untested on our account)
- fal catalog
output/model-catalog/fal-2026-10-06.json: minimax/h3-max/recast (2026-10-01, "recast the people in a video using reference photos… preserving the source motion, camera, cuts, and audio"); fal-ai/kling-video/v3/{standard,pro}/motion-control + v2.6; fal-ai/bytedance/dreamactor/v2; fal-ai/id-v2v; luma/agent/ray/v3.2/video-to-video; fal-ai/kling-video/o3/4k/video-to-video/reference.
- fal OpenAPI (
https://fal.ai/api/openapi/queue/openapi.json?endpoint_id=<id>): recast requires video_url (5-30 s, no shot >15 s) + reference_image_urls (one photo per new person, replaces people left to right), optional prompt, resolution 768P|1080P (default 1080P). Kling v3 standard motion-control requires image_url (characters AND background come from it; character >5% of frame), video_url (realistic-style character, full or upper body; ≤10 s for orientation image, ≤30 s for video), character_orientation; optional keep_original_sound (default TRUE), elements (one face element, orientation video only).
- fal price page 10-06: recast $0.30/s at 768P, $0.45/s at 1080P. Motion-control price not captured: read before quoting.
- Higgsfield CLI (
higgsfield model get, 10-06): hf_mult_motion_control "Genjutsu" (≥1 image ref, exactly 1 video ref, 480p-1080p, prompt optional); kling_video_edit "Kling 3.0 Omni Edit" (video ref + prompt required, std/pro/4k); seedance_2_0 (≤3 video refs, ≤9 images, ≤12 refs total); seedance_2_5 modes omni_reference | video_edit | video_extension. Use higgsfield generate cost <job_type> … (free) before any create.
starpop.ai articles (his written process, read 10-06; paraphrased)
- how-to-use-kling-3-0-to-make-ads (step 10, Motion Control): one uninterrupted, owned or licensed reference performance; a character image posed like the reference; video-primary orientation follows the clip (≤30 s), image-primary follows the image (≤10 s); a short optional prompt naming the gesture to transfer and what to preserve; sound rebuilt in the edit. Standard for concept tests, Pro for approved finals. Changed: the "say what to borrow" prompt rule and the one-take source rule (§7). Confirms our orientation limits, Standard-first drafts and permission rule. Note: his Starpop route drops the source sound, while fal's
keep_original_sound defaults TRUE: always set it explicitly.
- how-to-use-seedance-2-0-to-make-ads (video references): say exactly what to borrow from a video ref or the model takes framing, styling and pace too. Folded into the same §7 rule.
- everything-you-need-to-know-about-seedance-2-0: "character replacement" keeps the performance, camera and environment = our recast class, own/licensed footage only (§5). Extension duration = the new segment's length, not the total (registry Video extend note, proposed).
- omni-reference-vs-start-end-frames: plan seams, not clips: batch shots in one multi-ref generation with every transition stated, chain the last frame into the next generation, or cut deliberately; judge an omni model on its transitions, not frame fidelity. For mode T the hook→body join stays a deliberate cut (no change). Not adopted for photoreal people: last-frame chaining is the keyframe-chain pattern Fish closed 10-05.
- Not adopted: the Seedance "blurry reference for character variety" trick (filter bypass).