Research Context

This project studies text-driven Human-Human Interaction (HHI) motion generation, where a model must generate two-person motion from a natural-language description.

• HHI motion generation is harder than single-person motion generation because the output must preserve contact, distance, timing, role, and event structure between two people.
• Existing HHI models often rely on a sentence-level text embedding (CLIP text embedding), which can weaken fine-grained instructions such as body part, direction, role, or action-specific details.
• I approached this problem through two separate axes: word-level text-motion alignment and interaction-structure analysis.

My Contribution

I worked on fine-grained conditioning and interaction-representation analysis for HHI motion generation as a first-author KSC 2025 paper and graduation-thesis project.

• Proposed Word-Motion Cross Attention (WMCA), where flattened motion tokens attend directly to CLIP word-token embeddings instead of relying only on a single sentence-level text condition.
• Defined and analyzed interaction representations at three levels: sequence-level attributes, time-indexed relations, and event/state constraints covering contact, distance, synchrony, roles, and event progression.
• Evaluated these representations through text predictability, generator conditioning, candidate selection, redundancy, quality relevance, and their ability to explain denoising residuals missed by the generator.

Research Artifacts

Architecture, quantitative evaluation, and the fixed-generator feature audit document the research path.

Output

This work produced a KSC 2025 first-author paper and a graduation thesis on fine-grained condition utilization in text-driven HHI motion generation.

• KSC 2025: Word-Motion Cross Attention for Fine-Grained Text-Guided Human-Human Interaction Motion Generation.
• Graduation thesis: Fine-Grained Condition Utilization and Interaction Representation Analysis in Text-driven Human-Human Interaction Motion Generation.
• WMCA improved R-Precision Top-1/2/3 by +2.3 / +3.4 / +3.2 points while maintaining MM-Dist.
• The result is an alignment improvement rather than an overall generation-quality claim: FID worsened, and the feature audit showed that simple sequence-level interaction features remain insufficient to fully explain contact timing, body-part relations, roles, and event order.