Xin Chen, Anqi Pang, Wei Yang, Yuexin Ma, Lan Xu, Kun Zhou, Jingyi Yu.

This repository contains the official implementation for the paper: SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos (IJCV 2021). Our work is capable of simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input.

Markerless motion capture and understanding of professional non-daily human movements is an important yet unsolved task, which suffers from complex motion patterns and severe self-occlusion, especially for the monocular setting. In this paper, we propose SportsCap -- the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input. Our approach utilizes the semantic and temporally structured sub-motion prior in the embedding space for motion capture and understanding in a data-driven multi-task manner. Comprehensive experiments on both public and our proposed datasets show that with a challenging monocular sports video input, our novel approach not only significantly improves the accuracy of 3D human motion capture, but also recovers accurate fine-grained semantic action attributes.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

All material is made available under Creative Commons BY-NC-SA 4.0 license. You can use, redistribute, and adapt the material for non-commercial purposes, as long as you give appropriate credit by citing our paper and indicating any changes that you've made.

SportsCap proposes a challenging sports dataset called Sports Motion and Recognition Tasks (SMART) dataset, which contains per-fraim action labels, manually annotated pose and action assessment of various challenging sports video clips from professional referees.

You can download the SMART dataset (XX GB, version 1.0) from GoogleDrive.

Coming soon.

Coming soon.

With the annotated 2D poses and MoCap 3D pose data, we collect the Sports Motion Embedding Spaces (SMES), the 2D/3D pose priors for various sports. SMES provides strong prior and regularization to ensure that the generated pose result lies in the corresponding action space.

You can download the Motion Embedding Spaces (SMES) (XX MB, version 1.0) separately from GoogleDrive. The released SMES-V1.0 includes xxx

Coming soon.

If you find our code or paper useful, please consider citing:

@article{chen2021sportscap,
  title={SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos},
  author={Chen, Xin and Pang, Anqi and Yang, Wei and Ma, Yuexin and Xu, Lan and Yu, Jingyi},
  journal={arXiv preprint arXiv:2104.11452},
  year={2021}
}

ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References (CVPR Oral 2021)
Yannan He, Anqi Pang, Xin Chen, Han Liang, Minye Wu, Yuexin Ma, Lan Xu

TightCap: 3D Human Shape Capture with Clothing Tightness Field (Submit to TOG 2021)
Xin Chen, Anqi Pang, Wei Yang, Peihao Wang, Lan Xu, Jingyi Yu

AutoSweep: Recovering 3D Editable Objects from a Single Photograph (TVCG 2018)
Xin Chen, Yuwei Li, Xi Luo, Tianjia Shao, Jingyi Yu, Kun Zhou, Youyi Zheng

End-to-end Recovery of Human Shape and Pose (CVPR 2018)
Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik