MENU
Wentao Zhu is a final-year Ph.D. student in Computer Science at Center on Frontiers of Computing Studies, Peking University. He is a member of the Computer Vision and Digital Art group, advised by Prof. Yizhou Wang. Previously, he received Bachelor's degrees in Computer Science and Economics from Peking University in 2020.

Research

My research primarily focuses on building Human-aware AI systems that can perceive, understand, and interact with human beings. My long-term research goal is to create collaborative, embodied agents that can seamlessly integrate with human activities in real-world scenarios, such as home living and healthcare.

We are looking for self-motivated research interns, visiting students, and prospective students in 3D Vision, Embodied AI, and Foundation Models. Feel free to reach out!

Preprints

FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling
Hang Ye, Xiaoxuan Ma, Hai Ci, Wentao Zhu, Yizhou Wang
arXiv preprint arXiv:2411.19942
Paper / Code / Project Page / Video
Learning Human-aware Robot Policies for Adaptive Assistance
Jason Qin, Shikun Ban, Wentao Zhu, Yizhou Wang, Dimitris Samaras
arXiv preprint
Aligning Human Motion Generation with Human Perceptions
Haoru Wang*, Wentao Zhu*, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, Yizhou Wang
arXiv preprint arXiv:2407.02272
Paper / Code / Project Page / Video
Efficient Action Counting with Dynamic Queries
Zishi Li, Xiaoxuan Ma, Qiuyan Shang, Wentao Zhu, Hai Ci, Yu Qiao, Yizhou Wang
arXiv preprint arXiv:2403.01543
Paper / Code / Project Page / Video

Publications

2024

Real-time Holistic Robot Pose Estimation with Unknown States
Shikun Ban, Juling Fan, Xiaoxuan Ma, Wentao Zhu, Yu Qiao, Yizhou Wang
European Conference on Computer Vision (ECCV), 2024
Paper / Code / Project Page / Video
Language Models Represent Beliefs of Self and Others
Wentao Zhu, Zhining Zhang, Yizhou Wang
International Conference on Machine Learning (ICML), 2024
Paper / Code / Project Page
Human Motion Generation: A Survey
Wentao Zhu*, Xiaoxuan Ma*, Dongwoo Ro*, Hai Ci, Jinlu Zhang, Jiaxin Shi, Feng Gao, Qi Tian, Yizhou Wang
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024
Paper
ScoreHypo: Probabilistic Human Mesh Estimation with Hypothesis Scoring
Yuan Xu, Xiaoxuan Ma, Jiajun Su, Wentao Zhu, Yu Qiao, Yizhou Wang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Paper / Code / Project Page / Video

2023

Social Motion Prediction with Cognitive Hierarchies
Wentao Zhu*, Jason Qin*, Yuke Lou, Hang Ye, Xiaoxuan Ma, Hai Ci, Yizhou Wang
Neural Information Processing Systems (NeurIPS), 2023
Paper / Code / Project Page / Video
ChimpACT: A Longitudinal Dataset for Understanding Chimpanzee Behaviors
Xiaoxuan Ma*, Stephan P. Kaufhold*, Jiajun Su*, Wentao Zhu, Jack Terwilliger, Andres Meza, Yixin Zhu, Federico Rossano, Yizhou Wang
Neural Information Processing Systems (NeurIPS, Datasets and Benchmarks Track), 2023
Paper / Code / Project Page / Video
MotionBERT: A Unified Perspective on Learning Human Motion Representations
Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, Yizhou Wang
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Paper / Code / Project Page / Video
GFPose: Learning 3D Human Pose Prior with Gradient Fields
Hai Ci, Mingdong Wu, Wentao Zhu, Xiaoxuan Ma, Hao Dong, Fangwei Zhong, Yizhou Wang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Paper / Code / Project Page
3D Human Mesh Estimation from Virtual Markers
Xiaoxuan Ma, Chunyu Wang, Jiajun Su, Wentao Zhu, Yizhou Wang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Paper / Code / Project Page / Video

2022

Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection
Hang Ye*, Wentao Zhu*, Chunyu Wang, Rujie Wu, Yizhou Wang
European Conference on Computer Vision (ECCV), 2022
Paper / Code
CelebV-HQ: A Large-scale Video Facial Attributes Dataset
Hao Zhu*, Wayne Wu*, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, Chen Change Loy
European Conference on Computer Vision (ECCV), 2022
Paper / Code / Project Page / Video
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
Wentao Zhu*, Zhuoqian Yang*, Ziang Di, Wayne Wu, Yizhou Wang, Chen Change Loy
Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI), 2022
Paper / Code / Project Page

2020

TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
Zhuoqian Yang*, Wentao Zhu*, Wayne Wu*, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
Paper / Code / Project Page / Video

Services

Conference Reviewer / PC Member:
CVPR, ECCV, ICCV, ICLR, AAAI, NeurIPS, IEEE VR, SIGGRAPH Asia

Journal Reviewer:
TPAMI, IJCV, TOG, TVCG, TIP

Workshop Committee:
Populating Empty Cities – Virtual Humans for Robotics and Autonomous Driving @ CVPR 2024

Teaching

Robot Vision and Learning (Graduate Course in English, TA) Fall 2022
Practice of Programming in C & C++ (TA) Spring 2021
Career Planning and Leadership Development (TA) Fall 2020

I am fortunate enough to have worked with these brilliant undergraduate interns (ordered by year and last name):

Hang Ye (now Ph.D. student at PKU) ECCV'22, NeurIPS'23
Jason Qin (now Ph.D. student at Stony Brook University) NeurIPS'23
Yuan Xu (incoming Ph.D. student at PKU) CVPR'24
Shikun Ban (now visiting student at Columbia University) ECCV'24
Zhining Zhang (now visiting student at Johns Hopkins University) ICML'24
Haoru Wang (now visiting student at Technical University of Munich) NeurIPS'24