About Me
👋 I’m a committed algorithm researcher and backend engineer with 3 years of industry experience. Specializing in 3D synthetic data generation for computer vision tasks, I contributed to 9 papers in top-tier conferences.
Recently, I’ve been focusing on text/image-to-video generation, contributing to VAE training, dataset preparation, automated annotation, and video acquisition using crawler techniques. I’m well-versed in the entire T2V pipeline and passionate about pushing the boundaries of innovation in this field.
My expertise spans:
- Engine tools development: Unreal Engine, Blender, etc.
- Rendering techniques: PBR, differentiable rendering, NeRF, 3DGS, etc.
- 3D computer vision: Novel view synthesis, generative models on 3D, etc.
- Text/image-to-video generation: VAE, Diffusion models, dataset preparation (automated annotation, crawler techniques), etc.
- Backend development: FastAPI, Flask, scalable robust system design, etc.
- Open-source projects: CI/CD, Docker, Kubernetes, etc.
Languages and tools I’m working with:
Experience
» Text/Image-to-Video Generation
I’m well-versed in the entire T2V pipeline and passionate about pushing the boundaries of innovation in this field.
- Led VAE training, successfully reproducing results comparable to the VAE of CogVideo with similar performance.
- Contributed to the data annotation process by supporting the application of textual, visual, and qualitative analyses to ensure high-quality video datasets.
- Developed automated solutions for video content acquisition from various online platforms lacking direct API support.
» Synthetic Data Generation
»» XRFeitoria
I spearheaded the development of XRFeitoria , a rendering toolbox designed to streamline synthetic data generation. This tool simplifies the construction of pipelines for topics including human mesh recovery, novel view synthesis, etc, leading to the publishing of several papers within a short timeframe:
- (ECCV 2024) WHAC: World-grounded Humans and Cameras
- (CVPR 2024) AiOS: All-in-One-Stage 3D Wholebody Mesh Recovery
- (NIPS 2023) SMPLer-X: Advanced 3D Human Body Modeling
- (NIPS 2023) PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
- (ICCV 2023) SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
- (ICCV 2023) Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
- (ICCV 2023) SHERF: Generalizable Human NeRF from a Single Image
- (Preprint) HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
»» SynBody
I held a principal role in designing SynBody, a large-scale synthetic dataset with layered human models.
- Participated in designing SMPL-XL, which enriches SMPL-X in hair, garments, accessories, and textures.
- Built a System of synthetic data generation as a Software-as-a-Service (SaaS) platform, including layered human creation, motion retargeting, scene composition, rendering pipeline, and flow control.
- Published a paper presented at ICCV 2023 as co-first author.
» Demo videos
I’m proficient in crafting advanced technology demonstration videos.
Started to work on synthetic data generation for computer vision tasks.
- Patent filed with CNIPA: CN113011334A.
- One paper related to synthetic data.
Had the experience of reproducing the SOTA methods in video captioning. Code
Open-Source Projects
XRFeitoria: Rendering Toolbox for Synthetic Data Generation
- Control over engine (UE/Blender) through RPC using system python.
- Support multiple engine backends, including Unreal Engine and Blender.
- Render photorealistic images with ground-truth annotations.
- Manage assets/cameras, including import, place, export, and delete.
- CLI tools to render images from a mesh file.
- Streamline building rendering pipelines across various domains, effectively implemented in over 8 projects.
Publications
Differentiable Convex Polyhedra Optimization from Multi-view Images
Daxuan Ren
, Haiyi Mei,Hezi Shi
,Jianmin Zheng
,Jianfei Cai
,Lei Yang
WHAC: World-grounded Humans and Cameras
Wanqi Yin
,Zhongang Cai
,Ruisi Wang
,Fanzhou Wang
,Chen Wei
, Haiyi Mei,Weiye Xiao
,Zhitao Yang
,Qingping Sun
,Atsushi Yamashita
,Ziwei Liu
,Lei Yang
Digital Life Project: Autonomous 3D Characters with Social Intelligence
Zhongang Cai
,Jianping Jiang
,Zhongfei Qing
,Xinying Guo
,Mingyuan Zhang
,Zhengyu Lin
, Haiyi Mei,Chen Wei
,Ruisi Wang
,Wanqi Yin
, Xiangyu Fan, Han Du,Liang Pan
,Peng Gao
,Zhitao Yang
, Yang Gao,Jiaqi Li
, Tianxiang Ren,Yunkun Wei
,Xiaogang Wang
,Chen Change Loy
,Lei Yang
,Ziwei Liu
AiOS: All-in-One-Stage 3D Wholebody Mesh Recovery
Qingping Sun
,Yanjun Wang
,Ailing Zeng
,Wanqi Yin
,Chen Wei
,Wenjia Wang
, Haiyi Mei,Chi Sing Leung
,Ziwei Liu
,Lei Yang
,Zhongang Cai
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
Zhaoxi Chen
,Fangzhou Hong
, Haiyi Mei,Guangcong Wang
,Lei Yang
,Ziwei Liu
SMPLer-X: Advanced 3D Human Body Modeling
Zhongang Cai
,Wanqi Yin
,Ailing Zeng
,Chen Wei
,Qingping Sun
,Yanjun Wang
,Hui En Pang
, Haiyi Mei,Mingyuan Zhang
,Lei Zhang
,Chen Change Loy
,Lei Yang
,Ziwei Liu
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Zhitao Yang*
,Zhongang Cai*
, Haiyi Mei*, Shuai Liu*,Zhaoxi Chen*
,Weiye Xiao
, Yukun Wei, Zhongfei Qing,Chen Wei
,Bo Dai
,Wayne Wu
,Chen Qian
,Dahua Lin
,Ziwei Liu
,Lei Yang
Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
Wenjia Wang
,Yongtao Ge
, Haiyi Mei,Zhongang Cai
, Qingping Sun, Yanjun Wang,Chunhua Shen
,Lei Yang
,Taku Komura
SHERF: Generalizable Human NeRF from a Single Image
Shoukang Hu*
,Fangzhou Hong*
,Liang Pan
, Haiyi Mei,Lei Yang
,Ziwei Liu
HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
Shoukang Hu
,Fangzhou Hong
,Tao Hu
,Liang Pan
, Haiyi Mei,Weiye Xiao
,Lei Yang
,Ziwei Liu
Education
Master of Biomedical/Medical Engineering
Shandong University
I Published:
- Patent filed with CNIPA: CN114429436A.
Bachelor of Automation
Nanjing University of Science and Technology
I Published:
- Patent filed with CNIPA: CN108453742B.
- Mian ZHANG, Ying HUANG, Haiyi MEI, Yu GUO. Intelligent interaction method for power distribution robot based on Kinect. Journal of Shandong University(Engineering Science), 2018, 48(5): 103-108.
Extracurricular Activities
- Head of the Symphony Orchestra of School, 2016 - 2018.
- First Place Award, National University Piano Competition, Chinese Golden Bell Award for Music, 2017.
Get in Touch
🥳 Contact me for any questions or want to collaborate! Always open to new opportunities.
haiyimei [at] gmail.com