Tianyuan Zhang 「张天远」
I am a second-year PhD student at MIT EECS, advised by Prof. Bill
Freeman . Before that, I get my MS in Robotics at CMU, supervised by Prof.
Srinivasa Narasimhan , and my undergraduate in Peking
University, working with
Prof. Zhanxing Zhu ,
Dr. Xiangyu Zhang ,
and
Prof. Hang Zhao .
Email: tianyuan [at] mit [dot] edu
I acknowledge that information asymmetry can significantly hinder research opportunities for junior
students. If you’re interested in chatting about life, research, or potential collaborations, feel
free to email me.
CV  / 
Google Scholar
 / 
Github  / 
Attempts at photography     
Research
I'm have research experience in machine learning, physiscs-based vision, computational imaging and
computer graphics.
My current focus is on video generation and world models.
RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
Tianyuan Zhang ,
Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao Tan, He Zhang,
Yiwei Hu, Milos Hasan, William T. Freeman, Kai Zhang, Fujun Luan
arxiv , 2024
project page
/
paper /
code comming soon
We build a probabilistic inverse rendering model that reconstrcts and relights 3D objects with
sparse
input views.
Your browser does not support the video tag.
LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias
Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang ,
Fujun Luan, Noah Snavely, Zexiang Xu
arxiv , 2024
project page
/
paper /
code comming soon
Posed Novel view synthesis with minimal 3D inductive bias: color is constant along rays, and pinehole
camera
models.
PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation
Tianyuan Zhang ,
Hong-Xing "Koven" Yu, Rundi Wu, Brandon Y. Feng,
Changxi Zheng, Noah Snavely, Jiajun Wu, William T. Freeman.
ECCV , 2024
(Oral Presentation)
project page
/
github
/
paper
We bring static 3D objects to life by distilling material parameters from video generation models.
Physically Compatible 3D Object Modeling from a Single Image
Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang ,
Crystal Elaine Owens,
Chuang Gan, Joshua B. Tenenbaum, Kaiming He, Wojciech Matusik
NeurIPS , 2024
(Splotlight)
project page /
paper /
Recostruct 3D physical objects from single images by considering mechanical properties, external
forces, and rest-shape geometry.
Analyzing Physical Impacts using Transient Surface Wave Imaging
Tianyuan Zhang ,
Mark Sheinin, Dorian Chan, Mark Rau,
Matthew O'Toole, Srinivasa G. Narasimhan.
CVPR , 2023
project page
/
github
/
paper
/
videos
We image the "ripples" on solid surfaces caused by physical impacts, which contain information about
the object's physical properties and its interaction with the environment.
We showcase non-line-of-sight impact localization capabilities.
Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Zhewei Huang,
Tianyuan Zhang ,
Wen Heng, Boxin Shi, Shuchang Zhou
ECCV , 2022
github
/
arXiv
/
demos
We propose a real-time intermediate flow estimation (RIFE) method for video frame interpolation, it
runs 30+FPS
for 2X 720p interpolation on a 2080Ti GPU
Embracing Single Stride 3D Object Detector with Sparse Transformer
Lue Fan, Ziqi Pang,
Tianyuan Zhang ,
Yu-Xiong Wang, Hang Zhao, Feng Wang, Naiyang Wang, Zhaoxiang Zhang
CVPR , 2022
github /
arxiv /
In contrast to 2D, object size in 3D does not exhibit long-tail distributions. We propose a single
stride sparse Transformer (SST) for 3D object detection. We obtained impressive results on small
objects
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries
Yue Wang, Vitor Guizilini*,
Tianyuan Zhang* ,
Yilun Wang, Hang Zhao, Justin Solomon
CoRL , 2021
github /
arxiv /
A new paradigm of 3D object detection from multiview 2D images
MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries
Tianyuan Zhang ,
Xuanyao Chen,
Yue Wang, Yilun Wang, Hang Zhao
preprint , 2022
project page
/
github /
arXiv
End-to-End 3D tracking with multiview-cameras
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Xuanyao Chen,
Tianyuan Zhang ,
Yue Wang, Yilun Wang, Hang Zhao
preprint , 2022
project page
/
github /
arXiv
A unified framework for 3D detection from multi-sensor data. We achieved impressive results with
multiview-cameras and one-beam LiDAR.
Objects365: A Large-scale, High-quality Dataset for Object Detection
Shuai Shao*, Zeming Li*,
Tianyuan Zhang* ,
Chao Peng*, Gang Yu, Xiangyu Zhang, Jing Li, Jian Sun
ICCV , 2019
project page /
paper
We provide a high-quality large-scale object detection dataset, with 365
categories, 638K images,
and 10,101K bounding boxes
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Dinghuai Zhang*,
Tianyuan Zhang* ,
Yiping Lu*, Zhanxing Zhu, Bin Dong
NeurIPS , 2019  
arXiv
/
code
Accelerating adversarial training using Pontryagin`s Maximum Principle
Interpreting Adversarially Trained Convolutional Neural Networks
Tianyuan Zhang ,
Zhanxing Zhu
ICML , 2019
github /
arXiv
Discussion on the shape-bias and texture-bias of adversarially trainined convolutional neural
networks
Professional Services
Reviewer: CVPR' 2021,23, NeurIPS' 2020, ICLR' 2021,22,23 BlogPosts.