Tianyuan Zhang 「张天远」
I am a second-year PhD student at MIT EECS, advised by Prof. Bill
Freeman. Before that, I get my MS in Robotics at CMU, supervised by Prof.
Srinivasa Narasimhan, and my undergraduate in Peking
University, working with
Prof. Zhanxing Zhu,
Dr. Xiangyu Zhang,
and
Prof. Hang Zhao.
Email: tianyuan [at] mit [dot] edu
I acknowledge that information asymmetry can significantly hinder research opportunities for junior
students. If you’re interested in chatting about life, research, or potential collaborations, feel
free to email me.
CV  / 
Google Scholar
 / 
Github / 
Attempts at photography     
|
|
Research
I'm have research experience in machine learning, physiscs-based vision, computational imaging and
computer graphics.
My current focus is on video generation and world models.
|
|
RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
Tianyuan Zhang,
Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao Tan, He Zhang,
Yiwei Hu, Milos Hasan, William T. Freeman, Kai Zhang, Fujun Luan
arxiv, 2024
project page
/
paper
We build a probabilistic inverse rendering model that reconstrcts and relights 3D objects with
sparse
input views.
|
|
PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation
Tianyuan Zhang,
Hong-Xing "Koven" Yu, Rundi Wu, Brandon Y. Feng,
Changxi Zheng, Noah Snavely, Jiajun Wu, William T. Freeman.
ECCV, 2024
(Oral Presentation)
project page
/
github
/
paper
We bring static 3D objects to life by distilling material parameters from video generation models.
|
|
Physically Compatible 3D Object Modeling from a Single Image
Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang,
Crystal Elaine Owens,
Chuang Gan, Joshua B. Tenenbaum, Kaiming He, Wojciech Matusik
NeurIPS, 2024
(Splotlight)
project page /
paper /
Recostruct 3D physical objects from single images by considering mechanical properties, external
forces, and rest-shape geometry.
|
|
Analyzing Physical Impacts using Transient Surface Wave Imaging
Tianyuan Zhang,
Mark Sheinin, Dorian Chan, Mark Rau,
Matthew O'Toole, Srinivasa G. Narasimhan.
CVPR, 2023
project page
/
github
/
paper
/
videos
We image the "ripples" on solid surfaces caused by physical impacts, which contain information about
the object's physical properties and its interaction with the environment.
We showcase non-line-of-sight impact localization capabilities.
|
|
Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Zhewei Huang,
Tianyuan Zhang,
Wen Heng, Boxin Shi, Shuchang Zhou
ECCV, 2022
github
/
arXiv
/
demos
We propose a real-time intermediate flow estimation (RIFE) method for video frame interpolation, it
runs 30+FPS
for 2X 720p interpolation on a 2080Ti GPU
|
|
Embracing Single Stride 3D Object Detector with Sparse Transformer
Lue Fan, Ziqi Pang,
Tianyuan Zhang,
Yu-Xiong Wang, Hang Zhao, Feng Wang, Naiyang Wang, Zhaoxiang Zhang
CVPR, 2022
github /
arxiv /
In contrast to 2D, object size in 3D does not exhibit long-tail distributions. We propose a single
stride sparse Transformer (SST) for 3D object detection. We obtained impressive results on small
objects
|
|
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries
Yue Wang, Vitor Guizilini*,
Tianyuan Zhang*,
Yilun Wang, Hang Zhao, Justin Solomon
CoRL, 2021
github /
arxiv /
A new paradigm of 3D object detection from multiview 2D images
|
|
MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries
Tianyuan Zhang,
Xuanyao Chen,
Yue Wang, Yilun Wang, Hang Zhao
preprint, 2022
project page
/
github /
arXiv
End-to-End 3D tracking with multiview-cameras
|
|
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Xuanyao Chen,
Tianyuan Zhang,
Yue Wang, Yilun Wang, Hang Zhao
preprint, 2022
project page
/
github /
arXiv
A unified framework for 3D detection from multi-sensor data. We achieved impressive results with
multiview-cameras and one-beam LiDAR.
|
|
Objects365: A Large-scale, High-quality Dataset for Object Detection
Shuai Shao*, Zeming Li*,
Tianyuan Zhang*,
Chao Peng*, Gang Yu, Xiangyu Zhang, Jing Li, Jian Sun
ICCV, 2019
project page /
paper
We provide a high-quality large-scale object detection dataset, with 365
categories, 638K images,
and 10,101K bounding boxes
|
|
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Dinghuai Zhang*,
Tianyuan Zhang*,
Yiping Lu*, Zhanxing Zhu, Bin Dong
NeurIPS, 2019  
arXiv
/
code
Accelerating adversarial training using Pontryagin`s Maximum Principle
|
|
Interpreting Adversarially Trained Convolutional Neural Networks
Tianyuan Zhang,
Zhanxing Zhu
ICML, 2019
github /
arXiv
Discussion on the shape-bias and texture-bias of adversarially trainined convolutional neural
networks
|
Professional Services
Reviewer: CVPR' 2021,23, NeurIPS' 2020, ICLR' 2021,22,23 BlogPosts.
|
|