Tianyuan Zhang 「张天远」

I am a second-year PhD student at MIT EECS, advised by Prof. Bill Freeman. Before that, I get my MS in Robotics at CMU, supervised by Prof. Srinivasa Narasimhan, and my undergraduate in Peking University, working with Prof. Zhanxing Zhu, Dr. Xiangyu Zhang, and Prof. Hang Zhao.

Email: tianyuan [at] mit [dot] edu

I acknowledge that information asymmetry can significantly hinder research opportunities for junior students. If you're interested in chatting about life, research, or potential collaborations, feel free to email me.

CV  /  Google Scholar  /  Github /  Attempts at photography     

I have had research experience in machine learning, physiscs-based vision, computational imaging and computer graphics.

My current focus is on video generation, world models and infinite context learning.

Hardware-friendly Test-Time Training boosts FLOPs utilization by 10x, facilitates larger state-size and advanced optimizers, and can be implemented in PyTorch with just a few lines of code. Validated on novel view synthesis, language models, and AR video diffusion.

Next-token prediction in random orders for images.

We build a probabilistic inverse rendering model that reconstrcts and relights 3D objects with sparse input views. GPUs learn algorithms!

Posed Novel view synthesis with minimal 3D inductive bias.

We bring static 3D objects to life by distilling material parameters from video generation models.

Recostruct 3D physical objects from single images by considering mechanical properties, external forces, and rest-shape geometry.

We image the "ripples" on solid surfaces caused by physical impacts, which contain information about the object's physical properties and its interaction with the environment. We showcase non-line-of-sight impact localization capabilities.

We propose a real-time intermediate flow estimation (RIFE) method for video frame interpolation, it runs 30+FPS for 2X 720p interpolation on a 2080Ti GPU

In contrast to 2D, object size in 3D does not exhibit long-tail distributions. We propose a single stride sparse Transformer (SST) for 3D object detection. We obtained impressive results on small objects

A new paradigm of 3D object detection from multiview 2D images

End-to-End 3D tracking with multiview-cameras

A unified framework for 3D detection from multi-sensor data. We achieved impressive results with multiview-cameras and one-beam LiDAR.

We provide a high-quality large-scale object detection dataset, with 365 categories, 638K images, and 10,101K bounding boxes

Accelerating adversarial training using Pontryagin`s Maximum Principle

Discussion on the shape-bias and texture-bias of adversarially trainined convolutional neural networks

Reviewer: CVPR' 2021,23, NeurIPS' 2020, ICLR' 2021,22,23 BlogPosts.