Yisong Yue (he/him)
California Institute of Technology
1200 E. California Blvd.
CMS, 305-16
Pasadena, CA 91125

Office: 303 Annenberg

Contact Information >

I am a professor of Computing and Mathematical Sciences at the California Institute of Technology. My research interests lie primarily in machine learning, and span the entire theory-to-application spectrum from foundational advances all the way to deployment in real systems. I work closely with domain experts to understand the frontier challenges in applied machine learning, distill those challenges into mathematically precise formulations, and develop novel methods to tackle them.

Latitude AI: I am currently a (part-time) Principal Scientist at Latitude AI, where I work on machine learning approaches to behavior modeling and motion planning for autonomous driving.

Asari AI: I am a founding advisor for Asari AI, where I help design AI co-inventors (AI agents that can plan, abstract, and verify complex design tasks).

ICLR 2024: I am serving as the Senior Program Chair at ICLR 2024. The rest of Program Chair team includes Swarat Chaudhuri, Katerina Fragkiadaki, Emtiyaz Khan, and Yizhou Sun.
Diversity, Equity & Inclusion
I am committed to promoting diversity, equity, and inclusion in my research group, in my courses, within the CMS department, at Caltech more broadly, and within my research communities.
  • Diversity -- I recognize that diversity, in all its shapes and forms, strengthens us both culturally and intellectually.
  • Equity -- I will fight for equal treatment of all people, regardless of race, gender, sexual orientation, or any other attributes that do not define a person's academic and research potential.
  • Inclusion -- I will work to create an inclusive working environment, so that everyone feels their voices are heard and their contributions are recognized.
Read more about diversity, equity, and inclusion at the CMS Department and EAS Division at Caltech.
Current Research
My current research interests can be broadly organized into three overlapping groups:

AI for Autonomy: study how AI methods can enable novel capabilities in autonomous systems; characterize and address key technical bottlenecks (e.g., data-driven safety guarantees); deploy in real systems.
Selected Publications
  • Neural Gaits: Learning Bipedal Locomotion via Control Barrier Functions and Zero Dynamics Policies
    Ivan Dario Jimenez Rodriguez*, Noel Csomay-Shanklin*, Yisong Yue, Aaron D. Ames
    Conference on Learning for Dynamics and Control (L4DC), June 2022.
  • Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds
    Michael O’Connell*, Guanya Shi*, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, Soon-Jo Chung
    Science Robotics, May 2022.
    [arxiv][online][code][video][press release]
  • MLNav: Learning to Safely Navigate on Martian Terrains
    Shreyansh Daftry, Neil Abcouwer, Tyler Del Sesto, Siddarth Venkatraman, Jialin Song, Lucas Igel, Amos Byon, Ugo Rosolia, Yisong Yue, Masahiro Ono
    IEEE Robotics and Automation Letters (RA-L), May 2022
  • GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning
    Benjamin Rivière, Wolfgang Hoenig, Yisong Yue, Soon-Jo Chung
    IEEE Robotics and Automation Letters (RA-L), June 2020.
    (Best Paper Nomination)
    [pdf][arxiv][demo video]
  • Preference-Based Learning for Exoskeleton Gait Optimization
    Maegan Tucker*, Ellen Novoseller*, Claudia Kann, Yanan Sui, Yisong Yue, Joel Burdick, Aaron D. Ames
    International Conference on Robotics and Automation (ICRA), May 2020.
    (Best Paper Award)
    [pdf][arxiv][demo video][project]

AI for Science: study how AI methods can improve workflows in science and accelerate knowledge discovery; develop methods for automated experiment design and human-intelligible modeling; deploy in real systems.
Selected Publications
  • A Foundation Model for Cell Segmentation
    Uriah Israel, Markus Marks, Rohit Dilip, Qilin Li, Morgan Sarah Schwartz, Elora Pradhan, Edward Pao, Shenyi Li, Alexander Pearson-Goulart, Pietro Perona, Georgia Gkioxari, Ross Barnowski, Yisong Yue, David Ashley Van Valen
  • Self-Supervised Keypoint Discovery in Behavioral Videos
    Jennifer J. Sun*, Serim Ryou*, Roni Goldshmid, Brandon Weissbourd, John Dabiri, David J. Anderson, Ann Kennedy, Yisong Yue, Pietro Perona
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
  • DeepGEM: Generalized Expectation-Maximization for Blind Inversion
    Angela Gao, Jorge Castellanos, Yisong Yue, Zachary Ross, Katherine Bouman
    Neural Information Processing Systems (NeurIPS), December 2021.
  • End-to-End Sequential Sampling and Reconstruction for MR Imaging
    Tianwei Yin*, Zihui Wu*, He Sun, Adrian V. Dalca, Yisong Yue, Katherine L. Bouman
    Machine Learning for Health (ML4H), December 2021.
    (Best Paper Award)
  • Task Programming: Learning Data Efficient Behavior Representations
    Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Yue, Pietro Perona
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2021.
    (Best Student Paper Award)

Core AI/ML Research: study the underlying fundamental questions pertaining practical algorithm design, inspired by real-world applications in science and engineering.
Selected Publications
  • Online Policy Optimization in Unknown Nonlinear Systems
    Yiheng Lin, James A. Preiss, Fengze Xie, Emile Anand, Soon-Jo Chung, Yisong Yue, Adam Wierman
    Conference on Learning Theory (COLT), 2024
  • TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
    Sabera Talukder, Yisong Yue, Georgia Gkioxari
  • Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
    Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, Yisong Yue
    International Conference on Machine Learning (ICML), 2024
  • Automatic Gradient Descent: Deep Learning without Hyperparameters
    Jeremy Bernstein, Chris Mingard, Kevin Huang, Navid Azizan, Yisong Yue
    [arxiv][code][blog post]
  • LyaNet: A Lyapunov Framework for Training Neural ODEs
    Ivan Dario Jimenez Rodriguez, Aaron D. Ames, Yisong Yue
    International Conference on Machine Learning (ICML), July 2022.
  • Neurosymbolic Programming
    Swarat Chaudhuri, Kevin Ellis, Oleksandr Polozov, Rishabh Singh, Armando Solar-Lezama, Yisong Yue
    Foundations and Trends in Programming Languages, Volume 7: No. 3, pages 158-243, December 2021.
  • Batch Policy Learning under Constraints
    Hoang M. Le, Cameron Voloshin, Yisong Yue
    International Conference on Machine Learning (ICML), June 2019.
    (Oral Presentation)
  • Multi-dueling Bandits with Dependent Arms
    Yanan Sui, Vincent Zhuang, Joel Burdick, Yisong Yue
    Conference on Uncertainty in Artificial Intelligence (UAI), August 2017.
News & Announcements
  • SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code -- we developed a Large Language Model (LLM) Agent for converting text descriptions into Blender-executable Python scripts which render complex scenes with up to a hundred 3D assets. This process requires complex spatial planning and arrangemet, which we tackle through a combination of advanced abstraction, strategic planning, and library learning. [arxiv]
  • Tokenized Time Series Embeddings -- we developed a framework for learning a tokenization for downstream time series modeling, including training generalist models that can be applied to many domains. [arxiv][code]
  • Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion -- we developed a method for guided diffusion using non-differentiable rules, called Stochastic Control Guidance. Our approach is inspired by path integral control and can be applied in a plug-and-play way to any diffusion model. We demonstrate our approach on symbolic music generation. [arxiv][website]
  • SustainGym: we have released a suite of environments designed to test the performance of RL algorithms on realistic sustainability tasks. [project][code]
  • Farewell! We had three members depart the group during the 2022-2023 Academic Year:
    • Jennifer Sun completed her Ph.D. and will start as an Assistant Professor at Cornell.
    • Cameron Voloshin completed his Ph.D. and has started at Latitude AI.
    • Victor Dorobantu completed his Ph.D. and has started a postdoc at MIT.
  • ICLR 2024: I will be serving as the Senior Program Chair at ICLR 2024. The rest of Program Chair team includes Swarat Chaudhuri, Katerina Fragkiadaki, Emtiyaz Khan, and Yizhou Sun.
  • Automatic Gradient Descent: We have developed a new hyperparameter-free optimizer for deep neural networks. Our method has been demonstrated at ImageNet scale using ResNet50 architectures.
    [arxiv][code][blog post]
  • Conformal Generative Modeling: We have developed a new framework for generative modeling that works on a wide range of manifolds. [project]
[All Content © 2024 Yisong Yue]