Rogerio Bonatti

Senior Researcher, Microsoft
rogeriobonatti at gmail dot com

CV / LinkedIn / Publications / Twitter


I'm a Senior Researcher in the Applied Sciences Group (ASG) at Microsoft, where I develop AI-based experiences for the new Windows Copilot. I focus on user understanding, proactive recommendations, and decision-making using multi-modal foundation models. In particular, I create generative machine learning algorithms that take actions using language and vision.

Much of my past work before joining ASG was in the robotics space, where I deployed autonomous systems in manipulators, virtual and real embodied agents, all the way to flying robots.

I received my PhD in Robotics from Carnegie Mellon University's School of Computer Science, and interned at Microsoft and Facebook AI Research.

My work has been awarded with a Best Student Paper Finalist Nomination (IROS 2020), a Microsoft Research Dissertation Grant, a Siebel Scholarship and a Swartz Entrepreneurship Fellowship. Before CMU I studied Mechatronics Engineering at the University of São Paulo, Brazil and also had an amazing experience in management consulting at McKinsey & Co..

Selected News

Aug 2023 Papers accepted at ICCV 23 and IROS 23!
Feb 2023 Released our blog post on ChatGPT for robotics!
Jan 2023 Workshop proposal accepted to ICRA 23! Pretraining for Robotics
Jan 2023 Papers accepted at ICLR 23 (notable top 25%) and ICRA 23
June 2022 One paper accepted at IROS 2022
May 2022 Invited to give research talks at the Air Force Research Lab and at the Automate Show
July 2021 Defended my PhD thesis and now joined Microsoft as a Senior Researcher
June 2021 One paper accepted at IROS 2021
March 2021 Two papers accepted at ICRA 2021
Sept 2020 Our paper on cross-modal representations for drone racing is a finalist for the Best Student Paper Award at IROS 2020 (3 out of 2996 submissions)
Sept 2020 Honored to become a Siebel Scholar, class of 2021!



2016 - 2021

Ph.D., Robotics

Carnegie Mellon University, Robotics Institute

Pittsburgh, PA
Research Advisor: Prof. Sebastian Scherer
Thesis Committee: Sebastian Scherer, Jessica Hodgins, Oliver Kroemer, Ashish Kapoor, Nathan Ratliff

2011 - 2016

B.S., Mechatronics Engineering - Robotics

University of São Paulo, Polytechnic School

São Paulo, Brazil
GPA: 8.7/10, non-curved, top 3/800
Senior Thesis: Development of email classifier in Brazilian Portuguese for automatic response
Research Advisor: Prof. Fabio Cozman

Jan 2013 - Dec 2013

Study abroad, Mechanical Engineering

Cornell University, College of Engineering

Ithaca, NY
GPA: 4.08 / 4.30
One-year study program. Full scholarship by Brazil's Ministry of Education
Research Advisor: Prof. Ephrahim Garcia

Awards and honors

• 2020: Best Student Paper Finalist, IROS 2020
• 2020: Siebel Scholarship, class of 2021 ($35,000)
• 2020: Microsoft Research Dissertation Grant ($25,000)
• 2019: James R. Swartz Entrepreneurial Fellowship
• 2017-2019: Travel awards: RSS’17, ISER’18, IROS’19, CMU Provost
• 2017: CMU School of Computer Science Technology and Entertainment Proposal for Autonomous Drone Cinematographer ($60,000)
• 2014-2016: Fundacão Estudar Merit Scholarship for outstanding trajectory and academic potential, 28 recipients out of 31,000+ applicants (~$10,000)
• 2012-2014: The State of São Paulo Research Foundation (FAPESP) scholarship for undergraduate research (~$3,000)
• 2013: Dean's List, Cornell University
• 2013: Full scholarship from Brazil's Ministry of Education for one-year study program at Cornell University, College of Engineering (~$75,000)
• 2011: Entered the Polytechnic School of the University of São Paulo in 3rd/10,600 applicants
• 2011: "PACE Course Competition Engineering Design Graphics", 2nd place, sponsored by GM, Autodesk, HP, Oracle, Siemens
• 2010: High school valedictorian - Colegio Visconde de Porto Seguro, São Paulo, Brazil
• 2010: Gold Medal, Brazilian Physics Olympiad, OBF (top 20 from over 50,000 competitors)

Work Experience

May 2023 - Present, Redmond, WA

Microsoft AI

Senior Researcher, Applied Sciences Group

• Developing AI-based experiences for the new Windows Copilot
• Focused on user understanding, proactive recommendations, and decision-making
• Training large language models (LLMs), computer vision models, and generative AI tools
• In the process of submitting 2 patent applications

July 2021 - May 2023, Redmond, WA

Microsoft Research

Senior Researcher, Autonomous Systems and Robotics Research Group

• Developed multi-modal foundational models for robotics and decision-making. My research focus was to create generative machine learning models that fuse language, vision, and geometrical features to allow autonomous agents to take actions in the real world.
• Conference publications at ICCV, ICLR, ICRA, IROS, and workshop organizer at ICRA 23

May 2020 - Aug 2020, Pittsburgh, PA

Facebook AI Research

Research Intern

• Worked with Jessica Hodgins and Mustafa Mukadam on learning a manifold of movie emotion descrip- tors, and a generative model to automatically execute distinct styles for autonomous drone cameras.

May 2019 - Aug 2019, Redmond, WA

Microsoft Research

Research Intern

• Worked on imitation learning and sim-to-real transfer of deep control policies with Ashish Kapoor, Vibhav Vineet on learning efficient state representations for imitation learning that can generalize across simulation and real-life data

Aug 2015 - May 2016, São Paulo, Brazil

McKinsey & Co.

Business Analyst Intern

• Worked as a generatist consultant on Supply Chain, Logistics, Purchasing, and Commercial Policy Transformation across multiple industries

Jul 2014 - Aug 2014, São Paulo, Brazil

Itaú BBA

Winter Intern, Cost Optimization and Estimation Team

• Itaú BBA is Latin America's largest corporate investment bank, in a holding with $85B market cap

May 2013 - Aug 2013, Thousand Oaks, CA

Amgen Inc.

Summer Intern, Global Strategic Sourcing, Finance and Strategy Team

• Amgen is the largest biotech company in the world with $18.7B in sales in 2013
• Developed and applied statistical analysis to a marketing bench-marking database leading to an estimated 4x increase in savings ($8M) and 60% decrease (4 months) in the time required to perform data analysis


rogeriobonatti at gmail dot com