Research

*Projects from my undergraduate studies are archived on this page.

I have been developing generalizable frameworks for robot and aviation autonomy. My approach 1) begins with a solid theoretical analysis, 2) extends to leveraging data to address real-world uncertainties, and 3) is applied to robotic and aviation systems. Below are my main research contributions. For the full publication list, please visit my google scholar.

1. Theoretical Safety Analysis: Unification of Hamilton-Jacobi Reachability and Control Barrier Function Theory

At the core of dynamical system safety assurance is verifying a 1) dynamically feasible safe set where the system can indefinitely remain, and 2) designing a controller (or constraint on the control input) to realize it.

Verifying dynamically feasible safe sets, which are crucial for safety analysis, is challenging. Many model-based approaches use the concept of a certificate function, a scalar function whose level sets characterize the safe domain, and whose gradient can impose a constraint on the control input to ensure safety. Control Barrier Functions (CBFs) and Hamilton-Jacobi (HJ) reachability value functions are two prominent choices of the certificate functions. CBFs’ key concept is to constrain the robot to “smoothly brake” before it exits the safe domain. While this mechanism is easy to implement, constructing a valid CBF is challenging. In contrast, HJ reachability constructs a maximal safe set that meets safety specifications. However, the optimal control derived from its value function is often too conservative for practical use.

A line of my research aims at bridging the gap between CBFs and HJ reachability, using the best of both methods. We discovered that the CBF braking mechanism can be incorporated into the reachability formulation, which makes it feasible to use its (set-valued) optimal policy as the safety filter. Moreover, we discovered that all CBFs can be interpreted as reachability value functions. An important accompanying finding is that in this interpretation, discount factors in reachability play a crucial role, and they can help design machine learning-based approximate DP algorithms like to have good convergence properties.

We integrated CBF’s smooth braking mechanism into the reachability formulation.

Finally, in the recent tutorial paper published in IEEE Control Systems Magazine, we looked into how three prominent safety filter methodologies, HJ Reachability, CBFs, and predictive safety filters (PSF), connect to each other. Below is the key summary of the discussion in the article.

Connections between HJ Reachability, CBFs, and predictive safety filters (PSF) [CSM2023].

Relevant Publications:

[CDC2021a] J. J. Choi, D. Lee, K. Sreenath, C. J. Tomlin, and S. L. Herbert. Robust control barrier–value functions for safety-critical control. In IEEE Conference on Decision and Control (CDC), 2021.

[Arxiv2023a] J. J. Choi˚, D. Lee˚, B. Li, J. P. How, K. Sreenath, S. L. Herbert, and C. J. Tomlin. A forward reachability perspective on robust control invariance and discount factors in reachability analysis. submitted to SIAM Journal on Control and Optimization, 2023.

[CSM2023] K. P. Wabersich˚, A. J. Taylor˚, J. J. Choi˚, K. Sreenath, C. J. Tomlin, A. Ames, and M. Zeilinger. Data-driven safety filters: Hamilton-Jacobi reachability, control barrier functions, and predictive methods for uncertain systems. IEEE CSM, 2023.

2. Data-driven Uncertainty-aware Safety Filter Frameworks

A safety filter is a control architecture that decouples performance and safety, wherein the safety filter module intervenes in desired control actions whenever they are potentially unsafe. Data-driven safety filters integrate designs based on model-based principles with data-driven components to tackle real-world uncertainty.

Overview article:

[CSM2023] K. P. Wabersich˚, A. J. Taylor˚, J. J. Choi˚, K. Sreenath, C. J. Tomlin, A. Ames, and M. Zeilinger. Data-driven safety filters: Hamilton-Jacobi reachability, control barrier functions, and predictive methods for uncertain systems. IEEE CSM, 2023.

Safety Filter with Probabilistic Guarantees Using Nonparametric Regression.

Because Gaussian Process (GP) regression is a nonparametric learning method that provides probabilistic error bounds, we integrated it with CBFs, to address the effect of model error in the CBF’s barrier constraint. By developing a unique kernel structure in GP, we formulated a second-order cone program (SOCP)-based safety filter that is convex and solvable in real time. This SOCP controller is robust against both model errors and GP prediction errors, and if the feasible solution exists, it satisfies safety with high probability for the true system.

We further analyzed the filter’s feasibility conditions, which determine whether at each state, the data is sufficient to guarantee safety. Based on this analysis, we designed an online safe learning mechanism that triggers safe exploration actions whenever necessary to prevent the filter infeasibility. Crucially, we proved that this design guarantees safety recursively, with high probability, fulfilling the original purpose of incorporating GP. Finally, to prune data that do not contribute to safety characterization and improve GP inference time, we developed a kernel-based measure that gauges the usefulness of data points for characterizing safe control directions.

Relevant Publications:

[ACC2021] F. Castaneda˚, J. J. Choi˚, B. Zhang, C. J. Tomlin, and K. Sreenath. Gaussian process-based min-norm stabilizing controller for control-affine systems with uncertain input effects and dynamics. In American Control Conference (ACC), 2021.

[CDC2021b] F. Castaneda˚, J. J. Choi˚, B. Zhang, C. J. Tomlin, and K. Sreenath. Pointwise feasibility of gaussian process-based safety-critical control under model uncertainty. In IEEE CDC, 2021.

[Arxiv2022] F. Castaneda˚, J. J. Choi˚, W. Jung, B. Zhang, C. J. Tomlin, and K. Sreenath. Recursively feasible probabilistic safe online learning with control barrier functions. submitted to Transactions on Automatic Control (under 2nd review), 2022

[Arxiv2023b] J. J. Choi˚, F. Castaneda˚,W. Jung˚, Bike Zhang, C. J. Tomlin, and K. Sreenath. Constraint-guided online data selection for scalable data-driven safety filters in uncertain robotic systems. submitted to Transactions on Robotics (under 2nd review), 2023.

Model-based methods like CBFs for safety filter design are prone to model errors. Safety filters that ignore model errors may fail to prevent uncertain dynamics from driving the system state into unsafe regions (left). Data-driven safety filters integrate components that address the impact of uncertain dynamics, which is learned from real trajectory data (right) [Arxiv2023b].

Deep Learning-enabled Safety Filter Design.

While GP regression is effective for systems with limited data, its scalability is limited in higher-dimensional systems with complex uncertainties. For these scenarios, we can employ deep learning-based safety filters, learned through reinforcement learning (RL) and with neural networks. By leveraging the common structure across various safety filter methods, such as HJ reachability and CBFs, we developed a unified safety filter representation and training procedure compatible with any safe set representation. As the learned filter is task-agnostic, unlike many safe RL approaches, it enables safe multi-task learning. Finally, we addressed distribution shift, a key challenge when using simulation data, by framing the enforcement of the policy in distribution as another safety problem.

Relevant Publications:

[RSS2020] J. J. Choi˚, F. Castaneda˚, C. J. Tomlin, and K. Sreenath. Reinforcement learning for safety-critical control under model uncertainty, using control lyapunov functions and control barrier functions. In Robotics: Science and Systems (RSS), 2020.

[L4DC2024] W. Lavanakul˚, J. J. Choi˚, K. Sreenath, and C. J. Tomlin. Safety filters for black-box dynamical systems by learning discriminating hyperplanes. In L4DC, 2024.

[ICML2022] K. Kang, P. Gradu, J. J. Choi, M. Janner, C. J. Tomlin, and Sergey Levine. Lyapunov density models: Constraining distribution shift in learning-based control. In ICML, 2022.

3. Application to Safe Robot and Aviation Autonomy


Reachability-based Approach for Legged Robots

We extended the HJ reachability method, originally designed for continuous systems, to handle discontinuous dynamics, such as ground contacts. We introduced a simple yet impactful solution: a value remapping technique based on the Bellman principle of optimality, enabling the computation of reachable sets for hybrid systems with state resets, external disturbances, and input constraints—without incurring additional computational cost. Our method outperforms state-of-the-art approaches, such as Lyapunov- and Sum-of-Squares (SOS)-based techniques, by recovering significantly larger Regions of Attraction (RoAs) and providing a stabilizing controller that exploits state resets for faster stabilization. Unlike conventional predictive control methods, which require predefined gait sequences, the optimal control derived from our approach eliminates the need for explicit footstep or gait planning, resulting in a simpler, more flexible, and robust control strategy.

Left: Stabilizing the walking motion is translated into a reachability problem where the target set is the walking gait. Right: Then, the existing Dynamic Programming reachability framework are extended to hybrid systems with discontinuous reset maps by incorporating the Value Remapping principle in the algorithm.

Building on the foundation of our first work, we extended our approach by integrating deep learning to overcome the computational challenges of brute-force reachability methods. Traditional reachability methods, such as Hamilton-Jacobi (HJ) PDE solutions, are computationally infeasible for high-dimensional systems like legged robots, prompting us to explore neural network approximations. By using neural networks to approximate the reachability solutions for legged robot dynamics, we get real-time RoA estimates for multiple gaits, which can form a robust framework for gait stabilization and switching. From the learned value functions, we developed a novel RoA-driven gait-switching strategy, allowing the robot to dynamically select the most feasible gait when external perturbations push it outside the RoA of the current gait. Our work is the first to apply deep learning-based reachability to hybrid system dynamics and locomotion control. Unlike traditional deep RL controllers, which operate as opaque black-box models, our framework provides both stable locomotion and interpretable control policies, with explicit RoA estimates to ensure stability. 

Left: Estimate of regions of attractions of various gaits, from a single neural network reachability solution model. Right: Under strong perturbations that lead the robot state to exit the RoA of the current gait, (a) our RoA-driven gait-switching strategy effectively switches the gait and prevents the robot from falling, whereas (b) maintaining the initial gait leads to failure.

Relevant Publications:

[RAL2022] J. J. Choi, Ayush Agrawal, K. Sreenath, C. J. Tomlin, and Somil Bansal. Computation of regions of attraction for hybrid limit cycles using reachability: An application to walking robots. IEEE Robotics and Automation Letters (RA-L), 2022.

[CDC2024] X. Xia˚, J. J. Choi˚, A. Agrawal, K. Sreenath, C. J. Tomlin, and Somil Bansal. Gait switching and enhanced stabilization of walking robots with deep learning-based reachability: A case study on two-link walker. IEEE CDC, 2024.

Flight Envelope Protection for NASA Tiltwing eVTOL vehicle

Advanced Air Mobility (AAM) operations introduce new safety challenges across multiple layers of decision-making, from vehicle-level control to air traffic management (ATM).

Diagram showcasing the HJ reachability framework to verify the safe flight envelope for NASA’s Tiltwing vehicle during mode transitions. The vehicle transitions through a series of trim conditions (green) ranging from 20 to 62.5 m/s airspeed. The full system’s flight envelope is constructed from the computed reachable tubes (blue).

In collaboration with NASA AFRC and Bechamo LLC, we addressed the vehicle-level safety challenges posed by the hybrid dynamics of electric Vertical Takeoff and Landing (eVTOL) vehicles. A key focus was validating safe operations during the critical mode transitions between hover and cruise, where risks like stall can lead to loss of control. I developed a framework that applies HJ reachability computation to the full 6-DOF dynamics of NASA Tiltwing vehicle, ensuring safe recoverability to trim conditions and preventing loss of control during flight transitions.

Relevant Publication:

[JAHS2024] T. Hsu, J. J. Choi, D. Amin, C. J. Tomlin, S. C. McWherter, and M. Piedmonte. Towards flight envelope protection for the NASA tiltwing eVTOL flight mode transition using Hamilton–Jacobi reachability. In Journal of the American Helicopter Society (JAHS), 2024.

Impact

My work has been recognized as a new paradigm for integrating data-driven methods with control-theoretic safety tools. I was invited to co-lead a tutorial article on data-driven safety filters, featured in IEEE Control Systems Magazine, and to lecture at the “Safety Filters for Control” workshop at the 2024 L4DC conference. My work was featured in the IEEE CSM report on “Control for societal-scale challenges: Road map 2030”. The open-source library I developed, which enables users to apply CBF-based safety filters without implementing them from scratch, has earned over 250 stars on GitHub.