Funding

Our research would not have been possible without the funding provided by a broad range of grants and industry collaborations. We are deeply thankful to all the entities listed below.

Current Projects

Habitats Optimized for Missions of Exploration (HOME)

The scope of the HOME Space Technology Research Institute (STRI)’s research will be driven by two primary operational requirements of NASA’s deep-space habitats: (1) Keep humans alive while they are resident, and (2) Keep the vehicle alive while they are not. A highly autonomous deep-space habitat for human crews requires three major classes of control: autonomy, robotics, and humans – the interactions and interdependencies between these three domains comprise the research landscape for the HOME STRI. The goal of the STRI is to develop validated technology to help NASA ensure safe human habitability during crewed deep-space missions, and to maintain the habitat to support future crews even while uninhabited. Our lab will contribute to the development of early-stage technologies related to explainable AI and human-autonomy teaming.

Please see more information about the institute here.

XCPS: Explainable Cyber-Physical Systems

Greater complexity in (semi-)autonomous systems is a double-edged sword. More sophisticated automation may actually increase, rather than decrease the difficulty for a human operator to comprehend the system’s behaviors. The problem may be exacerbated when the operator is denied a direct observation of the internal functioning of the system, thus having to surmise from sensor data what may be occurring. This can be a daunting task, particularly when some sensors are susceptible to failure. Many such systems, e.g., driverless cars, nuclear power plants, and spacecraft, are safety critical: a small error may lead to unintended, high-regret consequences. The goal of this project is to enable safety-critical (semi-)autonomous systems to inspect their own observations and consequently explain themselves in a language understandable by humans.

Cognitive-Aware Human-Autonomy Teaming

In the foreseeable future, humans will continue to play crucial roles in many, if not all, safety-critical missions, such as disaster relief, space exploration, and robot-assisted surgery. Intelligent machines (i.e., fully and semi-autonomous systems) should effectively function with their human teammates in a cognitively compatible manner. The goal of this project is to enable an intelligent machine to infer and predict its human teammate’s cognitive states (e.g., cognitive load, intention, and trust) and act accordingly (e.g., providing meaningful feedback to its human teammate) to guarantee performance of the collective human-machine team. This is part of a collaborated work with Dr. Sanjay Joshi at UC Davis and Dr.s Allie Anderson and Torin Clark at CU Boulder.

This video shows one set of data we have collected from a human subject while he was remotely controlling a group of five robots in a triangular formulation to acquire a sequence of targets randomly projected on the ground. The video on the left was recorded by an overhead camera while the one on the right was recorded by a pair of gaze tracking glasses. The subject's EEG activities were also recorded (not shown here).

Identification and Control of Neural Cognitive Systems

Current brain-machine interface (BMI) technology is focused on controlling motor prostheses. In the future, however, we will see more devices that directly interface with cognitive systems. The goal of the proposed project is to develop tools for system identification and concomitant control of neural cognitive systems by combining expertise in the areas of system identification and control of technical systems and neural mechanisms underlying cognitive function. We will implant nonhuman primates that have been trained on different cognitive tasks with recording and stimulation electrode arrays in frontal cortical areas to obtain the required high-channel-count data for system identification, for testing the suitability of dynamical models and learning algorithms, and for testing the effectiveness of control algorithms and stimulation strategies with the goal to modify the current state of the system.

As an example, monkeys tend to make mistakes when performing a memory-guided saccade task. We will use system control to improve their performance.

Safety-Assured Imitation Learning for Vision-Based Control of UAS in Complex Environments

Many future military (e.g., scouting and patrolling) and civilian (e.g., search-and-rescue) missions require unmanned aerial systems (UAS) that are capable of flying inside complex and possibly GPS-denied environments (e.g., a forest) safely and effectively. The goal of this project is to tackle one technical challenge hampering the future development and deployment of such UAS: how to guarantee the safety of the UAS, e.g., avoiding collision with nearby obstacles, while maintaining its performance. A natural starting point is imitation learning, which allows an agent to learn what actions to take in an environment from the demonstrations of a human expert. But in a traditional imitation learning setting, it is possible, despite the fact that all expert demonstrations are technically safe, that the agent may generate an unsafe control policy. We aim to integrate imitation learning and formal methods at the fundamental level to develop safety-assured learning strategies.

Mitigating Ecological Destructive Events with a Network of UAVs

Our world is plagued with large-scale ecological disasters. Tragic examples include the Fukushima I Nuclear Power Plant calamity and the Deepwater Horizon oil spill. Today these disasters may be partially mitigated by teams of humans working in hazardous and labor intensive environments. To combat these destructive environmental processes, this project instead envisions the development of a network of UAVs that can combat these disasters, while keeping humans at a safe distance. We will use precision agriculture as a test bed to develop a multi-UAV system that will 1) use cameras mounted on UAVs to map out distribution of pest-infested plants in a heterogeneous grid pattern of crop plants with/without pest infestations, 2) transfer the map from the imaging UAV to a ground station (or a fixed wing aircraft) to compute an optimal mitigation solution, and 3) deploy actuating UAVs to implement the solution (release of natural enemies to the pest population on crops) to the plants with symptoms of pest infestation.

This is part of a collaborated work with Dr. Christian Nansen from the Department of Entomology and Nematology and Dr. Mohsen B Mesgaran from the Department of Plant Sciences. Our project has been featured by two local news stations: ABC10 and CBS13, as well as ASME.

The video one the left shows experimental dispersion of vermiculite mixed with predatory mites. We are in the process of developing algorithms and control systems of the dispensing device so that we can precision-target the release of natural enemies by taking into account drone altitude and speed and account for wind speed and wind direction. The video on the right features one of our preliminary test runs to explore the idea of integrating drone, computer vision, and AI to identify weeds.

UAVs for Large-Scale, Real-Time, Human-Health-Related Data Collection during Wild and Prescribed Fires

For environmental health scientists to study the impacts of wildfires on human health, gaseous pollutant concentration resulting from these burning events and quantitative information on the characteristics of aerosol distribution are needed. Currently ground sensors, manned airplanes, and satellites are wildly used for such purposes. However, the spatial and temporal resolution of data collected by these platforms are relatively low and not adequate for local and regional exposure assessments. Moreover, manned airplanes are prohibitively expensive and can be used only for sparse and sporadic measurements. The goal of this project is to explore the possibility of using a swarm of coordinated small (lightweight) UAVs, equipped with low-cost air-quality sensors, to provide three-dimensional (spatially and temporally resolved), real-time information on aerosol and gaseous pollutant concentrations over a large area affected by nearby wild/prescribed fires. This data should facilitate the local and regional human exposure assessments. This is part of a collaborated work with Dr. Ajith Kaduwela at UC Davis Air Quality Research Center.

Completed Projects

Smart Energy Management for Unmanned Aerial System Operation in Complex Military Missions  (07/18-06/20)

In recent years, unmanned aerial  systems (UAS) and other unmanned systems (UxS) have found their way onto the battleground to extend the reach of military forces, providing essential intelligence, surveillance and reconnaissance (ISR) as well as payload delivery and recovery. The increasing capability of such systems, particularly small and light-weight UxS swarms, has offered the Navy, the Marine Corps, and other military branches a significant opportunity to potentially reduce cost, increase system resiliency, and enhance operation effectiveness. However, one fundamental challenge impeding the wide deployment of small-scale UxS and UxS swarms is energy efficiency particularly in the context of complex military missions. This project aimed to tackle this challenge by investigating the fundamental physics governing the UAS energy performance and improving the mission KPIs through energy-oriented planning and control.This was a collaborated work with Dr. Xinfan Lin. The project was funded by the NEPTUNE 1.0 Program of the ONR.

This video shows our outdoor experiment procedure with which we were able to collect data sufficient to model the effects of wind on the UAV dynamics.

Data-Driven Personalized Training of Next-Generation Workforce (07/16-06/18)

Humans, in the role of either customers, designers, program managers, or workers, will always play a significant role in future manufacturing. However, the systems that the humans are interacting with are becoming increasingly complicated especially with the rise of large-scale industrial cyber-physical systems (CPSs), also called Industrial Internet of Things (IIoT). This project aimed at developing a data-driven method of modeling and analyzing manual expertise of humans while they are interacting with CPSs as well as traditional machines. The insights gained from this project can not only help us to build machines and CPSs that can collaborate with humans more efficiently and effectively but also help us to facilitate the knowledge transfer among generations. This was a collaborated work with Dr. Barbara Linke.

The video on the left shows the gaze data of an expert performing a manual grinding task; while the video on the right shows the gaze data of a novice performing the same task.