Detecting Concept Drift in Object Detection Models: A Collaborative AI-Human Approach for Defense Applications

Concept drift, a persistent challenge in machine learning, can lead to the deterioration of model performance over time. This is a pressing issue for the U.S. armed forces as they strive to integrate machine learning and other AI-based systems into their defense operations. In response, the U.S. Army Research Laboratory is keen to identify when concept drift affects the object detection models they employ. This study introduces a novel drift detection method to establish a metric that alerts users to concept drift and supports retraining as needed. The method leverages AI-human interaction to bolster the model’s trustworthiness and facilitate retraining when new data becomes available. Results demonstrate the method’s capacity to detect drift through calculated Drift Avoidance Values for target objects, such as tanks and humans. The system gathers diverse data—images, AI-generated confidence values, and human corrections—to inform these metrics. A “You Only Look Once” (YOLO) object detection model, initially trained on thousands of images, was tested on a set of 200 images, with retraining based on user corrections. This process produced a 3% accuracy improvement with only 200 additional images, a significant gain relative to typical retraining requirements. The study highlights the value of human input in reinforcing model reliability and demonstrates the method’s effectiveness in adapting to performance degradation. This approach supports the sustainability of object detection systems and decision-making in defense applications.

Aiden Garcia-Rubio is a mechanical engineering graduate student at The University of Texas at San Antonio (UTSA), earning a bachelor’s degree in Fall 2021 from Texas Tech University. Aiden attained the Engineer in Training (EIT) designation before beginning his graduate studies. He further supplemented his experience by obtaining yellow belt certification in Lean Six Sigma.  Aiden’s current research focuses on harnessing the power of machine learning and computer vision to detect objects through image and feature analysis under Dr. Krystel Castillo. Aiden takes inspiration from studying the cutting-edge object detection capabilities of machine learning and how it can contribute to the evolving landscape of technology and manufacturing.

Logan Heck is a graduate student/researcher at The University of Texas at San Antonio, holding a Bachelor's degree earned in Spring 2023. During his academic journey, Logan secured an internship at Albany Engineered Composites, gaining valuable industry experience. Currently immersed in his second year as a Ph.D. student, Logan focuses on advanced manufacturing and data analytics under Dr. Krystel Castillo advisory. A key emphasis of Logan's current research lies in harnessing the power of machine learning to detect intricate patterns within data. By utilizing the cutting-edge capabilities of machine learning, Logan aspires to contribute meaningfully to the evolving landscape of technology and manufacturing.