Trusted Artificial Intelligence Systems Engineering Challenge

The Trusted AI Challenge for Armaments SE is a novel approach to improving the performance of AI-enabled systems. Rather than focusing on improving AI models, this challenge asks student teams to develop SE methods to build and operate systems that provide trustworthy behaviors using components that are less trustworthy. Over the years, engineers have developed design patterns and control algorithms for producing and operating systems that are safe, reliable and secure. This challenge seeks to identify the similar design patterns and control algorithms to provide trustworthy systems behavior.

Another goal of the challenge is to develop engineering methods for design and use of AI-enabled systems in life-critical situations. Some studies have shown that people are more willing to trust AI-enabled systems when the perceived risks are lower. Other studies, however, have shown that people will follow an “Emergency Guide Robot” when artificial smoke filled a hallway and an alarm sounded. Surprisingly, most participants chose to follow the robot, even after the robot had proven itself unreliable, rather than use the clearly marked exit they had entered through. Making lethality a part of the Trusted AI Challenge is a key consideration in developing methods for design and operation of trusted AI-enabled systems.