Navy Phase I Awarded for “Automated Knowledge Base Extraction and Student Assessment”

Effective tools for evaluating the proficiency of warfighters at employing complex systems in an operational setting are essential to ensure operational capability within the Navy and other branches of the DoD. Generating exams with answer keys is a time-consuming process, made more difficult by the complexity of both the systems and the operational employment of those systems.  The burden is further increased when systems are updated frequently.  The proposed software tool will provide a general-purpose capability to efficiently generate test material from technical documents that describe the capabilities of a complex system and the operational employment of that system.  It will enable a person with domain knowledge related to a particular system (but no AI expertise) to effectively team with AI to complete the test generation task vastly more efficiently than would be possible manually, while achieving accuracy and reliability that would be impossible in a fully autonomous system.  The tool will support both updates of existing test questions and answers based on updated source documents, and generation of novel questions specifically targeted at assessing knowledge of new and updated system features and operational procedures.