The following is the Office of the Director, Operational Test & Evaluation (DOT&E) 2024 annual report. It was released 0n Jan. 31, 2025.
From the report
U.S. military Service members deserve to be equipped with combat-credible systems that meet the demands of today’s evolving threats and tomorrow’s unpredictable challenges. In a rapidly changing global security environment, where adversaries continue to develop sophisticated capabilities across all domains, our commitment to providing timely, rigorous, and independent evaluations has never been more critical. The U.S. military operates in an era of accelerating technological advancements, increasingly complex operational environments, and evolving global challenges. The DoD must rapidly and rigorously test and evaluate its systems to determine if they are not only operationally effective and suitable, but also survivable and lethal across contested domains.
To achieve these goals, DOT&E advanced our Strategy and Implementation Plan (I-Plan) this year. This plan lays out a clear path to make a strategic shift in test and evaluation (T&E) processes and builds on years of effort. These efforts started under Honorable Robert Behler, when he looked at the science and technology (S&T) of T&E and released the S&T Plan in January 2021. That work was codified under Honorable Nickolas Guertin in June 2022 with five strategic pillars and championed by Honorable Douglas Schmidt during his first six months tenure, after he was sworn in as Director, Operational Test and Evaluation on April 8, 2024. The I-Plan capitalizes on the latest advances in S&T to modernize our professional skillsets, enable our agility and efficiency, and inspire trust and confidence in system performance under wartime conditions. The plan’s five pillars represent DOT&E’s commitment to testing in ways that reflect operational realities, adapting to new technologies, and ensuring that our warfighters are equipped with the best possible capabilities.
Codifying Strategy in Policy and Guidance
This rapid pace of change from a technological, operational, and global geopolitical scale, demands that we must embrace a strategic shift in how we conduct T&E. Toward that end, in December 2024 Honorable Douglas Schmidt signed out a new policy for operational test and evaluation (OT&E) and live fire test and evaluation (LFT&E). This policy – a DoD instruction accompanied by five corresponding DoD manuals – will drive the DoD forward by making strategic shifts in how we execute the T&E mission. I am honored to have played a role in implementing this culmination of a multi-year effort to update policy and guidance. These documents encourage early engagement from OT&E and LFT&E stakeholders and engagement in OT&E and LFT&E activities across the acquisition life cycle. The policy calls for:
- Using the latest advances in science and technology to both plan tests and evaluate outcomes,
- Requiring the integration of all relevant information into OT&E and LFT&E planning and assessment activities,
- Mandating the consideration of risk as part of the test planning process,
- Requiring that OT&E and LFT&E planning start in parallel with the initiation of the program,
- Considering the time and resources required to correct deficiencies identified in test, and,
- Ensuring T&E against the full domain of kinetic and non-kinetic threats to address the rapidly evolving threat landscape.
This policy and the corresponding manuals encourage us to lean forward in a measured way. We need to research, pilot, and inform how our future T&E practices leverage digital transformation, digital engineering models, and data collected from across the acquisition life cycle. The intricacies of software and artificial intelligence (AI) models make these practices imperative because the complexity they impose cannot be adequately tested in dedicated operational tests alone.
The I-Plan’s goals, key challenges, and how the policy advances the current state of T&E is spelled out in each of the five pillars presented below.
Pillar 1 – Test the Way We Fight
Realistic operational conditions are the cornerstone of defensible T&E. Test conditions that systems are exposed to in operational tests routinely reveal new vulnerabilities and failure modes that should be remediated to avoid failure in combat. To reflect the evolving battlefield, there is a critical need for modeling and simulation (M&S) to undergo verification, validation, and accreditation (VV&A) with live data.
Pillar 1 focuses on developing T&E frameworks that reflect joint and coalition operations, particularly in highly contested and congested environments, such as cyber and electromagnetic spectrum (EMS) realms. DOT&E’s new policy and guidance emphasizes the importance of using all credible sources of information to inform OT&E and LFT&E plans and assessments.
To improve our ability to test in operationally realistic environments, we are leveraging training and exercises for data. Retrieving data from theater assessments to support the continual VV&A of complex M&S is crucial to this effort’s success. Early engagement by operational testers will ensure that program requirements incorporate testability considerations, so systems are instrumented to provide this critical data. Another challenge is testing in commercial cloud environments to ensure our cyber capabilities are robust enough to withstand attacks while adapting to new operational and economic realities.
Download the document here.
