The following is the Office of the Director, Operational Test & Evaluation (DOT&E) 2017 annual report. It was released in late January 2018.
From the Report:
Today, the building material of choice for our weapon systems is software. The amount of software source lines of code in today’s
weapon systems is growing exponentially. Software does not just increase the functionality of these systems, it fundamentally defines
the weapon system. However, as the number of lines of code increases so does the complexity of the system and cybersecurity vulnerabilities. The implications for T&E are profound. We are now making more changes that effect system capability through software than through hardware. For example, the F-35 Joint Strike Fighter’s effectiveness in combat relies on software missionvdata loads, which work in conjunction with the avionics software and hardware to drive sensor search parameters. These files arevcritical for F-35 identification and correlation of threat and friendly radar signals. This increased dependence of system capabilities on software dictates that T&E must become a continuous, risk-based process for the life cycle of the system.
As weapon systems increase their dependency on software, the potential cybersecurity attack surface also increases. DOT&E has been a steady voice in the need to improve the cybersecurity posture of our systems, networks, and human interactions with networked systems. DOT&E has advocated for improved cybersecurity testing to identify critical problems and their operational impact and is currently funding the development of automated test tools. The cybersecurity section, later in this report, provides a number of recommendations to improve the Department’s cybersecurity posture based on the past efforts of this office.
The cybersecurity of our weapons and networks needs increased attention. In support of that, the Department needs to evolve how we monitor our cybersecurity posture. The two-phase Cooperative Vulnerability and Penetration Assessment (CVPA) and Adversarial Assessment (AA) approach currently outlined in DOT&E test guidance is necessary to help inform the cybersecurity posture of DOD systems, but is not sufficient. This testing has greatly improved our understanding of cyber vulnerabilities, but in addition to dedicated assessments, DOD systems must be built to include technologies to continuously monitor cybersecurity, and automatically find and patch software vulnerabilities. Periodic assessments by Red Teams alone are not adequate, because the security of system software can change at any time due to operator errors, or adversary cyber-attacks. Red Teams are critical, but by themselves will never scale to meet the enormity of the cybersecurity challenge facing the Department.
One of my top priorities will be to update cybersecurity and risk-based testing guidance to reflect best business practices. Cybersecurity testing needs to move forward in the acquisition life cycle so that it can influence the system architecture from early development. I will advocate for additional resources for the development of automated software testing tools and the threat teams who use these tools. I will continue to advocate for rigorous cybersecurity testing and include evaluations of cybersecurity
vulnerabilities in my assessments of systems. In the context of the rapid pace of software development, I will look for ways to align
T&E activities with the velocity of the development of software systems.
