DoD’s Digital Transformation & CMMC – Caution and Concern

Key Finding:

In addition to contractor reputation, the survey explored the DoD’s digital transformation and CMMC. In general the findings show that there is a sense of pessimism about DoD’s digital maturation/transformation. Additionally, there is little optimism regarding CMMC (Cybersecurity Maturity Model Certification)

Introduction

This article is the third in a series on Lone Star’s 2020 DoD contractor reputation survey. In June 2020, Lone Star obtained 117 survey responses from a pool of U.S. defense experts. This survey was one in a series of similar surveys our firm has conducted over many years.

The Transformation

DoD’s digital transformation has many facets. Moving information and processing to the Pentagon’s Joint Enterprise Defense Infrastructure (JEDI) cloud computing environment is one example. Complex networked kill chains and weapons is another. Databases and analytics for logistics and sustainment is still another part of the trend. And, of course, the use of Artificial Intelligence (AI) for a wide range of applications, including processing huge flows of intelligence information is frequently cited.

Part of the transformation is cybersecurity. The most visible effort in this category is DoD’s Cybersecurity Maturity Model Certification or CMMC.

One goal of the 2020 survey was to get a sense of how our respondents saw DoD’s progress. No transformation is easy. The “Revolution in Military Affairs” of the Cold War had notable failure and challenges, in addition to notable successes. We wondered how this transformation was seen.

Reputation and Digitization

DoD digital vs net reputation graphicWe assigned a “digital” score to each of the 57 firms in our survey, representing the rough ratio of digital content in each firm’s business.

Shown here, we see there is some (weak) correlation between the “being digital” and positive reputation. Several of the top-rated firms were highly digital.

However, DXC Technology inherited the business and brand reputation of the Computer Sciences Corporation, which was rated highly digital, but also had one of the worst brand reputations tested.

Being digital may help, but clearly does not assure a good reputation. Generally, solutions and software firms did better than IT services firms.

Among firms rated as highly digital, but scoring below median in reputation, two exceptions to the services/solution were noted. Oracle and Google both rated below median in reputation rankings.

 

 

How DoD Fared as a Smart Customer

The survey asked five questions on DoD’s digital journey. Respondents reacted to questions posed in form of assertions. Respondents could agree or disagree on a five-choice Likert scale, from “Strongly Agree” to “Strongly Disagree.” There were 95 responses to this portion of the survey.

DoD Digital Progress

Respondents were not confident DoD’s digital transformation was on track:

  • They tended to agree with the proposition that CMMC would hinder access to innovative firms who might otherwise choose to do business with DOD.
  • They were neutral on the proposition that DoD had been wise to create a bespoke process for cybersecurity rather than adopt an ISO standard.
  • On successful adoption of artificial intelligence, the raw agree/disagree score was neutral. However, when weighting for “strong” agreement or disagreement, the weighted score swung negative (respondents don’t agree DoD “is making great progress” in AI.
  • Respondents disagreed with the proposition DoD was a competent customer for a collection of advanced digital capabilities (AI, Cloud services, Digital Twins, and the Internet of Things).
  • The strongest disagreement was with the proposition that DoD was a “successful adopter of data-driven improvement using Big Data and Analytics.”

This pessimistic view of DoD’s digital journey led to questioning prior initiatives. Critics have charged that past DoD digital “innovations” such as CMMI, Ada, the 1553 bus, and Mil-Std-1556 served to only isolate DoD from mainstream technology.

These critiques are largely untestable, although some evidence might be offered for the performance, parts costs, and test set availability for 1553 compared to commercial bus and network standards.

We had seen two prior studies showing a negative correlation between Contractor Performance Assessment Report (CPAR) scores and CMMI level achievements. These studies were proprietary and were done several years ago.  But we decided to test this hypothesis and researched the CMMI level of each of the 57 firms in our reputation survey.

There was a significant (52%) correlation between negative impressions and CMMI scores. There was no correlation between positive impressions and CMMI scores.  Since much of CMMC seems to take inspiration from CMMI, these findings suggest respondent pessimism might not be purely cynical.

Summary

DoD and the services have made noteworthy progress in adopting digital technologies. Many successes over the past decade can be cited, in particular since the 2014 announcement of the “Third Offset Strategy.”

But digital transformation (like any other) is hard. Large commercial organizations, including GE, Ford and P&G  have also had noteworthy failures. Respondents to our survey seem to agree with critics of both Congress and DoD. They see digital transformation as not meeting expectations of warfighter needs.

You can review the 2020 DoD Contractor Reputation Survey blog series parts 1 and 2 HERE.

The complete “Lone Star Analysis 2020 DoD Contractor Reputation Survey Report” will be available soon. To request a copy of the full report, please send your request to: lsa-marketing@lone-star.com.