When a new vehicle is being planned and designed, engineers, system architects, program managers, and acquisition personnel typically debate three procurement options for the subsystems:
- Design their own subsystem and build it themselves
- Hire a contractor to design and build the subsystem
- Locate and buy a commercial off-the-shelf (COTS) subsystem
Each approach may get you to the same goal, but each has positive and negative aspects. Those decisions can only be made internally. The white paper Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices suggests some ideas that should be introduced in any such debate or trade study.
The white paper focuses on data-at-rest (DAR) devices known as network attached storage (NAS) or network file servers. These network-attached storage devices are Ethernet-based and allow network clients to use the device as local storage. A network-attached storage example is shown in Figure 1.
This blog focuses on one of those considerations: Technology Readiness Levels. The other factors are flexibility, lead time, loaners, encryption, quality, reliability, export, cost, and risk. The perspective is that of a defense contractor in the United States (US).
Technology Readiness Levels
Technology Readiness Levels (TRL) are defined as a method of estimating technology maturity of Critical Technology Elements (CTE) of a program during the acquisition process. They are determined during a Technology Readiness Assessment (TRA) that examines program concepts, technology requirements, and demonstrated technology capabilitiesi. A CTE is a new or novel technology that a platform or system depends on to achieve successful development or production or to successfully meet a system operational threshold requirementii. In the example discussed in this blog, the CTE is a NAS device.
The Technology Readiness Levels system is based on a scale from 1 to 9, with 9 being the most mature and 1 being the least mature technology. It enables consistent, uniform discussions of technical maturity across different types of technologies. When making a build versus buy decision regarding a new network-attached storage device, the program manager and other authorities will consider the recommended Technology Readiness Levels when assessing program risk.
Scenario #1: Design and Build a new Network Attached Storage Device In-house
Engineering time and resources must be committed to the effort if the program manager and team decide to design and build their own new network-attached storage device in-house. Since the NAS device does not exist, it must be considered a Technology Readiness Level number 1 or 2 at best. Such a low Technology Readiness Level number indicates an increase in technical and schedule risk for the program.
The risks may be mitigated if the in-house engineering team has previously designed a network-attached storage device. However, any new development effort has technical risks, which may be exhibited anywhere along the program path.
As noted in the white paper Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices, the earlier a technical issue is uncovered, the better it is for the program. If a technical issue is discovered late in the program, it may be difficult and more costly to fix the issues.
It is one thing to design a new network-attached storage device. The production of such a device has its own level of risk. Build issues can be discovered as the manufacturing process is worked out. Supply issues can be discovered along the way as well.
Scenario #2: Hire a contractor to design and build a new Network Attached Storage device
If the program manager and team instead decide to have a third-party sub-contractor design and build the new network attached storage device, then engineering time and resources must still be committed to managing that effort. Less time and resources will be required for management but will still be required. Since the new network attached storage device does not exist, it must be considered a Technology Readiness Level number 1 or 2 at best. Such a low Technology Readiness Level number indicates an increase in technical and schedule risk for the program.
The technical risks may be mitigated if the third-party sub-contractor has previously designed a network attached storage device. As previously mentioned, any new development effort has technical risk, which may be exhibited anywhere along the program path. With a third-party design, less control can be exerted over the daily development and testing process.
As noted in the white paper, Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices, the earlier a technical issue is uncovered, the better it is for the program. If a technical problem is discovered late in the program, it may be difficult and more costly to fix the issues. When working with a third-party sub-contractor, the program manager and team will want to conduct regular design meetings to discuss and uncover technical issues.
Again, the production of such a new network attached storage device has its own level of risk. Build issues can be discovered as the manufacturing process is worked out. Supply issues can be discovered along the way as well.
Scenario #3: Purchase an existing COTS Network Attached Storage Device
If the program manager and team decide to purchase an existing COTS network attached storage device, then the Technology Readiness Level number will be much higher than for scenarios 1 or 2. The Technology Readiness Level number will depend on whether the design is complete, tested, and deployed. The program manager and team need to know precisely the state of maturity.
If the COTS network attached storage device has been deployed, it may be rated at a Technology Readiness Level level of 7 or higher. If the existing COTS product has not been deployed, it may be rated 6 or lower. If the COTS product is rated 5 or lower, it may still be in development. A Technology Readiness Level rating of 1 to 3 may indicate a future or roadmap product which indicates that it does not really exist yet.
More technical risk exists with a lower TRL number for a COTS network attached storage device, and less technical risk exists with a higher Technology Readiness Level. The example network attached storage device in Figure 1 has been deployed in over a dozen applications, from unmanned air vehicles (UAV) and unmanned underwater vehicles (UUV) to fixed-wing aircraft and fighters. The example network attached storage is considered Technology Readiness Level 9.
Summary
The program manager and team must consider many factors when deciding where and how to obtain a new network attached storage device. One of those factors should be the Technology Readiness Level number of the network attached storage.
The Technology Readiness Level number will provide a relative measurement of maturity. The maturity (or lack thereof) will affect technical and schedule risk.
If the program manager and team are seeking to avoid risk, they should seek a network attached storage device with a high TRL number. A level 9 will indicate that the network attached storage has not only been designed and tested but also that it has been deployed. Deployment of a network attached storage device will help discover any technical issues.
The program manager and team must not simply select a network attached storage device based on the Technology Readiness Level number alone. Other factors must be evaluated: storage capacity, speed, size, weight, power, cost, and data protection (encryption). Each program will have different requirements, but Technology Readiness Level should be a consideration in any trade study regarding a network attached storage device.
A new network attached storage device, whether designed in-house or by a third party, will have a low Technology Readiness Level number initially. After development, testing, and deployment, that network attached storage device can increase to TRL 9 like the example network attached storage in Figure 1, which is already at a Technology Readiness Level 9 number.
Include technology readiness level in your evaluation regarding a network attached storage device.
i Technology Readiness Level (TRL) - AcqNotes
ii Critical Technology Element - AcqNotes