Data-at-Rest Build vs. Buy: Technology Readiness Levels
When a new vehicle is being planned and designed, engineers, system architects, program managers, and acquisition personnel typically debate three procurement options for the subsystems:
- Design their own subsystem and build it themselves
- Hire a contractor to design and build the subsystem
- Locate and buy a commercial off-the-shelf (COTS) subsystem
Each approach may get you to the same goal, but each has positive and negative aspects. Those decisions can only be made internally. The white paper Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices suggests some ideas that should be introduced in any such debate or trade study.
The white paper focuses on data-at-rest (DAR) devices known as network-attached storage (NAS) or network file servers. These NAS devices are Ethernet-based and allow network clients to use the device as local storage. A NAS example is shown in Figure 1.
This blog focuses on one of those considerations: Technology Readiness Levels. The other factors are flexibility, lead time, loaners, encryption, quality, reliability, export, cost, and risk. The perspective is that of a defense contractor in the United States (US).
Technology Readiness Levels
Technology Readiness Levels (TRL) are defined as a method of estimating technology maturity of Critical Technology Elements (CTE) of a program during the acquisition process. They are determined during a Technology Readiness Assessment (TRA) that examines program concepts, technology requirements, and demonstrated technology capabilitiesi. A CTE is a new or novel technology that a platform or system depends on to achieve successful development or production or to successfully meet a system operational threshold requirementii. In the example discussed in this blog, the CTE is a NAS device.
The TRL system is based on a scale from 1 to 9, with 9 being the most mature and 1 being the least mature technology. It enables consistent, uniform discussions of technical maturity across different types of technologies. When making a build versus buy decision regarding a new NAS device, the program manager and other authorities will consider the recommended TRLs when assessing program risk.
Scenario #1: Design and Build a new NAS Device In-house
Engineering time and resources must be committed to the effort if the program manager and team decide to design and build their own new NAS device in-house. Since the NAS device does not exist, it must be considered a TRL number 1 or 2 at best. Such a low TRL number indicates an increase in technical and schedule risk for the program.
The risks may be mitigated if the in-house engineering team has previously designed a NAS device. However, any new development effort has technical risks, which may be exhibited anywhere along the program path.
As noted in the white paper Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices, the earlier a technical issue is uncovered, the better it is for the program. If a technical issue is discovered late in the program, it may be difficult and more costly to fix the issues.
It is one thing to design a new NAS device. The production of such a device has its own level of risk. Build issues can be discovered as the manufacturing process is worked out. Supply issues can be discovered along the way as well.
Scenario #2: Hire a contractor to design and build a new NAS device
If the program manager and team instead decide to have a third-party sub-contractor design and build the new NAS device, then engineering time and resources must still be committed to managing that effort. Less time and resources will be required for management but will still be required. Since the new NAS device does not exist, it must be considered a TRL number 1 or 2 at best. Such a low TRL number indicates an increase in technical and schedule risk for the program.
The technical risks may be mitigated if the third-party sub-contractor has previously designed a NAS device. As previously mentioned, any new development effort has technical risk, which may be exhibited anywhere along the program path. With a third-party design, less control can be exerted over the daily development and testing process.
As noted in the white paper, Data-At-Rest Build vs. Buy Considerations for Deployed Storage Devices, the earlier a technical issue is uncovered, the better it is for the program. If a technical problem is discovered late in the program, it may be difficult and more costly to fix the issues. When working with a third-party sub-contractor, the program manager and team will want to conduct regular design meetings to discuss and uncover technical issues.
Again, the production of such a new NAS device has its own level of risk. Build issues can be discovered as the manufacturing process is worked out. Supply issues can be discovered along the way as well.
Scenario #3: Purchase an existing COTS NAS Device
If the program manager and team decide to purchase an existing COTS NAS device, then the TRL number will be much higher than for scenarios 1 or 2. The TRL number will depend on whether the design is complete, tested, and deployed. The program manager and team need to know precisely the state of maturity.
If the COTS NAS device has been deployed, it may be rated at a TRL level of 7 or higher. If the existing COTS product has not been deployed, it may be rated 6 or lower. If the COTS product is rated 5 or lower, it may still be in development. A TRL rating of 1 to 3 may indicate a future or roadmap product which indicates that it does not really exist yet.
More technical risk exists with a lower TRL number for a COTS NAS device, and less technical risk exists with a higher TRL. The example NAS device in Figure 1 has been deployed in over a dozen applications, from unmanned air vehicles (UAV) and unmanned underwater vehicles (UUV) to fixed-wing aircraft and fighters. The example NAS is considered TRL 9.
The program manager and team must consider many factors when deciding where and how to obtain a new NAS device. One of those factors should be the TRL number of the NAS.
The TRL number will provide a relative measurement of maturity. The maturity (or lack thereof) will affect technical and schedule risk.
If the program manager and team are seeking to avoid risk, they should seek a NAS device with a high TRL number. A level 9 will indicate that the NAS has not only been designed and tested but also that it has been deployed. Deployment of a NAS device will help discover any technical issues.
The program manager and team must not simply select a NAS device based on the TRL number alone. Other factors must be evaluated: storage capacity, speed, size, weight, power, cost, and data protection (encryption). Each program will have different requirements, but TRL should be a consideration in any trade study regarding a NAS device.
A new NAS device, whether designed in-house or by a third party, will have a low TRL number initially. After development, testing, and deployment, that NAS device can increase to TRL 9 like the example NAS in Figure 1, which is already at a TRL 9 number.
Include technology readiness level in your evaluation regarding a NAS device.