Dolphin PCIe Fabric Communications Library Part 1:
Hardware and Architectures
Today’s embedded systems are made up of powerful processing subsystems, each of which is uniquely designed to serve a particular function. In high performance embedded computing systems, each of these subsystems are often fully functional processing nodes themselves and the limiting factor when optimizing for the highest system performance often lies in the processor-to-processor data paths. Passing data from processing node to processing node in the most efficient way possible, with the lowest latency and highest throughput, can often have the greatest impact on system performance.
Traditional defense systems built with VME technology have a shared parallel bus, across which all processing nodes communicate. These systems suffer from low overall throughput, not only due to slow data bus transfer speeds, but also because only one node can communicate at a time, further hindering performance. Once a node begins to transfer data, other nodes need to wait for their turn to access the shared bus, adding to longer and uncertain latencies.
Modern systems can also use Ethernet to pass data from node to node. With switched Ethernet architectures, nodes can communicate in parallel, all but eliminating the bottlenecks of a shared bus. However, processor speeds and capabilities have grown much faster than Ethernet speeds, and once again, data paths have become a performance bottleneck.
Almost every contemporary processor today uses the PCI Express (PCIe) bus as a high-speed interconnect for on-board peripherals. In most processing systems, the PCIe interface offers the fastest data path to and from the processor. Unfortunately, the PCIe bus was never designed to offer straightforward processor-to-processor communications, as its original design based off the PCI parallel bus was for a single “master” host.
Curtiss-Wright Defense Solutions has partnered with Dolphin Interconnect Solutions to bring their eXpressWare PCIe fabric software to the embedded VPX world. Uniquely optimized to take advantage of hardware features such as DMA and multi-core processing, eXpressWare can be used to exploit the highest levels of data fabric performance for the rugged defense industry.
In this first of our two part series, we introduce the use of fabrics for high-performance embedded systems, and focus on the hardware and architectural options available to the systems designer. In part two, we present several different software interfaces provided for applications development, comparing their advantages and tradeoffs. Finally, we present performance benchmarks using a variety of Curtiss-Wright 3U VPX modules.
Download our Dolphin PCIe Fabric Communications Library Part 1 Hardware and Architectures White Paper to read more about:
- Fabric technologies: PCIe SRIO, InfiniBand and Ethernet
- Common use of fabrics PCIExpress
- SBC configurations with Dolphin eXpressWare