For the past 20 years, embedded systems have increasingly emphasized the use of COTS technology, using generic products as building blocks for both processing and I/O functions to create application-specific systems that meet a wide range of requirements. The use of modular COTS products reduces the initial system’s development schedule and risk while enabling incremental system upgrades as new technology becomes available.
Most systems today use the PCI Mezzanine Card (PMC) standard, accompanied by PCI, to implement modular I/O. However, today’s processor chipsets from Intel, Freescale, and others are increasingly switching to PCI Express as a local bus, making PMC’s successor, the PCI Express compatible XMC (VITA 42), a more logical choice. This change in interconnect technology presents systems integrators with the challenge of building systems that embrace newer and faster I/O modules, support older I/O modules when required, and allow I/O interfaces to be migrated incrementally as the COTS ecosystem moves forward. Consequently, PMC/XMC hybrid module technology is blending old and new, but trade-offs must be considered including: trace length limits, onboard power draws, and firmware and software architecture support for both interfaces.
The evolution of modular PMC, then XMC, choices
In the early days of COTS, the “building block” was a complete card assembly such as a 6U VME card. A wide range of cards was available on the marketplace for both processing and I/O, and the open-standard interface between products was defined by the VMEbus. As technology improved, it became possible to put more and more functionality on a single 6U card, and it became more challenging for board vendors to pick the right set of features to make products that met the needs of a wide range of users. This led to the development of a range of both standard and proprietary I/O expansion options, usually with some kind of “mezzanine” card that was mounted onto a 6U carrier card. A key success factor for this “building block” approach is the use of modular I/O products built around open standards. The PMC module became a logical choice.
As technology progressed and PCI Express became more popular, however, the successor to the PMC was developed: the ANSI/VITA 42 Switched Mezzanine Card, or XMC. As with the PMC architecture, the XMC standard consists of a base standard that defines the mechanical interfaces and connectors and a set of “protocol standards” that map specific interconnect protocols to the XMC architecture.
PMC provides modular I/O
The most popular modular I/O standard for embedded systems is the PMC module. PMC modules are based on the PCI Local Bus standard and support a wide range of throughputs, from 32-bit 33 MHz (133 MBps) up to 64-bit 133 MHz (1,067 MBps), while maintaining forward- and backward-compatibility between modules and carriers. Although PMC modules have been in use since the early 1990s, the standard was only formally ratified as ANSI/IEEE Std 1386.1 in 2001. The PMC standard is formally a layered architecture and potentially makes possible a choice of local buses, yet as a practical matter the standard is rarely used with any interface other than PCI. At this point, there are hundreds of different PMC modules and carriers available from dozens of manufacturers, giving systems integrators a wide range of choices for PCI-based modular I/O.
By using COTS modular I/O products such as PMCs to implement the “ins and outs” of the system, systems integrators can implement functionally dense systems while still isolating the external interfaces from the internal processing modules. This approach makes it possible to take advantage of modular I/O products across several programs without custom development. And processing modules can be incrementally upgraded over the system life cycle without changing the I/O modules, allowing insertion of better technology with minimal integration effort and risk.
Modular I/O and XMC
The ANSI/VITA 42.3, or XMC.3, standard defines the PCI Express version of XMC. XMC.3 supports x1, x2, x4, x8, and x16 PCI Express connections for up to 4 GBps throughput in each direction.
The mechanical specifications for XMC modules and carriers include a number of features to support forward- and backward-compatibility with PMC modules. A typical XMC module with PMC and XMC connectors is shown in Figure 1.
The PMC connectors (P11 through P14) and the XMC connector (P15 and optionally P16) are in different locations on the module, and the connector heights are controlled to support the following interoperability requirements.
· All compliant XMC carriers can support both new XMC modules and legacy PMC modules
· All compliant XMC modules can be used on new XMC carriers and potentially on legacy PMC carriers. An XMC module with both XMC and PMC interfaces and the XMC connectors depopulated is compatible with legacy PMC carriers.
Additionally, the XMC standard intentionally prioritizes “hybrid” carriers (defined as a carrier that supports both PMC and XMC modules) over “hybrid” modules. These tenets are implemented on the expectation that modules will migrate relatively quickly to XMC-only versions and that carriers are more likely to need backward-compatibility with legacy PMC modules for a longer period. This concept of a hybrid PMC/XMC module is explored in the next section, along with its relevant design considerations.
Hybrid PMC/XMC module: Melding past with present
When considering migration support of a single I/O solution between PMC and XMC, one approach is simply to design two products: a PMC version and an XMC version. This potentially provides a more optimized product for each of the two interfaces, but requires that the user change products when the carrier card moves from PMC to XMC. Because the modules are relatively expensive, this adds cost to the carrier card upgrade and potentially introduces some integration risk through the use of different hardware, firmware, and software. In applications where a mix of carrier card technologies is used, this approach can also increase the spares requirements because two separate cards may be used in the system.
Alternatively, a single module can be designed that supports both PMC and XMC interfaces. A multichannel Serial FPDP module (utilizing the ANSI/VITA 17.1 SFPDP standard) provides a good example. A block diagram of this approach is shown in Figure 2. Although this is relatively straightforward to implement, there are some issues that need to be considered.
Consideration #1: Trace length limits
First, the PCI Local Bus standard imposes trace length limits that require either that the FPGA be located very close to the P11/P12/P13 connectors, or that a PCI-to-PCI bridge be used. With the XMC connector populated, a Xilinx Virtex-5 FPGA in the FF1136 package cannot be placed near the PMC connectors and still support the VITA 20 conduction-cooling interface in the middle of the card. Therefore, a PCI-to-PCI bridge can be used to isolate the PMC interface from the FPGA and meet the trace length requirements for operation at 133 MHz.
The PCI-to-PCI bridge provides two additional benefits. First, it allows the module to support both 3.3 V and 5 V PCI signal levels, which cannot be supported with a direct connection to a Virtex-5 FPGA. Second, it allows the FPGA side of the PCI-to-PCI bridge to always use PCI-X 64-bit 133 MHz mode, making the PCI interface firmware in the FPGA more straightforward. The net effect is that the module’s PCI interface supports a wider range of PCI configurations than a direct FPGA interface could support, increasing the PMC interoperability of the module.
Consideration #2: Onboard power draws
Another set of trade-offs has to do with the module’s onboard power supplies. For a hybrid module to be interoperable with both standards, it needs to support drawing power from either the PMC interface or the XMC interface, while also avoiding problems if both interfaces are present. Our example PMC interface provides 3.3 V and 5 V power, while the XMC interface provides a prime power rail that is specified to be 5 V and 12 V at the carrier card’s option. The onboard power conversion is designed to operate with either power input and does not “cross-connect” any of the power rails in the event both connectors are installed and connected to the carrier. Similar issues need to be considered for the reset and JTAG interfaces, again to make sure that the module works when installed in PMC, XMC, and hybrid PMC/XMC carrier cards.
Consideration #3: Firmware and software architectures
Finally, the firmware and software architecture needs to support both interfaces. In our example hybrid module, the firmware has been designed with a common set of front-end code (Serial FPDP core, DDR3 memory controllers, DMA, and buffer managers) and a back end that supports either PCI-X or PCI Express. To maximize the space available in the FPGA for the user’s application, firmware is in separate PCI-X or PCI Express versions, and the appropriate version is loaded into the FPGA when the card is powered up. A single version can also be configured with both interfaces at the cost of higher gate utilization. Because the PCI and PCI Express standards are compatible at the configuration and software level, one software driver can support both versions of the firmware, using the appropriate interface (PMC/PCI or XMC/PCI Express) to communicate with the front-end firmware and manage the Serial FPDP links.
The final solution: All-in-one PMC/XMC capabilities
The end result is a hybrid PMC/XMC hybrid module with effective trace length capabilities and onboard power draws, in addition to common front-end firmware and software driver. The hybrid module is then usable in both PMC and XMC environments without functional changes other than the 2x improvement in throughput when using PCI Express. The same architecture (Xilinx Virtex-5 FPGA, PCI-to-PCI bridge, and PMC and XMC interfaces) can be reused in a wide range of I/O interface applications that require migration between past and future standards without the need to upgrade and requalify I/O interfaces.
Andrew Reddig is President/CTO and cofounder of TEK Microsystems, Inc., which was founded while Andrew was a student at M.I.T. He can be contacted at [email protected].
TEK Microsystems Inc.