TCP offload engine (TOE) is a technology used in some network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 Gigabit Ethernet, where the processing overhead of the network stack becomes significant. TOEs are often used as a way to reduce the overhead associated with Internet Protocol (IP) storage protocols such as iSCSI and Network File System (NFS).
Purpose Originally TCP was designed for unreliable low speed networks (such as early dial-up modems) but with the growth of the Internet in terms of backbone transmission speeds (using Optical Carrier, Gigabit Ethernet and 10 Gigabit Ethernet links) and faster and more reliable access mechanisms (such as DSL and cable modems), it is frequently used in data centers and desktop PC environments at speeds of over one gigabit per second. At these speeds, the TCP software implementations on host systems require significant computing power. In the early 2000s, full-duplex gigabit TCP communication could consume more than 80% of a 2.4 GHz Pentium 4 processor, resulting in small or no processing resources left for the applications to run on the system. TCP is a connection-oriented protocol which adds complexity and processing overhead. These aspects include:
Connection establishment using the "three-way handshake" (SYNchronize; SYNchronize-ACKnowledge; ACKnowledge). Acknowledgment of packets as they are received by the far end, adding to the message flow between the endpoints and thus the protocol load. Checksum and sequence number calculations – again a burden on a general-purpose CPU to perform. Sliding window calculations for packet acknowledgement and congestion control. Connection termination. Moving some or all of these functions to dedicated hardware, a TCP offload engine, frees the system's main CPU for other tasks.
Freed-up CPU cycles A generally accepted rule of thumb is that a CPU cycle is needed for each bit sent or received (i.e., one hertz of CPU processing is required to send or receive 1 bit/s of TCP/IP). For example, 5 Gbit/s (625 MB/s) of network traffic requires 5 GHz of CPU processing. This implies that two entire cores of a 2.5 GHz multi-core processor will be required to handle the TCP/IP processing associated with 5 Gbit/s of TCP/IP traffic. Since Ethernet (10GE in this example) is bidirectional, it is possible to send and receive 10 Gbit/s (for an aggregate throughput of 20 Gbit/s). Using the one-cycle-per-bit rule, this equates to eight 2.5 GHz cores. Many of the CPU cycles used for TCP/IP processing are freed-up by TCP/IP offload and may be used by the CPU (usually a server CPU) to perform other tasks, such as file system processing (in a file server) or indexing (in a backup media server). In other words, a server with TCP/IP offload can do more server work than a server without TCP/IP offload NICs.
Reduction of PCI traffic In addition to the protocol overhead that TOE can address, it can also address some architectural issues that affect a large percentage of host-based (server and PC) endpoints. Many older endpoint hosts are PCI bus based, which provides a standard interface for the addition of certain peripherals such as Network Interfaces to Servers and PCs. PCI is inefficient for transferring small bursts of data from main memory across the PCI bus to the network interface ICs, but its efficiency improves as the data burst size increases. Within the TCP protocol, a large number of small packets are created (e.g., acknowledgements), and as these are typically generated on the host CPU and transmitted across the PCI bus and out the network physical interface, this impacts the host computer's IO throughput. A TOE solution, located on the network interface, is located on the other side of the PCI bus from the CPU host so it can address this I/O efficiency issue, as the data to be sent across the TCP connection can be sent to the TOE from the CPU across the PCI bus using large data burst sizes with none of the smaller TCP packets having to traverse the PCI bus.
History One of the first patents in this technology, for UDP offload, was issued to Auspex Systems in early 1990. Auspex founder Larry Boucher and a number of Auspex engineers went on to found Alacritech in 1997 with the idea of extending the concept of network stack offload to TCP and implementing it in custom silicon. They introduced the first parallel-stack full offload network card in early 1999; the company's SLIC (Session Layer Interface Card) was the predecessor to its current TOE offerings. Alacritech holds a number of patents in the area of TCP/IP offload. By 2002, as the emergence of TCP-based storage such as iSCSI spurred interest, it was said that "At least a dozen newcomers, most founded toward the end of the dot-com bubble, are chasing the opportunity for merchant semiconductor accelerators for storage protocols and applications, vying with half a dozen entrenched vendors and in-house ASIC designs." In 2005, Microsoft licensed Alacritech's patent base and, along with Alacritech, created the partial TCP offload architecture that has become known as TCP chimney offload. TCP chimney offload centers on the Alacritech "Communication Block Passing Patent". At the same time, Broadcom also obtained a license to build TCP chimney offload chips.
Types Instead of replacing the TCP stack with a TOE entirely, there are alternative techniques to offload some operations in cooperation with the operating system's TCP stack. TCP checksum offload and large segment offload are supported by the majority of today's Ethernet NICs. Newer techniques like large receive offload and TCP acknowledgment offload are already implemented in some high-end Ethernet hardware, but are effective even when implemented purely in software.
Parallel-stack full offload Parallel-stack full offload gets its name from the concept of two parallel TCP/IP Stacks. The first is the main host stack, which is included with the host OS. The second or "parallel stack" is connected between the Application Layer and the Transport Layer (TCP) using a "vampire tap". The vampire tap intercepts TCP connection requests by applications and is responsible for TCP connection management as well as TCP data transfer. Many of the criticisms in the following section relate to this type of TCP offload.
… excerpt ends here. Continue reading the full article.
