NVLink

Key takeaways

NVLink belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect NVLink to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of NVLink from memory before moving on to harder problems.

Reference excerpt

NVLink is a wire-based serial, multi-lane, near-range, communications link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices can use mesh networking to communicate instead of a central hub/switch. The protocol was first announced in March 2014 and uses a proprietary high-speed signaling interconnect (NVHS). For small numbers of GPUs, the NVLink lanes on a single device are sufficient for an all-to-all mesh connectivity. To accommodate higher GPU counts, NVLink since 2018 use a packet-switched architecture, where a central switch can serve up to 32 two-lane ports. The NVSwitch for NVLink 4.0 can produce some simple computation of its own (e.g., sum, broadcast) to reduce the need for communication thanks to the "SHARP" accelerator.

Principle NVLink is developed by Nvidia for data and control code transfers in processor systems between CPUs and GPUs and between GPUs and GPUs. NVLink specifies a point-to-point connection with data rates of 20, 25, and 50 Gbit/s (v1.0/v2.0/v3.0+ resp.) per differential pair. For NVLink 1.0 and 2.0, eight differential pairs form a "sub-link", and two "sub-links", one for each direction, form a "link". Starting from NVlink 3.0, only four differential pairs form a "sub-link". For NVLink 2.0 and higher, the total data rate for a sub-link is 25 GB/s, and the total data rate for a link is 50 GB/s. Each V100 GPU supports up to six links. Thus, each GPU is capable of supporting up to 300 GB/s in total bi-directional bandwidth. NVLink products introduced to date focus on the high-performance application space. Announced May 14, 2020, NVLink 3.0 increases the data rate per differential pair from 25 Gbit/s to 50 Gbit/s while decreasing the number of pairs per NVLink from 8 to 4. With 12 links for an Ampere-based A100 GPU, this brings the total bandwidth to 600 GB/s. The Hopper GPU microarchitecture, announced in March 2022, has 18 NVLink 4.0 links, enabling a total bandwidth of 900 GB/s. Thus, NVLink 2.0, 3.0, and 4.0 all have a 50 GB/s per bidirectional link data rate, and have 6, 12, and 18 links, correspondingly.

Performance The following table shows a basic metrics comparison based on standard specifications:

The following table shows a comparison of relevant bus parameters for real world semiconductors offering NVLink as one of their options:

Real world performance could be determined by applying different data transmission overhead costs, as well as usage rates. Those come from various sources:

128b/130b line code (see, e.g., PCI Express data transmission for versions 3.0 and higher) Link control characters Transaction header Buffering capabilities DMA usage on computer side Those physical limitations usually reduce the data rate to between 90-95 percent of the transfer rate. NVLink benchmarks show an achievable transfer rate of about 35.3 Gbit/s (host to device) for a 40 Gbit/s (2 sub-lanes uplink) NVLink connection towards a P100 GPU in a system that is driven by a set of IBM POWER8 CPUs.

Use with plug-in boards For the various versions of plug-in boards (a yet small number of high-end gaming and professional graphics GPU boards with this feature exist) that expose extra connectors for joining them into a NVLink group, a similar number of slightly varying, relatively compact, PCB based interconnection plugs does exist. Typically only boards of the same type will mate together due to their physical and logical design. For some setups two identical plugs need to be applied for achieving the full data rate. As of now the typical plug is U-shaped with a fine grid edge connector on each of the end strokes of the shape facing away from the viewer. The width of the plug determines how far away the plug-in cards need to be seated to the main board of the hosting computer system - a distance for the placement of the card is commonly determined by the matching plug (known available plug widths are 3 to 5 slots and also depend on board type). The interconnect is often referred as Scalable Link Interface (SLI) from 2004 for its structural design and appearance, even if the modern NVLink based design is of a quite different technical nature with different features in its basic levels compared to the former design. Reported real world devices are:

Quadro GP100 (a pair of cards will make use of up to 2 bridges; the setup realizes either 2 or 4 NVLink connections with up to 160 GB/s - this might resemble NVLink 1.0 with 20 GT/s) Quadro GV100 (a pair of cards will need up to 2 bridges and realize up to 200 GB/s - this might resemble NVLink 2.0 with 25 GT/s and 4 links) GeForce RTX 2080 based on TU104 (with single bridge "GeForce RTX NVLink-Bridge") GeForce RTX 2080 Ti based on TU102 (with single bridge "GeForce RTX NVLink-Bridge") GeForce RTX 3090 based on GA102 (with unique "GeForce RTX NVLink-Bridge (for 30 series products)") Quadro RTX 5000 based on TU104 (with single bridge "NVLink" up to 50 GB/s - this might resemble NVLink 2.0 with 25 GT/s and 1 link) Quadro RTX 6000 based on TU102 (with single bridge "NVLink HB" up to 100 GB/s - this might resemble NVLink 2.0 with 25 GT/s and 2 links) Quadro RTX 8000 based on TU102 (with single bridge "NVLink HB" up to 100 GB/s - this might resemble NVLink 2.0 with 25 GT/s and 2 links)

Service software and programming For the Tesla, Quadro and Grid product lines, the NVML-API (Nvidia Management Library API) offers a set of functions for programmatically controlling some aspects of NVLink interconnects on Windows and Linux systems, such as component evaluation and versions along with status/error querying and performance monitoring. Further, with the provision of the NCCL library (Nvidia Collective Communications Library), developers in the public space shall be enabled for realizing, e.g., powerful implementations for artificial intelligence and similar computation hungry topics atop NVLink. The page "3D Settings" » "Configure SLI, Surround, PhysX" in the Nvidia Control panel and the CUDA sample application "simpleP2P" use such APIs to realize their services in respect to their NVLink features. On the Linux platform, the command line application with sub-command "nvidia-smi nvlink" provides a similar set of advanced information and control.

… excerpt ends here. Continue reading the full article.

Illustrations

Worked examples

Example 1 — a first encounter with NVLink

Start with the simplest possible case. Write down what NVLink claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to NVLink before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about NVLink ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of NVLink

In research: NVLink appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses NVLink in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: NVLink is common in secondary-school and first-year university syllabi. It links to neighbouring topics Computer buses, Nvidia, Serial buses, so understanding it makes those chapters shorter.
In everyday life: Look for NVLink outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study NVLink in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what NVLink means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain NVLink out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is NVLink in simple terms?

NVLink is a wire-based serial, multi-lane, near-range, communications link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices can use mesh networking to communicate instead of a central hub/switch.

Why does NVLink matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study NVLink?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on NVLink.