computer science

Single instruction, multiple data

Single instruction, multiple data is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Single instruction, multiple data rather than just read about it. In short: Single instruction, multiple data (SIMD) is a type of parallel computing (processing) in Flynn's taxonomy. SIMD describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously.

Key takeaways

Single instruction, multiple data belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Single instruction, multiple data to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Single instruction, multiple data from memory before moving on to harder problems.

Reference excerpt

Single instruction, multiple data (SIMD) is a type of parallel computing (processing) in Flynn's taxonomy. SIMD describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously. SIMD can be internal (part of the hardware design) and it can be directly accessible through an instruction set architecture (ISA), but it should not be confused with an ISA. Such machines exploit data level parallelism, but not concurrency: there are simultaneous (parallel) computations, but each unit performs exactly the same instruction at any given moment (just with different data). A simple example is to add many pairs of numbers together, all of the SIMD units are performing an addition, but each one has different pairs of values to add. SIMD is especially applicable to common tasks such as adjusting the contrast in a digital image or adjusting the volume of digital audio. Most modern central processing unit (CPU) designs include SIMD instructions to improve the performance of multimedia use. In recent CPUs, SIMD units are tightly coupled with cache hierarchies and prefetch mechanisms, which minimize latency during large block operations. For instance, AVX-512-enabled processors can prefetch entire cache lines and apply fused multiply-add operations (FMA) in a single SIMD cycle.

Confusion between SIMT and SIMD

SIMD has three different subcategories in Flynn's 1972 Taxonomy, one of which is single instruction, multiple threads (SIMT). SIMT should not be confused with software threads or hardware threads, both of which are task time-sharing (time-slicing). SIMT is true simultaneous parallel hardware-level execution, such as in the ILLIAC IV. SIMD should not be confused with Vector processing, characterized by the Cray 1 and clarified in Duncan's taxonomy. The difference between SIMD and vector processors is primarily the presence of a Cray-style SET VECTOR LENGTH instruction.

History The first known operational use to date of SIMD within a register was the TX-2, in 1958. It was capable of 36-bit operations and two 18-bit or four 9-bit sub-word operations. The first commercial use of SIMD instructions was in the ILLIAC IV, which was completed in 1972. Vector supercomputers of the early 1970s such as the CDC Star-100 and the Texas Instruments ASC could operate on a "vector" of data with a single instruction. Vector processing was especially popularized by Cray in the 1970s and 1980s. Vector processing architectures are now considered separate from SIMD computers: Duncan's Taxonomy includes them whereas Flynn's Taxonomy does not, due to Flynn's work (1966, 1972) pre-dating the Cray-1 (1977). The complexity of Vector processors however inspired a simpler arrangement known as SIMD within a register. The first era of modern SIMD computers was characterized by massively parallel processing-style supercomputers such as the Thinking Machines Connection Machine CM-1 and CM-2. These computers had many limited-functionality processors that would work in parallel. For example, each of 65,536 single-bit processors in a Thinking Machines CM-2 would execute the same instruction at the same time, allowing, for instance, to logically combine 65,536 pairs of bits at a time, using a hypercube-connected network or processor-dedicated RAM to find its operands. Supercomputing moved away from the SIMD approach when inexpensive scalar multiple instruction, multiple data (MIMD) approaches based on commodity processors such as the Intel i860 XP became more powerful, and interest in SIMD waned. The current era of SIMD processors grew out of the desktop-computer market rather than the supercomputer market. As desktop processors became powerful enough to support real-time gaming and audio/video processing during the 1990s, demand grew for this type of computing power, and microprocessor vendors turned to SIMD to meet the demand. This resurgence also coincided with the rise of DirectX and OpenGL shader models, which heavily leveraged SIMD under the hood. The graphics APIs encouraged programmers to adopt data-parallel programming styles, indirectly accelerating SIMD adoption in desktop software. Hewlett-Packard introduced Multimedia Acceleration eXtensions (MAX) instructions into PA-RISC 1.1 desktops in 1994 to accelerate MPEG decoding. Sun Microsystems introduced SIMD integer instructions in its "VIS" instruction set extensions in 1995, in its UltraSPARC I microprocessor. MIPS followed suit with their similar MDMX system. The first widely deployed desktop SIMD was with Intel's MMX extensions to the x86 architecture in 1996. This sparked the introduction of the much more powerful AltiVec system in the Motorola PowerPC and IBM's POWER systems. Intel responded in 1999 by introducing the all-new SSE system. Since then, there have been several extensions to the SIMD instruction sets for both architectures. Advanced vector extensions AVX, AVX2 and AVX-512 are developed by Intel. AMD supports AVX, AVX2, and AVX-512 in their current products.

Disadvantages With SIMD, an order of magnitude increase in code size is not uncommon, when compared to equivalent scalar or equivalent vector code, and an order of magnitude or greater effectiveness (work done per instruction) is achievable with Vector ISAs. ARM's Scalable Vector Extension takes another approach, known in Flynn's Taxonomy more commonly known today as "Predicated" (masked) SIMD. This approach is not as compact as vector processing but is still far better than non-predicated SIMD. Detailed comparative examples are given at Vector processor § Vector instruction example.

Chronology

Hardware

CPUs Small-scale (64 or 128 bits) SIMD became popular on general-purpose CPUs in the early 1990s and continued through 1997 and later with Motion Video Instructions (MVI) for Alpha. SIMD instructions can be found, to one degree or another, on most CPUs, including Hewlett-Packard's (HP) PA-RISC Multimedia Acceleration eXtensions (MAX), IBM's AltiVec and Signal Processing Engine (SPE) for PowerPC, Intel's MMX and iwMMXt, Streaming SIMD Extensions (SSE), SSE2, SSE3 SSSE3 and SSE4.x, AMD's 3DNow!, ARC's ARC Video subsystem, SPARC's VIS and VIS2, Sun's MAJC, ARM's Neon technology, MIPS' MDMX (MaDMaX) and MIPS-3D. Intel's AVX-512 SIMD instructions process 512 bits of data at once.

… excerpt ends here. Continue reading the full article.

Illustrations

Worked examples

Example 1 — a first encounter with Single instruction, multiple data

Start with the simplest possible case. Write down what Single instruction, multiple data claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Single instruction, multiple data before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Single instruction, multiple data ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Single instruction, multiple data

In research: Single instruction, multiple data appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Single instruction, multiple data in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Single instruction, multiple data is common in secondary-school and first-year university syllabi. It links to neighbouring topics Classes of computers, Digital signal processing, Flynn's taxonomy, so understanding it makes those chapters shorter.
In everyday life: Look for Single instruction, multiple data outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Single instruction, multiple data in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Single instruction, multiple data means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Single instruction, multiple data out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Single instruction, multiple data in simple terms?

Why does Single instruction, multiple data matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Single instruction, multiple data?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Single instruction, multiple data.