Operand forwarding

CPU optimization technique to improve instruction-level parallelism

Operand forwarding (or data forwarding) is an optimization in pipelined CPUs to limit performance deficits which occur due to pipeline stalls.^[1]^[2] A data hazard can lead to a pipeline stall when the current operation has to wait for the results of an earlier operation which has not yet finished.

Example

ADD A B C  #A=B+C
SUB D C A  #D=C-A

If these two assembly pseudocode instructions run in a pipeline, after fetching and decoding the second instruction, the pipeline stalls, waiting until the result of the addition is written and read.

Without operand forwarding
1	2	3	4	5	6	7	8
Fetch ADD	Decode ADD	Read Operands ADD	Execute ADD	Write result
	Fetch SUB	Decode SUB	stall	stall	Read Operands SUB	Execute SUB	Write result

With operand forwarding
1	2	3	4	5	6	7
Fetch ADD	Decode ADD	Read Operands ADD	Execute ADD	Write result
	Fetch SUB	Decode SUB	stall	Read Operands SUB: use result from previous operation	Execute SUB	Write result

In some cases all stalls from such read-after-write data hazards can be completely eliminated by operand forwarding:^[3]^[4]^[5]

With operand forwarding (enhanced)
1	2	3	4	5	6
Fetch ADD	Decode ADD	Read Operands ADD	Execute ADD	Write result
	Fetch SUB	Decode SUB	Read Operands SUB: use result from previous operation	Execute SUB	Write result

Technical realization

The CPU control unit must implement logic to detect dependencies where operand forwarding makes sense. A multiplexer can then be used to select the proper register or flip-flop to read the operand from.

References

^ "CMSC 411 Lecture 19, Pipelining Data Forwarding". University of Maryland Baltimore County Computer Science and Electrical Engineering Department. Retrieved 2020-01-22.
^ "High performance computing, Notes of class 11". hpc.serc.iisc.ernet.in. September 2000. Archived from the original on 2013-12-27. Retrieved 2014-02-08.
^ Gurpur M. Prabhu. "Computer Architecture Tutorial". Sections "Forwarding". and "Data Hazard Classification".
^ Dr. Orion Lawlor. "Pipelining, Pipeline Stalls, and Operand Forwarding".
^ Larry Snyder. "Pipeline Review".

External links

Introduction to Pipelining

Processor technologies

Models

Abstract machine
Stored-program computer
Finite-state machine
- with datapath
- Hierarchical
- Deterministic finite automaton
- Queue automaton
- Cellular automaton
- Quantum cellular automaton
Turing machine
- Alternating Turing machine
- Universal
- Post–Turing
- Quantum
- Nondeterministic Turing machine
- Probabilistic Turing machine
- Hypercomputation
- Zeno machine
Belt machine
Stack machine
Register machines

Architecture

Instruction set
architectures

Types	Orthogonal instruction set CISC RISC Application-specific EDGE TRIPS VLIW EPIC MISC OISC NISC ZISC VISC architecture Quantum computing Comparison Addressing modes
Instruction sets	Motorola 68000 series VAX PDP-11 x86 ARM Stanford MIPS MIPS MIPS-X Power POWER PowerPC Power ISA Clipper architecture SPARC SuperH DEC Alpha ETRAX CRIS M32R Unicore Itanium OpenRISC RISC-V MicroBlaze LMC System/3x0 S/360 S/370 S/390 z/Architecture Tilera ISA VISC architecture Epiphany architecture Others

Execution

Instruction pipelining	Pipeline stall Operand forwarding Classic RISC pipeline
Hazards	Data dependency Structural Control False sharing
Out-of-order	Scoreboarding Tomasulo's algorithm Reservation station Re-order buffer Register renaming Wide-issue
Speculative	Branch prediction Memory dependence prediction

Parallelism

Level	Bit Bit-serial Word Instruction Pipelining Scalar Superscalar Task Thread Process Data Vector Memory Distributed
Multithreading	Temporal Simultaneous Hyperthreading Simultaneous and heterogenous Speculative Preemptive Cooperative
Flynn's taxonomy	SISD SIMD Array processing (SIMT) Pipelined processing Associative processing SWAR MISD MIMD SPMD

Processor
performance

Transistor count
Instructions per cycle (IPC)
- Cycles per instruction (CPI)
Instructions per second (IPS)
Floating-point operations per second (FLOPS)
Transactions per second (TPS)
Synaptic updates per second (SUPS)
Performance per watt (PPW)
Cache performance metrics
Computer performance by orders of magnitude

Types

By application	Embedded system Microprocessor Microcontroller Mobile Ultra-low-voltage ASIP Soft microprocessor
Systems on chip	System on a chip (SoC) Multiprocessor (MPSoC) Cypress PSoC Network on a chip (NoC)
Hardware accelerators	Coprocessor AI accelerator Graphics processing unit (GPU) Image processor Vision processing unit (VPU) Physics processing unit (PPU) Digital signal processor (DSP) Tensor Processing Unit (TPU) Secure cryptoprocessor Network processor Baseband processor

Word size

Core count

Components

Functional units	Arithmetic logic unit (ALU) Address generation unit (AGU) Floating-point unit (FPU) Memory management unit (MMU) Load–store unit Translation lookaside buffer (TLB) Branch predictor Branch target predictor Integrated memory controller (IMC) Memory management unit Instruction decoder
Logic	Combinational Sequential Glue Logic gate Quantum Array
Registers	Processor register Status register Stack register Register file Memory buffer Memory address register Program counter
Control unit	Hardwired control unit Instruction unit Data buffer Write buffer Microcode ROM Counter
Datapath	Multiplexer Demultiplexer Adder Multiplier CPU Binary decoder Address decoder Sum-addressed decoder Barrel shifter
Circuitry	Integrated circuit 3D Mixed-signal Power management Boolean Digital Analog Quantum Switch