Information and Noise — How Messages Survive Damaged Channels

Giving Every Character the Same Five Beats So a Machine Can Read It — the Baudot Code

French telegraph engineer Émile Baudot represented characters by combinations of five equal time units in a multiplex system patented in 1874. Moving from a variable human rhythm to fixed-length mechanical selection allowed several senders to share timed slots. Five units provide only thirty-two states, so shifts between letters and figures were necessary; this was neither ASCII nor a completed modern theory of digital communication.

Asking How Many Distinguishable Pulses Fit through Finite Bandwidth — Nyquist

At AT&T, Harry Nyquist analyzed pulse rate, waveform, and bandwidth in a noiseless telegraph channel, making mathematical the reason signals cannot be packed infinitely fast. His 1928 paper extended earlier work from 1924 and became a shoulder for later sampling and communication theory. It did not by itself solve noisy-channel capacity or complete the theorem now called Nyquist–Shannon sampling.

Taking the Logarithm of Possible Choices So Information Adds — Hartley

Bell System researcher Ralph Hartley proposed a quantity based on the logarithm of the number of distinguishable alternatives, so information from successive choices adds. His engineering model connected bandwidth, time, and available signal levels and became an explicit precursor cited by Shannon in 1948. Its emphasis on equiprobable choices and noiseless distinction did not yet provide average uncertainty for general probability distributions or a complete theory of noisy channels.

Translating the Algebra of True and False into Open and Closed Switches — Shannon’s Thesis

Observation and institutions · lens spotlight

As an MIT master’s student, Claude Shannon showed that Boolean operations on true and false propositions could analyze and simplify relay and switching circuits. Abstract logic became a common language for practical circuit design. The 1937 thesis was not yet the theory of entropy and channel capacity; related independent work by Victor Shestakov and the contributions of device physics, memory, and timing remain outside a lone-inventor story.

Setting Meaning Aside to Define Entropy and Channel Capacity — Shannon

Observation and institutions · lens spotlight

Shannon’s two-part A Mathematical Theory of Communication linked a probabilistic source’s average uncertainty to entropy and a noisy channel’s maximum reliable rate to capacity. Abstracting away meaning made telephony, telegraphy, and radio comparable in one language. The theorem says that suitable sufficiently long codes can make error probability arbitrarily small below capacity; it neither guarantees zero error for every finite message nor automatically supplies a practical code.

Using Overlapping Parity Checks to Point at One Wrong Position — Hamming Codes

Richard Hamming began from punched-card errors that stopped computing jobs and published codes that add structured check bits, allowing a syndrome to locate an error. Hamming(7,4) sends four data bits in seven and can correct a single flipped bit. Redundancy became a coordinate system for repair, but the basic code does not correct an arbitrary number of errors or appear unchanged in every storage device.

Merging the Two Rarest Symbols to Shorten the Average Code — Huffman

In a term-paper problem for Robert Fano’s MIT course, graduate student David Huffman devised the algorithm that repeatedly merges the two least-probable symbols and published it in 1952. Frequent symbols receive short codewords and rare ones longer words, yielding an optimal prefix code for known individual-symbol probabilities. It is not an absolute optimum for every compression problem, context model, block code, or dictionary scheme.

Reading a Message as a Finite-Field Polynomial Evaluated at Many Points — Reed and Solomon

Observation and institutions · lens spotlight

MIT Lincoln Laboratory researchers Irving Reed and Gustave Solomon published a code that treats message symbols as a polynomial over a finite field and transmits its values at several points. A low-degree polynomial can be recovered when some values are damaged, forming a strong foundation against burst errors. Their short 1960 paper proposed a new code and theoretical decoding; efficient later decoders and applications in space links, discs, and QR codes did not all arrive fully formed in it.

Drawing a Sparse Check Network that Could Approach the Limit — Gallager

Observation and institutions · lens spotlight

Robert Gallager’s 1960 MIT doctoral thesis analyzed low-density parity-check codes built from sparse parity-check matrices and iterative inference. He sought protection whose storage and computation costs grew moderately with code length, but the method was too expensive for contemporary hardware and was largely forgotten for decades. The thesis predates the shortened 1962 paper and 1963 book, and not every LDPC construction automatically achieves capacity.

Measuring One String by the Shortest Program That Produces It — Kolmogorov

Andrey Kolmogorov’s 1965 paper proposed an algorithmic approach to the information in an individual string: the length of its shortest generating program. It can distinguish a regular string from an incompressible one even when their lengths match. Ray Solomonoff’s earlier work and Gregory Chaitin’s independent development remain visible; universal-machine choices differ only by an additive constant, but the exact complexity is generally uncomputable.

Growing a Dictionary from Repeated Phrases without a Source Probability Table — Lempel and Ziv

Technion researchers Jacob Ziv and Abraham Lempel published a universal sequential compression algorithm that refers back to previously seen strings, learning a dictionary from repetition without a known source distribution. The 1977 LZ77 method remains distinct from 1978’s LZ78 and later variants such as LZW and DEFLATE. Both authors were on sabbatical at US institutions, so Haifa marks their durable institutional and collaborative base rather than a known writing room.

Letting Two Decoders Exchange Soft Guesses and Move toward the Limit — Turbo Codes

Observation and institutions · lens spotlight

Research at ENST Bretagne in Brest combined two recursive convolutional codes, an interleaver, and iterative exchange of bit-likelihood information. The performance reported by Berrou, Glavieux, and Thitimajshima at the 1993 ICC in Geneva startled the field by showing a practical code close to the Shannon limit. Research base and conference site remain distinct, and ‘close’ describes performance under particular channel, length, and decoding assumptions—not a proof of exact capacity achievement.

Making a Square That Factory Scanners Can Read Quickly Even When Damaged — QR Code

A DENSO development team responded to factory demand for fast tracking of varied automotive parts with a compact two-dimensional code carrying more data and Japanese characters. Three finder patterns aid scanning from any orientation, while Reed–Solomon error correction tolerates some damage. The result belongs to the team led by Masahiro Hara, manufacturing needs, and standardization; damage recovery does not guarantee a link’s safety, the content’s authenticity, or privacy.

Polarizing Identical Channels into Nearly Perfect and Nearly Useless Paths — Polar Codes

At Bilkent University, Erdal Arıkan proved that recursively combining and splitting identical binary-input channels makes the synthesized channels approach either near-perfect or near-useless extremes at long block lengths. Sending information only through the good channels produced an explicit, efficiently encodable and decodable capacity-achieving family for a broad class of symmetric binary-input memoryless channels. This does not make it best for every channel and short length, nor mean that all of 5G uses only polar codes.