The WatchersHow knowing the enemy became an institution

Espionage is sometimes called the second-oldest profession, and it is certainly older than the state. Long before standing armies or bureaucracies, a ruler who wanted to survive needed to know what rivals intended. The Hebrew Bible has Moses sending twelve scouts ahead into Canaan, and Joshua's two spies sheltered at Jericho by Rahab. In China around 500 BC, Sun Tzu gave the final chapter of The Art of War entirely to espionage — "The Use of Spies" — classifying agents into five types and insisting that foreknowledge cannot be had from gods or omens, only from people who know the enemy.

Kadesh shows the craft's dark twin, born in the same breath: wherever there are spies, there is deception — the deliberate feeding of false information to make an enemy misjudge. For most of history, all of this was personal and improvised. A king, a few trusted agents, a payment in gold or favour; no office, no archive, no successor. Intelligence was a relationship, not an institution — and when the ruler fell, the network vanished with him.

As early-modern states grew richer and more anxious, espionage scaled up — but it still organized itself around a single powerful figure. Walsingham built, and often personally funded, a standing network of informants and code-breakers that intercepted the Catholic plots against Elizabeth, most decisively the Babington conspiracy. In Venice, the Council of Ten ran one of history's most feared intelligence and counter-intelligence machines, complete with paid informers and the bocche di leone — stone "lion's mouth" postboxes where citizens dropped anonymous denunciations. In France, Richelieu fielded his own agents and mail-opening cabinet.

These were the spymasters: under them, intelligence became a recognized instrument of statecraft, methodical and well-resourced. But it remained tied to a patron and a personality. Walsingham paid for much of his service out of his own purse, and when he died in 1590, deep in debt, much of his hard-built network frayed with him. The state had learned to value intelligence — but it had not yet learned to institutionalize it. That step would take another three centuries.

The 19th and early 20th centuries finally turned the spymaster's personal web into a permanent institution — a bureau with a budget, files, ranks, and a career path that outlived any individual. Austria-Hungary's Evidenzbüro, founded in 1850, was among the first standing military-intelligence departments; Tsarist Russia's Okhrana grew into a sprawling secret police. In 1909, amid a public panic about German spies, Britain created the Secret Service Bureau, which split into the agencies now known as MI5 — counter-espionage at home, under the army's Vernon Kell ("K") — and MI6, foreign intelligence under the navy's Cumming ("C").

Bureaucratized intelligence brought enormous reach, and a new set of pathologies. The Dreyfus Affair in France — in which a military counter-intelligence office framed an innocent Jewish officer for treason and then closed ranks to protect the lie even as evidence of his innocence mounted — became the cautionary tale of the age: a permanent secret service can defend the state, but it can also defend itself, at the cost of the truth and an innocent man. Intelligence was now a standing organ of government, with all the power, and all the dangers, that implies.

As agencies matured, so did tradecraft — the disciplined art of recruiting and running human sources. Officers learned to read the motives that turn a person into an agent (often summarized by the blunt acronym MICE: money, ideology, compromise, ego), to pass material through dead drops and cover identities, and above all to run double agents. Britain's Twenty Committee, under the Oxford don John Masterman, made this an art form: its "XX" (double-cross) network controlled nearly every German agent in the country and scripted what they reported home.

From that control flowed history's great deceptions. Operation Mincemeat floated a corpse dressed as a British officer ashore in Spain, its briefcase stuffed with false invasion plans, and persuaded Hitler to reinforce Greece while the Allies struck Sicily. Operation Fortitude conjured an entire phantom army in southeast England to fix German divisions at Calais while the real blow fell on Normandy. The mirror image of all this is counter-intelligence — hunting the other side's agents inside your own house — a hall of mirrors so disorienting that, in the Cold War, the CIA's obsessive molehunts nearly paralyzed the agency from within.

Collecting secrets is only half the job; the harder half is analysis — turning a flood of fragments into knowledge a leader will act on. And the recurring failure is rarely a lack of information. It is the failure to recognize, or to believe, the information already in hand. In Pearl Harbor: Warning and Decision (1962), Wohlstetter framed this as signal versus noise: the true indicators of the 1941 attack were real but buried in a din of misleading chatter, and snapped into a clear pattern only after the bombs fell. Months earlier, Stalin had explicit warnings of Operation Barbarossa from multiple sources and waved every one away, certain they were British plots to drag him into war.

The institutional answer to surprise was more institution. The United States, blindsided at Pearl Harbor, built a permanent civilian analytic agency — the CIA, established in 1947 — and the vast intelligence "communities" that followed around the world. Yet the deepest lesson of this whole history is a humbling one that no reorganization has ever fixed: intelligence can inform power, but it cannot compel power to listen. The watcher's oldest frustration — to see the danger clearly and not be believed — turns out to be built into the craft itself, as true for Cassandra as for the analyst at her screen tonight.