The Better Letter: Process, Probability, and Power
Outcome bias is a broader and deeper problem than we might imagine.
Last week, I explained one reason why I believe in God. Some of my reasons are more prosaic. The Houston Astros, convicted cheaters whose “punishment” was pretty much nonexistent, were visiting the division-leading Oakland A's recently. Carlos Correa, a convicted cheater whose “punishment” was pretty much nonexistent, stepped to the plate and God suspended the laws of physics to allow the following to happen. H/T Jonathan Last, to whose outstanding daily newsletter you should subscribe (click here).
If you like The Better Letter, please subscribe, share it, and forward it freely.
Thanks for visiting.
Process
Since I moved west in 1995 and left the Merrill Lynch trading floor in lower Manhattan, my (adopted) hometown San Diego Padres haven’t done very well. The bright spots are few and far between. They started well by making the postseason in 1996 and the terrific 1998 team made it to the World Series, only to lose to an all-time great Yankees team. They made the postseason again in 2005 and 2006 but were immediately eliminated. They haven’t been back since, and haven’t finished above .500 in a decade. They have never won a World Series.
I love them nonetheless.
Even though I won’t be able to sit in my seats in Petco Park this season, I watch games on television and closely follow the team via the (digital) sports pages. I am quite optimistic about this season. We have a promising pitching staff, a slew of talented young players, and the most exciting player in baseball – soon to be the best – 21-year old Fernando Tatis, Jr.
The hated Dodgers still have the best roster in the division and all of baseball. But in a pandemic-shortened season of only 60 games, with twice the number of postseason berths at stake, our chances of playing in October are much, much higher.
We’re talking high variance.
We all know that the outcomes of many activities in life are the result of a combination of both skill and luck. Baseball, like investing (the subject of my day job), is one of these. Understanding the relative contributions of luck and skill can help us assess past results and, more importantly, anticipate future results.
In Major League Baseball, over a traditional 162-game season, the best teams generally win roughly 60 percent of the time. The dreaded Dodgers won 106 games in 2019 (65 percent of the time) while my Padres won but 70. But over shorter stretches, better teams often lose. Last year’s disgusting Dodgers lost to the Washington Nationals in the postseason and the Nats went on to win the World Series even though they didn’t win their division (making the postseason as a wildcard entry) and won 13 fewer games than the *!#*@~! Dodgers during the season.
That’s high variance at work.
Moreover, it’s not unusual to see significant deviations from those percentages for significant periods and noteworthy streaks. For example, my Padres won 14 straight games in 1999, a team record, yet finished the season 14 games under .500.
Since mean reversion establishes that the expected value of the whole season is roughly 50:50 (or slightly above or below that level), 65 percent being great means that there is a lot of randomness in baseball. The best teams still lose nearly four out of every ten games overall, to inferior teams.
Baseball is a “variable game,” where “even the top prospects fail an inordinate amount of the time.” So do the top teams and the top players. Tony Gwynn was the greatest hitter of his generation. He had over 3,000 Major League hits.
He also made over 6,000 outs.
Daniel Kahneman describes randomness as noise and is convinced that “bias has been overestimated at the expense of noise.” Moreover, “noise and bias are independent sources of error, so that reducing either of them improves overall accuracy. . . and the procedures by which you would reduce bias and reduce noise are not the same.” Indeed, “as a first rule, there is more noise than people expect, and there’s more noise than they can imagine because it’s very difficult to imagine that people have a very different opinion from yours when your opinion is right, which it is.” Often wrong, never in doubt.
“And you find variability within individuals, depending morning, afternoon, hot, cold. A lot of things influence the way that people make judgments: whether they are full, or whether they’ve had lunch or haven’t had lunch affects the judges, and things like that.”
Variance makes intuitive sense – the difference between ball four and strike three can be tantalizingly small (even if and when the umpire gets the call right).* So can the difference between a hit and an out, a win and a loss.
What constitutes skill in a field where probabilities dominate? We tend to focus on outcomes, even when we recognize that there is a lot of randomness built into those outcomes. Luck can overcome skill over long periods (like the bad 1999 Padres winning 14 straight) and be the deciding factor at key moments, as when pitcher Cole Hamels was victimized by three consecutive errors in the deciding game of the 2015 American League Divisional Series.
Despite remarkably long odds, Richard Lustig has won the lottery seven different times, Roy Sullivan survived being struck by lightning seven times, and the roulette wheel at the Rio in Las Vegas landed on 19 seven consecutive times. None of those lucky seven anomalies had a skill component.
The key to improving our odds of success in probabilistic fields is to focus on process. A good process won’t always lead to a good outcome, but it provides the best and most consistent opportunity to do so. “You can’t really control the results,” Padres third baseman and cornerstone Manny Machado said this week. “You’ve just got to keep trusting the process, keep working on my approach, my timing, my work ethic, and hopefully the baseball gods will start repaying me.”
A great hitter focuses upon a good approach, his mechanics, being selective, and hitting the ball hard. If he does that he will make outs sometimes, but overall the hits will take care of themselves. Maintaining good process is really hard to do psychologically, emotionally, and organizationally. But it is absolutely imperative for consistent success in probabilistic endeavors.
With respect to my day job, comprehensive research has shown that even professionals are prone to being taken in by recent outcomes – hiring “hot” (lucky) money managers rather than those with more skill. A better approach — in investing and elsewhere — is (as Sam Hinkie would say) to build a good process and trust it.
_______________
* Over a lengthy test period, Major League home-plate umpires made every pitch call correctly on one team roughly twice per season. Among the 114 umpires with at least 5,000 called pitches during that time, the range between the least accurate and most accurate umps is narrower than 4 percentage points, ranging from 86.2 percent at the low end to 90.1 percent at the high end.
Probability
Tom Stoppard’s Rosencrantz and Guildenstern are Dead presents Shakespeare’s Hamlet from the bewildered point of view of two of the Bard’s bit players, the comically indistinguishable nobodies who become headliners in Stoppard’s play. The show opens before our heroes have even joined the action in Shakespeare’s epic. They have been “sent for” and are marking time by flipping coins and getting heads each time.
Guildenstern keeps tossing coins and Rosencrantz keeps pocketing them. Significantly, Guildenstern is less concerned with his losses than in puzzling out what the defiance of the odds says about chance and fate. “A weaker man might be moved to re-examine his faith, if in nothing else at least in the law of probability.”
The coin tossing streak depicted provides us with a chance to consider these probabilities. Guildenstern offers among other possible explanations the one mathematicians and investors should favor: “a spectacular vindication of the principle that each individual coin spun individually is as likely to come down heads as tails and therefore should cause no surprise each individual time it does.” In other words, past performance is not indicative of future results.
Even so, how unlikely is a streak of this length?
The probability that a fair coin, when flipped, will turn up heads is 50 percent (the probability of any two independent sequential events both happening is the product of the probability of both). Thus, the odds of it turning up twice in a row is 25 percent (½ x ½), the odds of it turning up three times in a row is 12.5 percent (½ x ½ x ½), and so on. Accordingly, if we flip a coin 10 times (one “set” of ten), we would only expect to have a set end up with 10 heads in a row once every 1024 sets.
Rosencrantz and Guildenstern got heads nearly 100 consecutive times. The chances of 100 in a row are 1/7.9 x 1031. In words, we could expect it to happen once in 79 million million million million million (that’s 79 with 30 zeros after it) sets. By comparison, the universe is about 13.9 billion years old, in which time only about [1 with 17 zeros after it] seconds have elapsed. Looked at another way, if every person who ever lived (around 110 billion of us) had flipped a 100-coin set simultaneously every second since the beginning of the universe (again, about 13.9 billion years ago), we could expect all of the 100 coins to have come up heads two times.
If anything like that had happened to you (especially on a bet), you’d agree with Nassim Taleb that the probabilities favor a loaded coin.** But, then again, while 100 straight heads are less probable than 99, which are less probable than 98, and so on, any exact order of tosses is as likely (actually, unlikely) as 100 heads in a row. The maths are exactly the same. We notice the unlikelihood of 100 in a row because of the pattern. More “normal” combinations look random and thus expected.
Looked at another way, if there will be one “winner” selected from a stadium of 100,000 people, each person has a 1 in 100,000 chance of winning. But we aren’t surprised when someone wins, even though the individual winner is shocked.
The point here is that the highly improbable happens all the time. In fact, much of what happens is highly improbable. As then Washington State (now Mississippi State) football coach Mike Leach explained about college football upsets, “Everybody's all surprised every time this stuff happens. It surprises me everybody gets surprised because it happens every year like this that there are surprises. The most surprising thing would be if there weren't any surprises. So, therefore, in the final analysis, none of it's really that surprising.”
We shouldn’t be surprised when the market remains “irrational” far longer than seems possible. But we are.
There are an estimated 97 million songs built off of just 12 notes, but far less than one percent of those songs resonate while the rest live in oblivion. Some of the oblivion songs and artists are great. Many of the resonating songs and artists are mediocre. Some of the resonating songs and artists are truly dreadful. How can it make sense, for example, that Milli Vanilli sold 30 million records while Eva Cassidy spent her entire career singing in obscurity?
High variance. As Churchill said, “All this shows how much luck there is in human affairs, and how little we should worry about anything except doing our best.”
Even after 100 heads in a row, the odds of the next toss being heads remain one-in-two (the “gambler’s fallacy” is committed when one assumes that a departure from what occurs on average or in the long-term will be corrected in the short-term). We look for patterns to convince ourselves that we have found a “secret sauce” that justifies our making big bets on less likely outcomes. In this regard, we are dumber than rats – literally.
In numerous studies (most prominently those by Edwards and Estes, as reported by Philip Tetlock in Expert Political Judgment), the stated task was predicting which side of a “T-maze” holds food for the subject rat. Unbeknownst both to observers and the rat, the maze was rigged such that the food was randomly placed (without a pattern), but 60 percent of the time on one side and 40 percent of the time on the other.
The rat quickly “gets it” and waits at the “60 percent side” every time and is thus correct 60 percent of the time. Human observers keep “seeing” patterns and choose sides in rough proportion to recent results. As a consequence, the humans are right only 52 percent of the time. They (we!) are much dumber than rats. Overall, we insist on rejecting probabilistic strategies that accept the inevitability of randomness and error.
As Kahneman explained, “to compute probabilities you need to keep several possibilities in your mind at once. It’s difficult for most people. Typically, we have a single story with a theme. People have a sense of propensity, that the system is more likely to do one thing than the other, but it’s quite different from the probabilities where you have to think of two possibilities and weigh their relative chances of happening.”
We prefer to think linearly, manufacturing a storyline, in effect, with a beginning, middle, and end. That’s why we are so susceptible to the narrative fallacy. We inherently prefer stories to data. Contingencies and (perhaps random) consequences don’t correspond to the way we like to see the world. We are — pretty much all the time — either looking backward and creating a pattern to fit events and constructing a story that explains what happened along with what caused it to happen, fitting what we see or assume we see into a preconceived narrative, or both.
Dealing effectively with probabilities requires that we recognize the power of the random and contingent. No matter how good a story we have concocted with respect to what we expect to happen, no matter how careful our analysis, stuff happens that can and often does mess up and mess with our hopes, dreams, and schemes.
If a weather forecaster says that there is an 80 percent chance of rain and it remains sunny, instead of waiting to see if it rains 80 out of 100 times when his or her forecast called for an 80 percent chance of rain, we race to conclude — perhaps based upon that single instance — that the forecaster isn’t any good.
We’re terrible with probabilities.
_______________
** A coin flip, with exactly 50/50 odds, requires 67,600 flips to be 99 percent sure it is not a weighted coin. Oh, and the odds aren’t really exactly 50/50.
Power
Sometimes, somewhere under the rainbow, skies aren’t blue and, despite great process, our dreams don’t come true. When that happens, especially when the hold-up is recalcitrant people and seemingly inflexible structures, the urge to impose outcomes — outcomes we’re convinced are just and right — is staggering.
Historically, for example, conservatism has amplified Chief Justice Marshall’s view in Marbury v. Madison that the role of the judiciary is to say what the law is, not what it should be. As Sarah Isgur wrote recently, legal conservatism “is supposed to be about process and not outcome.” Today, however, alleged conservatives routinely reject process-oriented scholarship, legal or otherwise. Outcomes are all.
Similarly, President Trump repeatedly invokes legally dubious (I’m being generous) executive orders despite conservatism’s long-held fear of executive overreach. In his most recent set, not only did the president bypass Congress, the branch of government constitutionally in charge of federal spending, but he intends on funding his new order of jobless benefits by taking money from FEMA.
Perhaps worst of all, it’s an artistic failure — it’s a straight rip-off of House of Cards. In Season 3, Frank Underwood sought to consolidate his power via America Works.
Congress refuses to approve it, so the president uses FEMA money to fund it. FEMA objected to the television scenario…
…but hasn’t yet commented on the real thing.
Anyway, the political left does the same thing, of course. In The Cult of Smart, Fredrik deBoer argues specifically for a change in goals for education: equality of outcomes (hereinafter referred to as “dumbing things down”). In his celebrated best-seller, How to Be an Antiracist, Ibram X. Kendi describes how he wants outcomes to control, no matter how obtained.
“There is no such thing as a nonracist or race-neutral policy … If discrimination is creating equity, then it is antiracist. If discrimination is creating inequity, then it is racist.”
Inequality is defined as any difference in racial outcome, irrespective of base rates and other outcomes. Accordingly, “the most threatening racist movement is not the alt-right’s unlikely drive for a white ethno-state, but the regular American’s drive for a ‘race-neutral’ one.”
Kendi demands constitutionally mandated and governmentally enforced equality of racial outcomes across-the-board. Like Yuval Noah Harari, it’s all about power and using it to get a desired result: “the only remedy to past discrimination is present discrimination. The only remedy to present discrimination is future discrimination.” He’s a totalitarian.
The root of my general conservatism (classic liberalism) is my doubt about the perfectibility of humans. I am skeptical about human nature. I think human sinfulness is self-evident.
As every parent can attest, we all have totalitarian instincts about what we think is right or important. If taking action upon those instincts increases our power, control, or status, they can seem irresistible. And, to be clear, Christians and Christianity are not remotely immune.
Unfortunately, we humans can’t begin to agree on what is right or important, even when we broadly share the same perspectives, as the recent Democratic presidential nomination process demonstrates. That’s why totalitarian governments are so prone to threatening, discriminating against, imprisoning, purging, and killing their citizens.
So, the next time you feel like deemphasizing or bypassing process for outcome, even (especially) when you think the outcome right, true, or just, you might want to consider how you’d feel if the other side (or some other side) were in charge and imposed their favored outcomes on you.
It isn’t a process you’d trust.
Totally Worth It
Jimmy Fallon, the original Hamilton cast and The Roots sing “Helpless” from the show with at-home instruments, wherein Alex’s “top-notch brain” is used as a pick-up line.
The wild story of Creem. Is Christianity under attack? The most delightful thing I saw this week, unless it was this. The silliest. The sweetest. The saddest. The most ironic. The weirdest. The coolest. The best retort. Bureaucracy at work. Drone delay at Fenway. If you really could foresee the future, wouldn’t you keep working at a hedge fund and make a fortune that way?
Benediction
This week’s benediction features Eva Cassidy, obscure regional band singer, taken by cancer at 33, “discovered” by NPR and some Brits after her death, and now a gift for the ages. The song is a gospel-infused Curtis Mayfield classic, People Get Ready.
Contact me via rpseawright [at] gmail [dot] com or on Twitter (@rpseawright). Don’t forget to subscribe and share.
Thanks for reading.
Issue 25 (August 14, 2020)