BOOK: “The Signal and the Noise”
Considered one of the most insightful leaders in Predictive Analytics (PA)
Silver, N. (2012). The signal and the noise: Why so many predictions fail–but some don’t. New York: Penguin.
Summary
Silver (2012) describes the September 11, 2001 attack on the United States (US) was only one of the many “failures of prediction” (p. 11)that greeted us in the new millennium. We were blind to the data. The outcomes of the attack and turmoil of the Twenty-first Century he bests describes as a failure of imagination and the inability to identify such events. That which we could not even imagine because of missing“the signal from the noise.” The data was there, but the inability to discount our own biases and “noise” he condemns and identifies as the main thesis of his book.
As described in his citing of the 9/11 Commission Report findings, the most serious disappointments found were the “failures of imagination” (p. 423). The problem stems from the lack of thought to even contemplate such outcomes as September 11th, the bombing of Pearl Harbor, or the 2009 Recession could have even been a possibility. The data was there, but we failed because of our own biases—the real noise–and the hubris that these “outlier” events could never happen to America.
It took the 2000 presidential election to draw Silver (2012) into the world of predictive analytics, and to create his hallmark website, FiveThirtyEight.com. Silver’s (2012) leveraging of Big Data availability and the aggregation of polls resulted in his exemplary predictions that propelled him to fame (Tetlock & Gardner, 2016). In 2008, he correctly predicted the winner of the presidential contest in forty-nine of fifty states as well as all the winners of all thirty-five Senate seats (Silver, 2012); however, he does warn, that Big Data has yet to prove its impacts on predictive analytics. He states it “…will produce progress—eventually” (p. 12).
The book takes the reader through the failures of mainstream prediction-making by the pundit “hedgehogs;” he places greater reliability on the “foxes” (p. 53). He references the work of Tetlock and Gardner (2016) who describe hedgehogs as having a narrower view of the world and are more likely the chosen “experts” on the news because of their bold, but often failed predictions. Silver (2012) shows his fondness for the foxes who know “many little things” that makes them more reliable, and apt to provide far better predictions.
The failures are caused by our inability to separate the “signal from the noise.” He affirms this is often difficult in such fluid examples of, weather and earthquake predictions, and his primary passion, professional sports betting. He describes his own predictive model he developed in 2003 as a bored consultant (p. 9), the Player Empirical Comparison and Optimization Test Algorithm (PECOTA). During the era of the Moneyball (Lewis, 2004) phenomenon, where statistical approaches took on the mainstream “old guard” of professional sport scouts, Silver’s (2012) own predictive model outpaced even the best models of the day. It became the foundation, and his entry into the realm of predictive analytics.
The book focuses around Thomas Bayes and his famous theorem (Triola, 2005). In Chapter 8, Silver(2012) discusses the history and achievement of Haralabos “Bob” Voulgaris, a successful basketball sports bettor who relies upon foxlike approaches leveraging the principles of Bayesian thinking. From Mr. Voulgaris’ relentless collection of basic court statistics to Twitter postings of key players at the bars the night prior to a big game (p. 239), he draws upon this “past” data to add to his success as a professional gambler. Mr. Voulgaris, like many predictive analysts,depends upon the Bayesian prior probabilities to upgrade his “posterior probabilities”(Triola, 2005) to a refined and lucrative winning outcome.
Silver’s (2012) thesis is vital and timely. In order to separate the “noise” from the “signal” one must continually refine one’s predictions. The Bayesian theorem is that needed mindset and approach that he espouses throughout the book. Determine your prior probabilities, refine them as new information becomes available, and be prepared to issue updated and improved predictions.
Evaluation
The Signal and the Noise is a thoughtful and detailed account of the challenges faced by the predictive analytic community, however, it is not groundbreaking especially regarding the adoption of Bayesian mathematics. The works of Hubbard and Seiersen (2016)and Tetlock and Gardner (2016) too value the reliance upon the use and refinement of prior probability being a critical part of future predictions. These authors, to include Silver (2012), have “mainstreamed” and commercialized Bayes in the current millennium, and I find no fault with that, nor is this review critical of their success.
The book is an excellent account of the challenges of, for example, predicting hurricanes, earthquakes, and elections that impact us all. His work highlights the many challenges of the Big Data revolution to yet take us to the next level of improved analytic successes. His focus is vital and compelling. In order to make better predictions with the available data we must do much better using the tools of Bayes, and more specifically, recognize how to discern our own innate biases that create the noise to the goal of discerning those valuable signals that afford the most optimal predictions.
References
Hubbard, D., & Seiersen, R. (2016). How to measure anything in cybersecurity risk. Hoboken, NJ: John wiley & sons.
Lewis, M. (2004). Moneyball: The art of winning an unfair game. New York: WW norton & company.
Silver, N. (2012). The signal and the noise: Why so many predictions fail–but some don’t. New York: Penguin.
Tetlock, P., & Gardner, D. (2016). Superforecasting: The art and science of prediction. New York: Random house.
Triola, M. F. (2005, October 25). Bayes’ theorem. Retrieved from University of Washington: http://faculty.washington.edu/tamre/BayesTheorem.pdf
Dr. Russo is currently the Senior Data Scientist with Cybersenetinel AI in Washington, DC. He is a former Senior Information Security Engineer within the Department of Defense’s (DOD) F-35 Joint Strike Fighter program. He has an extensive background in cybersecurity and is an expert in the Risk Management Framework (RMF) and DOD Instruction 8510, which implement RMF throughout the DOD and the federal government. He holds a Certified Information Systems Security Professional (CISSP) certification and a CISSP in information security architecture (ISSAP). He has a 2017 Chief Information Security Officer (CISO) certification from the National Defense University, Washington, DC. Dr. Russo retired from the US Army Reserves in 2012 as a Senior Intelligence Officer.
