In July 2016, Political Calculations launched a new series, "Examples of Junk Science", where we specifically focused on presenting examples of where flawed analysis and the misuse of data led to the spread of false or misleading information in the fields of investing, finance and economics.
In doing so, we've also sought to present explanations for why the examples of failed analysis and abused data would lead to results that are not valid, which we hope will be beneficial for analysts who seek to make positive contributions in these fields, where the ultimate goal is to improve the general quality of analysis that is produced. We hoped to do that in part by highlighting real and recently produced examples, so that anyone can more quickly recognize and be better able to challenge pseudoscientific junk should they come across it.
Shortly after starting the series however, we realized that the personalities of many of the producers of the worst examples of junk science were as much a part of the story as the flawed analysis and data abuse they generate. To describe these people as "nasty pieces of work" just doesn't quite cover it, which is why we also committed to discussing the psychological elements shared by the worst offenders as extensively as we have.
The Examples of Junk Science (EOJS) series represents the most collaborative effort in Political Calculations' history, and we cannot thank all those who agreed to participate and freely share their examples enough! We would also like everyone who has recognized the value of the series by sharing the various examples we've presented with their own readers or by extending the discussions in their own forums.
Going forward, EOJS will be a series with occasional episodes rather than a weekly or biweekly affair, but rest assured, we have an unfortunately ample supply of additional examples! For now however, since we're so close to the end of the year and our annual hiatus here at Political Calculations, it was high time to organize the examples we've already presented to date to hopefully facilitate your holiday reading enjoyment.
| The Examples of Junk Science Series | ||
|---|---|---|
| Date | Post | Description |
| 19 August 2009 | How to Detect Junk Science | We first presented the checklist we would later use as the connecting element among all the examples of junk science that we've discussed to date as part of this series back in 2009! We do plan to clean up its presentation a bit and to also update the examples of each category in a brand new post sometime in the near future! |
| 8 July 2016 | Falsifiabilty Fail | The hallmark of real science is that theories can be shown to be false whenever their predictions are contradicted by real world data. But for those purposefully engaged in pseudo science, the outcomes of experiments and analyses are often predetermined so that the theory being put forward cannot possibly be shown to be false. We reveal how that was done in one especially egregious example in the inaugural post of the Example of Junk Science series! |
| 21 July 2016 | Valuation Fallacies | The replication of research findings is often a thankless task, but also a very necessary one to verify that the results of an experiment or analysis are valid. In this case, a very different conclusion about the meaning of high price-to-earnings (P/E) ratios is made after the results of a financial analyst were replicated. |
| 29 July 2016 | A Psychological Profile for Pseudoscientists | There are some pretty disturbing personality traits that characterize the people who intentionally engage in junk science. We review the personality markers identified by a psychologist who profiled three formerly prominent scientists whose bad actions led to their respective falls from grace. |
| 10 August 2016 | Members of the Club | When their assertions are challenged, many pseudoscientists attempt to dismiss critiques by attacking the qualifications of their critics, where they claim that only those who share their rare academic pedigree are competent to challenge them. This example was ripped from the headlines of a current controversy in the field of neurology. |
| 12 August 2016 | A New Junk Science Category | When called out for engaging in scientific misconduct, a common tactic that pseudoscientists use is to "personalize the issue", where they engage in defamatory attacks to demean and diminish their critics to deflect attention away from their bad actions. We wonder if we should add a new category to our checklist to account for this specific activity. |
| 17 August 2016 | An Anti-Junk Science Checklist | We've often made use of our checklist for how to detect junk science, but in this example, we borrow a checklist of behaviors that contribute to the generation of sound results from mathematicians! |
| 26 August 2016 | Playing with Percentages | Percentages are awfully common in discussions of finance, and in the hands of those seeking to deceive, they can be super misleading. |
| 1 September 2016 | How to Identify and Deal with Toxic People at Work | The people who choose to engage in pseudo science are different from regular people, where they might often be described as having highly toxic personalities. We discuss how to deal with such toxic people if they turn out to be among your co-workers. |
| 6 September 2016 | Ignoring Inconsistencies | When confronted with contradictory data that demolishes their preferred outcomes, pseudo scientists often respond by ignoring it altogether, sometimes in ways that take on comical proportions as in this case of a junk science repeat offender. Be sure to follow the links in this post! |
| 22 September 2016 | Goals of Guile | Why would a charitable nonprofit choose to produce false statistics to misrepresent how many families are in poverty? Chalk this example up to "motive"…. |
| 30 September 2016 | The Semantics of Science Versus Pseudoscience | A discussion of whether "pseudoscience" is the right work to use to describe the shoddy practices and fraudulent activities that characterize junk science. |
| 7 October 2016 | Antitrustworthy Analysis | David Gelfand, who worked as an anti-trust litigator at the U.S. Department of Justice, tells the story of some of the shenanigans that firms seeking to merge tried to pull with their regression analyses. There's a lot of lessons to be learned here, especially for budding analysts and economists, on how their work is valued and perceived in the real world. |
| 21 October 2016 | Taxing Treats | In several U.S. cities, voters had the choice of whether or not to impose new taxes upon sugary soft drinks in 2016, but the decision to only single out soft drinks for such taxes by those proposing the taxes was anything but sound science. |
| 3 November 2016 | Models of Mathiness | In September 2016, NYU economist and incoming World Bank Chief Economist Paul Romer upset a lot of apple carts in the economics profession by accusing them of cooking their analyses through the models they use to back their desired outcomes. |
| 11 November 2016 | The Political Polls of 2016 | The political polls of 2016 were almost anything but scientific, but one particular example separated itself from the pack! |
| 1 December 2016 | A Pseudoscience Whistleblower Story | Tyler Schultz was a junior technician at Theranos, who blew the whistle on the company in the media after he discovered it was engaged in junk science. This is the story of how the company's leaders retaliated against him. |
| 20 December 2016 | Theoclassical Economists | We check in on the reaction of the three Nobel-prize winning economists who were called out very publicly for engaging in pseudoscience by Paul Romer. |
But wait, there's more!
All of the following posts preceded the Examples of Junk Science series, but could just as easily have appeared as part of it, or were subsequently modified to document a portion of the junk science generated by a particularly prolific producer.
| Posts of Note and Bonus Examples | ||
|---|---|---|
| Date | Post | Description |
| 20 August 2015 | Does Your Story Stand Up to Scrutiny? | This is the post where we first conceived what would become the "Examples in Junk Science" series, which would take almost another year to launch. |
| 9 September 2015 | Mathiness, Productivity and Wages | We discuss a controversy surrounding a chart produced to support a highly politicized agenda. |
| 20 November 2015 | Digging Deeper Into GDP | This post was originally a discussion of the new state-level GDP resources that the BEA began making available in late 2015, but it later became the focus of a smear attack by one particularly obsessive pseudoscience repeat offender, who just couldn't stop playing with their pseudo science party tricks! |
| 11 December 2015 | The Cat Came Back | Did we include super creepy cyber stalking among the list of bad behaviors in which pseudo scientists engage. If not, we should have! |
| 11 March 2016 | How to Bake a Highly Deficient Cake | What happens when you leave out a key ingredient in the recipe for baking a cake? The same thing that happens when you leave out highly relevant bits of data in an economic analysis! |
| 10 June 2016 | The Greatest Anti-Scientist of All Time | Junk science often isn't recognized for its harmful potential. In this pre-series preview, we identify the scientist whose particular brand of pseudo science cost the lives of millions upon millions of people. |
| 15 June 2016 | An Unexpected Sneak Peek of a Massive Downward GDP Revision for the U.S.? | This post was originally intended as a "live" economics science project, where we would propose an eye-catching hypothesis that we would test and systematically develop toward valid conclusions over the next several weeks. It's certainly all that, but due to the vitriolic attacks launched by one particular pseudo science practicioner we called out after we had posted it, it also became the focal point for their commitment to using junk science to prop up their preferred narrative. (They failed so badly at diagnosing a simple error we made that we were compelled to post instructions for how to replicate it in the wrap up post for the project!) |
At this point, the only aspect from our checklist of how to detect junk science that we haven't yet been able to check off is the category of "Clarity". If you come across a recent example from the fields of investing, financial or economics that captures that particular deficiency, please let us know!