Unexpectedly Intriguing!
August 19, 2009

Every day, people everywhere are bombarded by messages in the media that suggest that they need to drastically change how they live their lives or to take some particular action due to some sort of scientific finding. But do they really?

The short answer is "it depends." Most often, it depends upon the quality of the science or evidence supporting the action that is being promoted, and the only way to avoid serious disruption to one's daily life is to develop the ability to determine the difference between good science and bad science.

That's not necessarily easy, since bad science comes in many different forms. For instance, there's the "defective" variety, where otherwise good scientists have made an error in their methods that leads to incorrect conclusions, and which is probably the least harmful as it is likely to be quickly corrected, often by the scientists who made the error.

Then there's "deceptive" or "junk" science. Here, we find the veneer of good science, but twisted in a way to support a particular outcome. Taken to the extreme, junk science transforms into outright pseudoscience (or "false" science), where science is just used as a prop to support whatever outrageous claim is being made.

The trick then is to work out what is science and what is pseudoscience. To that end, we've adapted Steve Lower's invaluable guide for recognizing the differences between good and bad science, organizing it by distinguishing aspects and updating it while adding a number of our own observations and comments.

How to Distinguish "Good" Science from "Junk" or "Pseudo" Science
Aspect Science Pseudoscience Comments
Goals The primary goal of science is to achieve a more complete and more unified understanding of the physical world. Pseudosciences are more likely to be driven by ideological, cultural or commercial (money-making) goals. Some examples of pseudosciences include: astrology, UFOlogy, Creation Science and aspects of legitimate fields, such as climate science, nutrition, etc.
Progress Most scientific fields are the subjects of intense research which result in the continual expansion of knowledge in the discipline. Pseudoscientific fields generally evolve very little after being first established. What small amount of research and experimentation that is carried out is generally done more to justify the belief than to extend it. The search for new knowledge is the driving force behind the evolution of any scientific field. Nearly every new finding raises new questions that demand exploration. There is little evidence of this drive in the pseudosciences.
Challenges Scientists in legitimate fields of study commonly seek out counterexamples or findings that appear to be inconsistent with accepted theories. A challenge to accepted dogma in the pseudosciences is often considered a hostile act, if not heresy, which leads to bitter disputes or even schisms. Science advances by accommodating change as new information is obtained. Frequently, the person who shows that a generally accepted belief is incorrect or incomplete is more likely to be considered a hero than a heretic.
Inconsistencies Observations or data that are not consistent with current scientific understanding generate intense interest for additional study among scientists. Original observations and data are made accessible to all interested parties to support this effort. Observations of data that are not consistent with established beliefs tend to be ignored or actively suppressed. Original observations and data are often difficult to obtain from pseudoscience practitioners, and is often just anecdotal. Providing access to all available data allows others to independently reproduce and confirm findings. Failing to make all collected data and analysis available for independent review undermines the validity of any claimed finding. Here's a recent example of the misuse of statistics where contradictory data that would have avoided a pseudoscientific conclusion was improperly screened out, which was found after all the data was made available for independent review.
Models Using observations backed by experimental results, scientists create models that may be used to anticipate outcomes in the real world. The success of these models is continually challenged with new observations and their effectiveness in anticipating outcomes is thoroughly documented. Pseudosciences create models to anticipate real world outcomes, but place little emphasis on documenting the forecasting performance of their models, or even in making the methodology used in the models accessible to others. Have you ever noticed how pseudoscience practitioners always seem eager to announce their new predictions or findings, but never like to talk about how many of their previous predictions or findings were confirmed or found to be valid?
Falsifiability Science is a process in which each principle must be tested in the crucible of experience and remains subject to being questioned or rejected at any time. In other words, the principles of a true science are always open to challenge and can logically be shown to be false if not backed by observation and experience. The major principals and tenets of a pseudoscience cannot be tested or challenged in a similar manner and are therefore unlikely to ever be altered or shown to be wrong. Pseudoscience enthusiasts incorrectly take the logical impossibility of disproving a pseudoscientific principle as evidence of its validity. By the same token, that scientific findings may be challenged and rejected based upon new evidence is taken by pseudoscientists as "proof" that real sciences are fundamentally flawed.
Merit Scientific ideas and concepts must stand or fall on their own merits, based on existing knowledge and evidence. These ideas and concepts may be created or challenged by anyone with a basic understanding of general scientific principles, without regard to their standing within a particular field. Pseudoscientific concepts tend to be shaped by individual egos and personalities, almost always by individuals who are not in contact with mainstream science. They often invoke authority (a famous name for example, or perhaps an impressive sounding organization) for support. Pseudoscience practicioners place an excessive amount of emphasis on credentials, associations and recognition they may have received (even for unrelated matters) to support their pronouncements. They may also may seek to dismiss or disqualify legitimate challenges to their findings because the challengers lack a certain rare pedigree, often uniquely shared by the pseudoscientists.
Clarity Scientific explanations must be stated in clear, unambiguous terms. Pseudoscientific explanations tend to be vague and ambiguous, often invoking scientific terms in dubious contexts. Phrases such as "subtle energy fields" and "sustainable development" may sound impressive, but they are essentially meaningless.
Precision If numbers are presented in support of a scientific explanation, they must be stated with the precision and accuracy required by their level of significance as determined by known measurement error in the data from which are derived, neither more nor less. Pseudoscience practitioners will often present numbers with a level of precision and accuracy that exceeds that supported by the known accuracy of real world data in order to give the appearance of greater validity for their claims. A recent example of pseudoscientific deception by precision include certain economists suggesting that "a Keynesian multiplier of 1.57" specifically applies for government stimulus spending, when a wide range of studies suggest the actual multiplier may be "anywhere from 0 to 1.5" (note the difference in the number of decimal places and potential range of values!)

Other Useful Guides to Identifying Pseudoscience

Here's a short sampling of additional questions you can ask to help determine if you're confronting a claim based on science or pseudoscience.

Documented Debunkings

Sometimes, those claims you're hearing for the first time are things that have actually been around in one form or another for a long time. Here's a short list of sites that have already done the trustworthiness legwork for you!

  • Snopes.com - The Internet's premier reference for urban legends, folklore, myths, rumors and misinformation.
  • Quackwatch - If it's medical or health-related and not for real, you'll likely find it here.
  • Climate Skeptic - Like Quackwatch, but aimed at the poorly supported aspects of global climate change science.
  • JunkScience.com - Steven Milloy's site surveying a number of highly questionable scientific claims made in today's media reporting.
  • Junkfood Science - Sandy Szwarc's blog covering ongoing issues with media reporting of nutrition-based junk science.
  • Mythbusters - did you think we'd create a list like this and forget the Mythbusters?
  • John Stossel - the media's leading questioner of questionable claims, from consumer issues through politics.

Labels:

About Political Calculations



blog advertising
is good for you

Welcome to the blogosphere's toolchest! Here, unlike other blogs dedicated to analyzing current events, we create easy-to-use, simple tools to do the math related to them so you can get in on the action too! If you would like to learn more about these tools, or if you would like to contribute ideas to develop for this blog, please e-mail us at:

ironman at politicalcalculations.com

Thanks in advance!

Recent Posts

Applications

This year, we'll be experimenting with a number of apps to bring more of a current events focus to Political Calculations - we're test driving the app(s) below!

Most Popular Posts
Quick Index

Site Data

This site is primarily powered by:

This page is powered by Blogger. Isn't yours?

Visitors since December 6, 2004:

CSS Validation

Valid CSS!

RSS Site Feed

AddThis Feed Button

JavaScript

The tools on this site are built using JavaScript. If you would like to learn more, one of the best free resources on the web is available at W3Schools.com.

Other Cool Resources

Blog Roll

Market Links
Charities We Support
Recommended Reading
Recommended Viewing
Recently Shopped

Seeking Alpha Certified

Archives
Legal Disclaimer

Materials on this website are published by Political Calculations to provide visitors with free information and insights regarding the incentives created by the laws and policies described. However, this website is not designed for the purpose of providing legal, medical or financial advice to individuals. Visitors should not rely upon information on this website as a substitute for personal legal, medical or financial advice. While we make every effort to provide accurate website information, laws can change and inaccuracies happen despite our best efforts. If you have an individual problem, you should seek advice from a licensed professional in your state, i.e., by a competent authority with specialized knowledge who can apply it to the particular circumstances of your case.