Carl Sagan's Baloney Detection Kit

CARL SAGAN'S BALONEY DETECTION KIT

Based on the book The Demon Haunted World by Carl Sagan

2 Oct 11: Carl Sagan's books, including this one, are now available as ebooks from Kindle - but only for residents of the USA :(

The following are suggested as tools for testing arguments and detecting fallacious or fraudulent arguments:

Wherever possible there must be independent confirmation of the facts
Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
Arguments from authority carry little weight (in science there are no "authorities").
Spin more than one hypothesis - don't simply run with the first idea that caught your fancy.
Try not to get overly attached to a hypothesis just because it's yours.
Quantify, wherever possible.
If there is a chain of argument every link in the chain must work.
"Occam's razor" - if there are two hypothesis that explain the data equally well choose the simpler.
Ask whether the hypothesis can, at least in principle, be falsified (shown to be false by some unambiguous test). In other words, is isttestable? Can others duplicate the experiment and get the same result?

Additional issues are

Conduct control experiments - especially "double blind" experiments where the person taking measurements is not aware of the test and control subjects.
Check for confounding factors - separate the variables.

Common fallacies of logic and rhetoric

Ad hominem - attacking the arguer and not the argument.
Argument from "authority".
Argument from adverse consequences (putting pressure on the decision maker by pointing out dire consequences of an "unfavourable" decision).
Appeal to ignorance (absence of evidence is not evidence of absence).
Special pleading (typically referring to god's will).
Begging the question (assuming an answer in the way the question is phrased).
Observational selection (counting the hits and forgetting the misses).
Statistics of small numbers (such as drawing conclusions from inadequate sample sizes).
Misunderstanding the nature of statistics (President Eisenhower expressing astonishment and alarm on discovering that fully half of all Americans have below average intelligence!)
Inconsistency (e.g. military expenditures based on worst case scenarios but scientific projections on environmental dangers thriftily ignored because they are not "proved").
Non sequitur - "it does not follow" - the logic falls down.
Post hoc, ergo propter hoc - "it happened after so it was caused by" - confusion of cause and effect.
Meaningless question ("what happens when an irresistible force meets an immovable object?).
Excluded middle - considering only the two extremes in a range of possibilities (making the "other side" look worse than it really is).
Short-term v. long-term - a subset of excluded middle ("why pursue fundamental science when we have so huge a budget deficit?").
Slippery slope - a subset of excluded middle - unwarranted extrapolation of the effects (give an inch and they will take a mile).
Confusion of correlation and causation.
Straw man - caricaturing (or stereotyping) a position to make it easier to attack..
Suppressed evidence or half-truths.
Weasel words - for example, use of euphemisms for war such as "police action" to get around limitations on Presidential powers. "An important art of politicians is to find new names for institutions which under old names have become odious to the public"

Above all - read the book!

The Critical Thinking Community
CSICOP/Skeptical Inquirer
Australian Skeptics
Quackwatch
The Annals of Improbable Research - with the Ignobel Awards and the AIR Teachers Guide.
Carl Sagan Productions Ann Druyan's comment on this web page:

12 Nov 2000 Project Voyager: OneCosmos represents the realization of a shared lifelong hope to organize and lead a team that will deliver the vision of Cosmos through every available screen: spectacular living Internet, engaging television and cinematic works of art.
Examining the role of think tanks by Sharon Beder, Engineers Australia, November 1999.
Innumeracy.com.
Faith-Based Reasoning - Scientific American June 2001. In one case [global warming], the president invokes uncertainty; in the other [missile defence], he ignores it. In both, he has come down against the scientific consensus.
InConcept.
Scientific American, Nov 01: Baloney Detection: How to draw boundaries between science and pseudoscience, Part I. Part II.(updated URLs)
Scientific American May 02: The Exquisite Balance - It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas.... If you are only skeptical, then no new ideas make it through to you.... On the other hand, if you are open to the point of gullibility and have not an ounce of skeptical sense in you, then you cannot distinguish the useful ideas from the worthless ones - Carl Sagan, 1987.
Skeptic.com
The Skeptic's Dictionary by Robert T. Carroll.
Cable Science Network - new TV service dedicated to science.
21 Feb 06 New Scientist: Algorithm detects Canadian politicians' spin - Spin, in this case, is defined as “text or speech where the apparent meaning is not the true belief of the person saying or writing it”...
9 Mar 06 National Geographic: Was Darwin Wrong?
11 May 05 Karl Kruszelnicki: Mysterious Killer Chemical - We live under the illusion that we understand the world around us...Dihydrogen Monoxide FAQ - a common household compound can be hazardous
27 May 06 SciAm: Up the Lazy Creek - "motivational deficiency disorder"...numerous news outlets picked up the BMJ press release and ran it without a hint of skepticism. That's just motivationally deficient journalism. BMJ abstract: Scientists find new disease: motivational deficiency disorder + People are easily duped about new diseases, conference is told.
27 Jun 06: By coincidence, I received the 24 June copy of New Scientist and July copy of Scientific American on the same day. In New Scientist Richard Koch and Chris Smith ask "why is science under attack like never before?". They suggest that the rest of society is now much more critical of science, which has revealed a darker side such as atomic weapons and "poisoning of the planet". There is another, more likely reason for the demise of science that is revealed in the article "The Political Brain" by Michael Shermer in Scientific American. He describes MRI studies of the brain that have revealed how the brain suppresses the rational, reasoning portion of the brain in favour of emotions that reinforce confirmation bias - "whereby we seek and find confirmatory evidence in support of already existing beliefs and ignore or reinterpret disconfirmatory evidence".
Science, of course, is built on that rational, reasoning function of the brain and is founded on skepticism. This does not bode well for politicians, religious fanatics or marketers of consumers products who utilise emotional responses to ply their trade. Is it any wonder that that science and skepticism are discriminated against when these same groups now have a huge influence on the media?
6 Oct 06 New Scientist (subs): Mind fiction: Why your brain tells tall tales (see 27 June item above)
Ig Nobel prizes for 2006!
9 May 07: Perpetual Motion and the Big Wither.
25 Feb 08 New Scientist ($): Interview: The man who would prove all studies wrong - "People aren't willing to abandon their hypothesis. If you spend 20 years on a specific line of thought and suddenly your universe collapses, it is very difficult to change jobs." + Comment: Why peer review thwarts innovation
25 Feb 08 SciAm: Adam's Maxim and Spinoza's Conjecture - "...we should reward skepticism and disbelief and champion those willing to change their mind in the teeth of new evidence. "
Jun 08 Kids.net.au: Scientific method.
21 Feb 10 New Scientist: CERN on trial: could a lawsuit shut the LHC down? - Social scientists have identified a number of phenomena that can skew attempts to reach objective assessments of risk. For instance, cognitive dissonance describes the tendency of people to seek information that is consistent with their beliefs and to avoid information that is inconsistent. "Groupthink" describes a process by which intelligent individuals, working in a group, can reach a worry-free outlook that is not justified by the facts. And the phenomenon of confirmation bias - the tendency to filter information so as to confirm working hypotheses - was cited by the Columbia Accident Investigation Board as one explanation for why space shuttle programme managers ignored sure signs of trouble.
14 May 10 New Scientist: Special Report - Living in Denial
4 Dec 10 SciAm: The Conspiracy Theory Detector - How to tell the difference between true and false conspiracy theories + Michael Shermer’s video on Baloney Detection.
27 May 11: This page now copied at the official Carl Sagan website - thank you
27 Aug 11 SciAm: What Is Pseudoscience?
2 Oct 2011: Carl Sagan's books are now available from Kindle - but only for residents of the USA :(
20 Dec 12 SciAm: The Mind’s Compartments Create Conflicting Beliefs
25 Jun 13 SMH: Why superfoods are not so super after all
9 Sep 14 Atheist TV - provide a counter-balance to the myriad of religious (and psuedo-science) programming available on television.
11 Sep 14 American Atheists: AN INTERVIEW WITH DOUGLAS ADAMS + Recovering From Religion.
13 Jan 16 Lawers Weekly: Research reveals the problem with ‘overwhelming evidence’ - A new study has suggested that total agreement between witnesses or overwhelming evidence in a court case should raise suspicions of bias and lower confidence in a result.
25 Jan 17 ScienceAlert: The internet is freaking out over this spooky prediction by Carl Sagan about the future.
23 May 17 SciAm: Revamped "Anti-Science" Education Bills in U.S. Find Success.
22 Nov 17 Sapiens Hub: “In questions of science, the authority of a thousand is not worth the humble reasoning of a single individual.”
― Galileo Galilei” Excerpt From Science: A Collection Of Quotes
11 Mar 19 Scientific American March 2019: Why we believe conspiracy theories - has a useful section "Telling fact from fiction" that is similar to the above kit (copyright SA): ...ask three key questions when interpreting conspiracy claims. One: What is your evidence? Two: What is your source for that evidence? Three: What is the reasoning that links your evidence back to the claim? Sources of evidence need to be accurate, credible and relevant...False conspiracy theories have several hallmarks,..First, the theories include contradictions. For example, some deniers of climate change argue that there is no scientific consensus on the issue while framing themselves as heroes pushing back against established consensus. Both cannot be true. A second telltale sign is when a contention is based on shaky assumptions. A third sign that a claim is a far-fetched theory, rather than an actual conspiracy, is that those who support it interpret evidence against their theory as evidence for it.
2 Aug 19 The Guardian: Revealed: Johnson ally’s firm secretly ran Facebook propaganda network. - "...campaigns in support of coal power, tobacco, and against cyclists."
4 Sep 19 ScienceAlert: Unnerving Chinese Deepfake App Lets You Replace Celebrity Faces With Your Own - likely to be used for creating fake videos to stir up trouble. Update: Japan Today: Facebook, Microsoft launch contest to detect deepfake videos.
28 Oct 19 New Scientist: Deepfakes are being used to dub adverts into different languages.
15 Nov 19 New Scientist: Deepfakes are terrible for democracy, but Facebook is a bigger threat.
1 Apr 20 The Conversation: How not to fall for coronavirus BS: avoid the 7 deadly sins of thought.

Anti-Quackery Ring
[ Join Now \| Ring Hub \| Random \| << Prev \| Next >> ]

Less serious sites:

Journal of Irreproducible Results
Dihydrogen Monoxide FAQ - a common household compound can be hazardous

Created by Michael Paine in January 1998.