Question 2: Falsification!

Ok folks,
Here it is question number 2 for you to ponder over the next two weeks, posted as a text question this time as I want to wanted to include a couple of links. This one explores falsification and the presentation of scientific findings, so here we go:
Falsifiability (or refutability) is the logical possibility that an assertion can be shown false by an observation or a physical experiment. In 1934 Karl Popper published “The Logic of Scientific Discovery” where he outlined the principles of the hypothetico-deductive scientific framework and a set of methodological rules called Falsificationism. Here he presents the idea that a thesis is “unscientific” (even if it were a false thesis) if we could not get rid of it by confronting it with an observation that contradicts it.
For example “antibiotics kill all micro-organisms” is scientific (even though obviously false) as we could test this and show this to be untrue (e.g. with a single case of a viral infection). It is therefore a scientific and falsifiable statement. On the other hand “Some witchcraft does cure illness” is unfalsifiable, and unscientific, as no amount of experimentation or observation could ever disprove the premise.
This idea of conjecture and refutation is an important one in science, and moves us beyond simple positivism. In science we develop ideas and theories through the observation of phenomena (inductive processes) and then propose theories to explain them, and then test our theories through (whenever possible) deductive processes. In this way we change our terms of reference and overthrow previously established orders as we move ahead with new knowledge (e.g. the Copernican revolution that changed our understanding of the Solar system).
We should then, be careful to consider the importance of both negative and positive data in science as the importance of publishing negative clinical studies is a vital part of the scientific process. It would be easy to ignore one side and focus on the ones that support our own pet theory. Given these thoughts what do you think of this story published recently in both US and UK newspapers with different viewpoints (see http://content.nejm.org/content/vol360/issue13/index.dtl for the papers they are based on)?
Take a look  at the stories at:
http://www.washingtonpost.com/wp-dyn/content/article/2009/03/18/AR2009031801623.html?hpid=topnews
and this one:
http://www.guardian.co.uk/society/2009/mar/18/prostate-cancer-screening-study
Lastly try and discuss in pairs and post a joint response, even if you don’t reach agreement! 🙂
Cheers Bernie
Ref:
Popper, Karl (1954) The Logic of Scientific Discovery, Basic Books, New York, NY

Ok folks,

Here it is question number 2 for you to ponder over the next two weeks, posted as a text question this time as I want to wanted to include a couple of links. This one explores falsification and the presentation of scientific findings, so here we go:

Falsifiability (or refutability) is the logical possibility that an assertion can be shown false by an observation or a physical experiment. In 1934 Karl Popper published “The Logic of Scientific Discovery” where he outlined the principles of the hypothetico-deductive scientific framework and a set of methodological rules called Falsificationism. Here he presents the idea that a thesis is “unscientific” (even if it were a false thesis) if we could not get rid of it by confronting it with an observation that contradicts it.

For example “antibiotics kill all micro-organisms” is scientific (even though obviously false) as we could test this and show this to be untrue (e.g. with a single case of a viral infection). It is therefore a scientific and falsifiable statement. On the other hand “Some witchcraft does cure illness” is unfalsifiable, and unscientific, as no amount of experimentation or observation could ever disprove the premise.

This idea of conjecture and refutation is an important one in science, and moves us beyond simple positivism. In science we develop ideas and theories through the observation of phenomena (inductive processes) and then propose theories to explain them, and then test our theories through (whenever possible) deductive processes. In this way we change our terms of reference and overthrow previously established orders as we move ahead with new knowledge (e.g. the Copernican revolution that changed our understanding of the Solar system).

We should then, be careful to consider the importance of both negative and positive data in science as the importance of publishing negative clinical studies is a vital part of the scientific process. It would be easy to ignore one side and focus on the ones that support our own pet theory. Given these thoughts what do you think of this story published recently in both US and UK newspapers with different viewpoints (see http://content.nejm.org/content/vol360/issue13/index.dtl for the papers they are based on)?

Take a look  at the stories at:

http://www.washingtonpost.com/wp-dyn/content/article/2009/03/18/AR2009031801623.html?hpid=topnews

and this one:

http://www.guardian.co.uk/society/2009/mar/18/prostate-cancer-screening-study

Lastly try and discuss in pairs and post a joint response, even if you don’t reach agreement! 🙂

Cheers Bernie

Ref:

Popper, Karl (1954) The Logic of Scientific Discovery, Basic Books, New York, NY

5 thoughts on “Question 2: Falsification!

  1. Question 2 our response

    With response to the question regarding falsification and looking at the two newspaper articles we have come to the conclusion that the two articles are not falsifiable. In order to come to this decision we discussed numerous points about the two newspaper articles and how they affect the validity of the articles and the subject at hand.
    One of the first things we looked into was the wordings of a hypothesis. A hypothesis needs to have a clear statement which can be tested to prove that it is either wrong or right, creating the falsification of the hypothesis. This however we both agreed could not be taken into account with the two articles because of the titles containing the words ?could? and ?may?. These words create a hypothesis which cannot be tested because they give no clear indication as to the full belief of what is being said. As a result we both agree that if a scientific hypothesis is one that is falsifiable, that is testable and could be proven wrong with testing. However, neither newspaper article had headlines that could be tested — neither were falsifiable.
    We also looked into the slants of the newspapers and how these could affect the validity of the information given, for example the UK newspaper the guardian focused purely on the effect within the UK and only slightly recognised that the research was undertaken throughout Europe, whereas the Washington post made reference with links to both the articles it was getting it?s information from, these being the European and US research project. Furthermore we looked into how the journalist involved in writing the article could only take what they want to hear from the research articles and create a bias response within their article. This created a small agreement between us that in order to believe what is being written we would both want to look at the primary source of the article.
    We also believed that both articles do not hold enough information to justify the titles written to them for example the guardian states little of the information to how many people actually were helped whereas the Washington post has added stats taken from the articles, although the Washington post does have a larger amount of information which can be sourced directly to the two research articles however overall we both agree that they need more information and research taken into them. As a result we both agree that out of the two articles we both believe that the Washington post is the better of the two articles because of the detail placed into it and that it shows information from both of the research projects with commenting on how they have slightly conflicting results

  2. Hi folks,

    Great response 🙂 Indeed this are at first rather perplexing stories that seem to contradict each other!

    Agreed, the hypothesis was poorly constructed, but these areas would seem to represent an area where falsification was possible, and therefore valid scientific inquiry if the studies were carried out properly. I.e. a well conducted study could demonstrate that a screening protocol did improve outcomes. However, the actual issue here may have been in the reporting as you have noted.

    Ben Goldacre took issue with the reports on his Bad Science website (see our links). He noted the results are actually pretty straightforward: the study took over 160,000 men between the ages of 55 and 69 and randomly assigned them either to get PSA screening, or to be left alone. The differences were marginal. There were 20% fewer deaths in the screening group but actually what that means is 1410 men would need to be screened to prevent one death. For each death prevented, 48 people would need to be treated: and prostate cancer treatment is as you may know,has some serious side effects (e.g. impotence and incontinence).

    It also looks like the Guardian reporter deliberately ignored one whole half of the research reported. There were in fact two large studies on PSA testing published in that issue of the Journal (actually side by side), and the other one indicated that screening did not significantly reduce deaths!

    As I noted in the introduction, we need to be careful to consider the importance of both negative and positive data in science. Here the outcomes needed careful consideration. Journalists argue we need professionals to mediate and explain science, but often get it wrong so we need to make our own determination that the evidence has been properly reported, and not misrepresented! Thoughts?

    Bernie

  3. Having read both the newspaper articles we agreed that the USA article was far more informative as it had a balance of views both for and against. Allen pointed out that is a topic which has been in the Canadian media a lot recently and the information given was as he would have expected.
    The UK article however was much more biased and focused too much on the positive aspects of PSA testing and did not even mention the negative ones.
    We both support the idea that both positive and negative information should be made public so that people are given the right to make informed decisions on a given topic. Having attended a lecture on ‘Drug Trials’ Allen confirmed that drug companies only publish results which produce positive results and very rarely mention testing which produces finding which are not as they would have liked.
    The video form the NEJM which was quite interesting, the fact that the USA Newspaper focused on the American trial and the UK paper focused on the European trial made it difficult to compare the results. Differences and are more useful than similarities when carrying out scientific observations but the results could have been interpreted in completely different ways It was also interesting to see that the family doctor took the position against PSA screening and the urologist took the positive side.
    As to our topic, for something to be scientific it has to be able to be proved and disproved. Falsifiability is the belief that for any hypothesis to have meaning, it must be inherently disproved before it can be accepted as scientific proof. For pure scientists the idea of falsifiability is a useful tool for generating theories. This is because no theory is completely correct and If a falsifiable theory is tested and the results are significant, then it can become accepted as a scientific truth. We also need to remember that just because a hypothesis is deemed scientific it does not mean that it can not be disproved in the future.
    We agree falsifiability is important but more importantly, society needs a better method of distributing scientific information to the public. The public need to be given all the information and not be misled by huge companies who publish the facts they want the public to see and hide the ones that do not sell or promote their products.

    Sarah Dodds and Allen McLean

  4. A quite like this one! Here is another good example of poor science in a study that was undertaken exemplifying poor understanding of the principle of falsification!
    Benson H., Dusek J.A., Jeffery A., Sherwood J., Lam P., Betha C.F., Carpenter W., Levitsky S., Hill P., Clem D.W., Jain, M., Drumel D., Kopecky, S., Mueller, P., Marek, D., Rollins S., & Hibberd P. (2006) Study of the Therapeutic Effects of Intercessory Prayer (STEP) in cardiac bypass patients: A multicenter randomized trial of uncertainty and certainty of receiving intercessory prayer. American Heart Journal. 151(4):934-942
    This would seeem a classic example of wasted money and poor scientific process engaged in by physicians, as no ammount of experimentation could ever prove the assertion here false! Thoughts?
    Bernie

  5. In reviweing the point raised in question two, I believe that the first article in the washington post is rather different to that of article 2. There is both very positive data in article one and relatively negative data as even though ‘the test detects real cancer’ there is not much doctors can do about the cancer after it has been diagnosed.

    The first article is indeed trying to falsify the hypothesis that the PSA test will diagnose and help more men with prostate cancer. It is very negative in approach to the benefit men in the study are getting from early detection.

    Article two is written in a very different style and im not sure in what context it is trying to proove or disprove the hypothesis…it seems like more of a report on the screening programme.

    I think that very often subjective information isn?t considered as important as objective information in terms of finding evidence to support or falsify a hypothesis. Therefore, articles often contain an overflow of statistics, and when readers look at these statistics, they are not made aware of how these statistics were collected, under what circumstances where they collected, and by whom. In the case of prostate cancer screening, any studies are trying to find some objective data regarding death rates from cancer and screening. I think for many people, quality of life doesn?t mean ?living? or ?living longer?; whereas for some others, perhaps their responsibilities make them feel that living longer is what they desire in this life even if it means living with disabilities. So I think the underlying question is really whether screening improves the quality of life of cancer patients or potential cancer patients; and that is a very subjective question. I think when people are trying to find an ?answer? to very subjective questions; objective data may not be the best to use as evidence. That?s why it is important for researchers to present arguments and counterarguments, so that they?re not arguing for a specific hypothesis, but presenting a debate where the decision of what hypothesis should be is for the reader to make.

    Just as Bernie said, ? We should then, be careful to consider the importance of both negative and positive data in science as the importance of publishing negative clinical studies is a vital part of the scientific process.?

Comments are closed.