Hi folks,
There is a video of me outlining Question 3 on the beach here in Vancouver on our Facebook page:
http://www.facebook.com/video/video.php?v=501850728157
Take a look and here are the key points in text for your consideration over the next two weeks:
Good science and experiment design reduces error, but we can never get rid of it completely and some error always remains, so we have to accept and deal with it (hence replication and the use of inferential statistices etc.). We therefore adopt strategies that we know will reduce errors as much as possible.
Verification error is an important factor to consider here. This is also known as “confirmation bias” in the behavioral sciences ie. the process of trying to fit results to match preconceived ideas.
In psychology, social sciences, and human health sciences, results are naturally open to personal interpretation, especially for emotive issues, such as ethics, cultural perspectives or racism. A researcher must incorporate mechanisms to reduce the chance of confirmation bias, or risk losing validity and credibility.
You will most likely be aware of the concepts of validity and reliability:
VALIDITY
Validity establishes whether the results obtained meet all of the requirements of the scientific research method.
Internal Validity
Internal Validity is a measure of the accuracy of the scientific research in terms of the degree to which changes in the dependent variable can be attributed to manipulations of the independent variable. Here an experiment is said to possess internal validity if it properly demonstrates a causal relation between the variables. An experiment can demonstrate a causal relation by satisfying three criteria:
1) the “cause” precedes the “effect” in time (temporal precedence),
2) the “cause” and the “effect” are related (covariation), and
3) there are no alternative plausible explanations for the observed covariation (nonspuriousness)
A fun example is the Pirates and Global warming corrolation as depicted in this rather dubious graphical representation (from Venganza.org ) apart from the very dubious cause and effect link, what else is wrong with the graph?!:
External Validity
External validity is the process of examining the results and questioning whether there are any other possible causal relationships. This is most difficult to achieve and asks the question of generalizability. I.e. To what populations, settings , treatment variables and measurement variables can an effect be generalized (Campbell & Stanley 1966)
RELIABILITY
Reliability is of course another aspect of minimizing errors. Here we are concerned that The results must be more than a one-off finding and repeatable. We must make sure the measurement tools we are using acurately measure what we are asking them to. Experiments are more difficult to repeat and are inherently less reliable.
Modern nursing research often focuses on qualitiative, humanistic, phenomenological approaches exploring subjective elements of phemonema, and the lived experience of individuals. This is certainly an important part of abductive and inductive process in scientific inquiry (Quiroz & Merrell, 2005) and produces outcomes that are not generalizable but that could be subject to further exploration and inform hypothesis development and experiment. However, many nursing researchers reject attempting to do this as positivisitic and reductionist in approach, arguing that we don’t need to generalize or reduce errors as the objective of this type of work is understanding the human experience which is not quantifiable. The classic argument being “you can’t measure love.” An example of this would be Doane & Varcoe (2007) who use this sort of approach, proposing “relational inquiry” as an alternative methodology in nursing science:
So this weeks question is: Why bother with quantification and error reduction at all in health research? As this approach is by its very nature subjective, and humanistic and surely a scientific quantification approach here would represent reductionism. Therefore error reduction has no part in modern humanistic research and any attempt at error reduction would seem a meaningless and fruitless exercise in this context. Thoughts?
References
Campbell, D.T., Stanley, J.C. (1966). Experimental and Quasi-Experimental Designs for Research. Skokie, Il: Rand McNally.
Doane G. H. Varcoe C.(2007) Relational Practice and Nursing Obligations. Advances in Nursing Science 30(3)
Quieroz J. & Merrell F. (2005) Abduction: Between Subjectivity and Objectivity, Semiotica 153 (1/4) 1–7.
http://www.experiment-resources.com
http://www.venganza.org/about/open-letter/
http://www.experiment-resources.com/index.html
Follow
The axes on the above graph are totally wrong and super misleading.
It is definitely important to conduct humanist/qualitative researches and find out what is really going on with different experiences people have. For example, child birth is an experience that’s unique to each mother, and one can never generalize in people’s experiences. Just like the popular saying, “you’re entitled to your opinions.” If someone has a particular opinion on a certain subject, I would respect his/her opinion, but I wouldn’t believe his/her opinion and now act according to this one person’s opinions. For example, if mint chocolate has analgesic effects on Emily’s menstrual pain, it does not mean it will work for me. We are healthcare workers and the priority of nursing care is safe patient care. If one idea (phenomenon) cannot be generalized, or supported with a certain degree of quantitative data, it is not safe to be used for the general public. For example, if one study (phenomenology) finds that the use of patient-controlled analgesic greatly improves the quality of life of labouring Northern Chinese mothers, without quantitative studies, I think it’s definitely unsafe and unethical to assume that it applies to every labouring mother. I think every experiment should have a balance of quantitative and qualitative data, a purely scientific experiment may neglect the reality, and a purely qualitative experiment may not be evidence-based.
I agree a humanistic approach, where possible provides a good balance and another point of view when analysing data. However it can not be avoided that maybe the factors involved with this approach may blind the researcher in regards to highlighting quantifiable data. Statistically quantifiable data that is. Will the researcher be blinded in recognising the true result of his data because of their opinion? I feel that a careful approach has to be used here in order to decide on the best paitient care for a specific person. And I agree strongly with Jayne that it would be entirely unethical to assume every person is the same throughout the world and there care should be the same also.
The x-axis goes from bigger number to smaller number, which doesn’t make sense, and isn’t correct. if anything, it should go from zero to infinity from left to right.
From Canada: Jae-Young Kwon
I think that even though quantification has some form of subjectivity and biasing, we must still have a system that measures certain aspects of research such as how much expertise or knowledge one has in the field. For example, when we’re doing qualitative research on effective teaching strategies for professors and interviewing them, we would try to reduce the error in the findings by collecting comparable data from professors who have the expertise in their field by certain number of publications, excellent evaluations from students, and the number of years teaching. Another great example is the statement “you can’t measure love.” However, one might argue that love can be measured by certain criteria. The researcher can define love as the number of kisses the husband gives to his wife per day and the time spent together gazing at each other’s eyes or a lack thereof by the number of times he forgets their wedding anniversary. As a result, I think it depends on what the researcher wants to focus on. If the researcher thinks he/she cannot quantify human experience they are right, on the other hand, if the researcher thinks he/she can quantify it, they are also right.
From UK: Mim Wells
Although there are issues with making an investigation valid I think it is important to try. Some investigations can’t be quantified eg if they are based on emotions and this makes their validity harder to determine, but I feel for those based on data and statistics different techniques can be used to increase their validity and reliability. These investigations should be repeated under the same conditions and therefore the results would show either a consistent result or show a mix of results with little coherence and therefore show that the effects may not be from the variables tested. If an experiment can be repeated and tested against other finding I think it should even if it doesn’t give a clear result. If no investigation’s were repeated and tested many results may have been published as fact which may have been found later on to not work or have detrimental effects which would have been found if they had repeated the investigation. Therefore even though it may be difficult to make an investigation fully valid with no errors all attempts should be made to try so that the results found and published are as accurate as possible.
From Ethiopia
As for me there is one point we have to understand before coming to that conclusion. As we all see in our daily life; health in itself is a subjective definition. If not, we couldn’t get any thing health. So that we would make our research focused on illness rather. But thanks to common sense, we have the way to standardize our subjective and humanistic approach. Because most of the research findings in health are based on subjective data(from human either directly or indirectly-who express things as he/she feels) it is hard to interpret them in a way that is completely different. Therefore,we have to make our quantification clear: to what variable and how. For me the problem is not in quantification: it is in wrong generalization and that is where we want to focus – we need to be careful on.
Great comments thanks,
I would agree we can measure pretty much anything if we put our minds to it, but from a sceintific approach should alk is it meaningful to do so. In health care we are frequently forced to decide between competing therapeutic interventions, and decide which is best (or even if they work). To do that on purely humanistic and subjective grounds is problematic, and it is here that I would suggest we mostly need science.
As far as the graph goes, it shows that misrepresentation of data is one of the key problems with measuring and is often cited as a central problem with science by post-modern academics. In this case the lower scale is inconsistent. Striving for “good science” and exposing “bad science” is one way we can help challenge such arguments.
I think the point about common sense is good, particularly in selecting what we are going to study or explore (as we have finite resources). However, the human mind is easily fooled and common sense is not as simple as it appears. After all, common sense tells us the earth is flat!
Bernie
This is mine, Emily and Dori’s response to question 3!
As we all see in our daily life; health in itself is a subjective definition. But thanks to common sense, we have the way to standardize our subjective and humanistic approach.
Because most of the research findings in health are based on subjective data(from human either directly or indirectly-who express things as he/she feels) it is hard to interpret them in a way that is completely different. Therefore, we have to make our quantification clear: to what variable and how. For me the problem is not in quantification: it is wrong generalizations.
For me and most people we accepted quantifiable data as part of everyday life, it is something that we all understand to some degree and can make decisions from there.
It is therefore even more important in medical research, as health/medicine is something that affects everyone and by displaying in a format that is understandable to the majority of the population is essential. Despite the risk that both error reduction and quantification could be
tainted by subjectivity either through experimenter bias or otherwise, in many types of health research it would still be absolutely necessary to
use both. The reason is that if no attempts are made to quantify a variable, for example, to correlate decreased BP with use of Ramipril,
then we would not know if giving the medication works or not.
Not knowing would be the result of lack of quantification, in this case, would lead to
a larger question: If we don’t quantify the results of a medical treatment, why use it at all? In fact, how would we even know that we need
to develop it in the first place if we don’t quantify blood pressure?
If we never quantify anything for fear of subjectivity bias, we would see a lack of healthcare interventions being put in place, perhaps with negative consequences.
The same argument applies to error reduction. In addition, if we do not attempt to reduce the error inherent in any experiment, we could end up
with standard deviations that are high enough to render the results of a given experiment insignificant and thus we would miss the potential
effects of an experimental condition.
I think, then, that the risk that subjectivity plays is outweighed by the benefit of error reduction and quantification. I’d rather biased quantification than no quantification at all, if it meant that lives could
be improved or saved in so doing.
Tim, Emily & Dori