A Tale of Two Cultures: Why integrative medicine is fatally flawed.

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, as Dickens wrote. I thought of those lines when I recently found myself a guest speaker at an Integrative Medicine Conference in Shanghai China.

How on earth did a skeptic like me end up there you may well ask? Well, it’s a long story, but in a nutshell I got an email invite as a guest speaker through a Chinese colleague in IT who asked me at short notice if I was interested in speaking a forthcoming medical conference in Shanghai, all expenses paid. The idea was to present some of our exploratory work on virtual reality (VR) as an adjunctive distraction therapy in pain management. I thought why not, as I happened to be free, it was only for a few days and I had never visited China before. So, a week before I was set to go I finally got sent the official conference literature in Chinese. Using Google Translate I discovered the conference was actually titled the “2016 Design and Implementation of Clinical Trials in Integrative Medicine Conference.” I was somewhat surprised at this to say the least, and wondered if these folks had ever looked at my professional profile or read anything I have written (including this blog)? I could have pulled out at this stage, but thought they had graciously invited me, paid for my flights etc, and I could discuss research techniques to support evidence-based healthcare, and talk about rapid evidence assessments in the context of our work in my presentation. Also, I must admit I was somewhat curious as to what this sort of conference would look like, and am far too past-it in my career these days to worry about negative career impact from attending academically dodgy conferences. So, despite my misgivings, I decided to go for it, and vowed to do my best to give it a fair shake, so to speak.

My hosts were very hospitable, and a day after I arrived I found myself sitting in the front row of the Lecture Room in Shuhang Hospital, Shanghai University of Traditional Chinese Medicine (TCM) ready to deliver my talk. I got a tour of some of the hospital beforehand. I asked several nurses, physicians and other staff how they used TCM here? I was surprised to find they all looked a little uncomfortable discussing it, and several noted that actually all of the hospitals offer science-based medicine with the usual pharmaceutical and surgical treatments found in the west as the primary health care interventions.TCM was used as complementary therapeutics. A few also openly admitted that TCM (which is based on an eclectic collection of a lot of vastly different philosophies and therapeutics, from demonology to acupuncture) was not highly regarded here and was seen as more of a politically sensitive response to health problems by most health professionals.

To those unfamiliar with TCM, although aspects of it have been around for at least 2,000 years or so, it was actually devised by Chairman Mao (see David Gorski’s entertaining summary here), and continues to be promoted by the government. So any critique of it within China is, to say the least, a rather sensitive issue. That is not to say it may have its benefits, as in many rural areas there is no medical care apart from TCM practitioners, so it probably is better than nothing. And, that’s the point really. Whilst much of TCM  appears to have no clinical efficacy, and some is definitely harmful, some of the remedies do actually work, but generally no where near as well as other modern scientifically validated therapies.

The problem I have always had with integrative medicine is that if you throw magical and empirical therapies together with some misplaced notion of “inclusivity,” you end up with  a melange of vacuous nonsense. The scientific medicine that actually works is devalued by being placed on the same platform as magic, whilst the magical treatments supported by pseudoscience and bad science are elevated to academic respectability (but still don’t actually work). Unfortunately, my experience in Shanghai did nothing to divest me of this view.

Before my presentation there was a rather excellent one on the use of electroacupuncture for vagal nerve stimulation by Dr. Luis Ulloa, and its potential to improve outcomes in the treatment of sepsis by reducing the inflammatory response. It was a highly creditable and very well delivered conference presentation of sound scientific work. I was very impressed with the creativity, scholarship and level of work involved; overall typical good science at work. However, to be clear this work’s relationship to TCM and acupuncture was purely in name only. It actually involved the electrical stimulation of the vagus nerve in animal studies through a needle inserted in an identified acupuncture point that corresponded physiologically with the location of the vagus nerve. It used established scientific knowledge of anatomy and neurophysiology, and at no point ever used any reference to meridians to manipulate the flow of life energy (qi) or any other TCM explanations of acupuncture. At this point my hopes were raised. Was this what the future of integrative medicine was to be, replacing mythology and faith-based explanations with sound empirical work? Maybe my misgivings had been premature.

But alas it was not to be, and things went downhill very rapidly from then on. The other presentations stretched credibility beyond belief, and demonstrated surgical ethics that were highly suspect at best. Firstly, the Director spoke proudly of how an open heart surgery operation in 1975 had been performed using acupuncture for anaesthesia, and then a speaker who was one of the  original anesthesiologists in that surgery came on stage to say how they pioneered these techniques. These were even commemorated on this Chinese stamp:

stamp

The hapless patient here was a 15 year old girl, and the reality was that it was later reported, that in addition to acupuncture the patient  also had powerful sedatives and analgesics (midazolam, fentanyl, and droperidol) and also large volumes of local anaesthetic injected into the chest. They also had to be taught abdominal breathing exercises for days before the procedure to maintain ventilation (but they did maintain an endotracheal tube for emergency use – thank goodness).

This added to the mythology of the effectiveness of acupuncture as anesthesia in the west, arising from a 1972 story when journalist James Reston wrote about his emergency appendectomy during American President Nixon’s trip to China. He was initially reportedly as “anaesthetized” by acupuncture needles. But this soon became established as a PR stunt, as again, it turned out he actually had conventional anaesthesia, and acupuncture afterwards to help pain control in the recovery period. if you ever want a patient’s first hand account of the benefits of acupuncture anesthesia I highly recommend professional ballet dancer Li Cunxin‘s autobiography. He is not a fan, having experienced it as a young man.

The presentation went on to outline how they now regularly employed acupuncture anesthesia for open heart surgery as it was shown to save costs considerably both in terms of the costs of anaesthetics and recovery. They also cited a 2011 paper that claimed “during operation, patients were kept on spontaneous breathing.” I took the following picture during the presentation which had some rather gruesome photos of patients they claimed to be undergoing open heart surgery using acupuncture anesthesia.

img_5422

The trouble with all this is it includes some science mixed with half-truths and a large dose of pure propaganda. Firstly, I have no doubt it is possible to do thoracic surgery with minimal conventional anaesthesia and adjunctive pain control measures,but that doesn’t mean it’s a good idea. Scientifically we know pain is multifactor complex neurophysiological and psychosocial phenomenon. Placebo and distractive effects can be very powerful here. In all likelihood a range of alternative placebos together with pharmaceuticals would probably have had similar effects to those claimed for acupuncture here. However, experimenting with these during major surgical procedures is completely unethical by any modern medical standards (and I would suggest borders on medical torture). Some may deconstruct this as an old-colonialist view of a very different culture, but to suggest it is reasonable to use low doses of established  anesthetic drugs with acupuncture and keep the patient conscious to save money is simply morally indefensible, and hardly patient-centred medicine. As a nurse I would have to say that anyone who has witnessed open chest surgery, and thinks it is good for the patient to be conscious during the procedure is either a sadist or has no respect for human suffering.  I’ll leave it to your imagination to think what it would sound and feel like as the surgeon saws through your sternum. Certainly, there is always the: “but what if there is no alternative” argument but the problem here is – there are good alternatives (and that argument really does represent an old-colonialist viewpoint). There are many better proven anesthetic and pain control measures that could be used than this, and this falls far short of the “do no harm” principle.

Secondly, a cursory review of the actual work reveals the whole thing is a politically motivated sham anyhow. Even one of the Chinese doctors I spoke with there thought the whole thing was politically driven nonsense. By mixing ancient chinese magical theories of qi energy, meridians and needles with doses of modern sedatives, local anesthetics and narcotic analgesics it is impossible to determine what effect the actual acupuncture is having compared to the drugs, let alone any placebo effect. Use of sham acupuncture was cited as being inferior in some of the work presented, and the presentations on acupuncture anesthesia all contained copious statistical reports with p values supporting positive outcomes. No one could claim the researchers were not statistically competent. In fact, I have never seen so many P values and positive results presented in a single medical presentation. Nevertheless, apart from the well established problems with P values, statistical inference relies on robust methods and hypotheses to be meaningful. It’s a case of garbage in, garbage out, and I have never seen such demonstrations of overt confirmation bias as I experienced here. If the findings claimed here were actually true and replicable, I guarantee anaesthetists all over the world would be throwing out their Boyle anaesthesia machines and embracing acupuncture as the technique of choice.

Unfortunately, evidence for the basic theory and science of acupuncture in TCM just isn’t there, and by adding multiple confounding variables in studies any possible effect is completely obfuscated. In our own initial work with VR and pain, we are at an exploratory phase and although it looks promising as a distractive therapy, the reasons why it seems to work are unclear. Also, we have found the effect size varies considerably with individuals, and we are far from identifying if it will turn out to be a useful therapeutic adjunctive. This is for a new technique, and yet the results of the first ever well-controlled study of acupuncture (claimed to be over 2000 years old) for back pain were only published in 2007 in the Archives of Internal Medicine. This study found:

  1. Both acupuncture and sham acupuncture appeared to work better than conventional drugs, physical therapy, and exercise alone.
  2.  However, acupuncture worked no better than sham acupuncture.

So, clearly there was some effect but most likely placebo. It well established that pain perception is hugely impacted by psychological factors, but magical theories of qi and meridians have never been independently empirically demonstrated. Following further studies even NICE in the UK has now rejected the use of acupuncture for low back pain.

So overall, the whole conference experience was very dispiriting. Mixing good scientific work with myth-based bad science and pseudoscience is going to get us nowhere fast. From my perspective the experience was akin to me going to an aviation conference and watching a presentation on the latest developments in carbon-fibre technologies to create improved aerofoil designs, followed by one explaining how to weave charmed threads into a magic carpet to gain maximum levitation. At the end of the day only one of these approaches is ever going to get off the ground. Sadly, the whole integrative science/medicine enterprise is academically bankrupt, and the sooner universities here promoting it realize this, the better.

Oh, and how did my presentation go down? Well, it was politely received (probably didn’t state enough P values – even with my small samples – for this audience).  I did find one of the audience kept eying me with a death-ray glare throughout though. So I suspect at least one person had actually looked up my background.

Season’s Greetings to all

Onwards and upwards.

Bernie

 

 

 

 

 

 

 

 

 

Sampling and Probability: probably…

Hello all,

A belated Happy New Year to all (note the logical form: year is NOT plural, except apparently in North America)!

We thought we would kick off the year with a quick discussion on sampling theory as it seems a subject fraught with confusion. To illustrate this point I note a section from the Statistics Canada website which was cited to me last year by a graduate student (postgrad for readers in Blighty). The Stats Canada site notes the following about non-probability sampling:

Non-probability Stats Canada clip

Now I certainly don’t claim to be a statistical expert as my expertize with inferential statistics is fairly limited. But I have a bit of a logic background from programming, so I do know a little about logical clauses. The problematic part for me is:

“in non-probability sampling, there is an assumption that there is an even distribution of characteristics within the population. This is what makes the researcher believe that any sample would be representative and because of that, results will be accurate.” 

Something about that didn’t seem quite right, and it seems inconsistent with the later statement:

in non-probability sampling, since elements are chosen arbitrarily, there is no way to estimate the probability of any one element being included in the sample.

Logically, if it is a non-probability sample, then the sample will not be representative of the probability of a phenomenon being present in a population. If a phenomenon is equally evident in all members of the population then it is a probability sample as the sample is subject to probabilistic inference. In the case of everyone demonstrating the phenomenon the probability of finding it in your sample would be 100%. In effect, if the first statement is true, then the second cannot be as they are mutually exclusive. I believe what they are trying to suggest, is that a non-probability sample is a targeted sample, selected to include from a frame (the set of people from whom the sample are drawn) who all exhibit the same characteristic, or have experiences the same phenomenon. Technically, this is not the same as an “even-distribution” though.

The way I was taught, and understand the difference between probability, and non-probability samples is as follows (and is also consistent with the second clause).

Non-Probability Sampling

Non-probability sampling does not depend upon the rationale of probability theory, and with it there is no way to estimate the likelihood for any particular element being included within a sample. Researchers may use this approach when a representative sample is unnecessary (such as to explore the existence of a phenomena or determine personal experience), or when a probability sample is unavailable.  Even with samples that are not representative of a population we can still explore the elements to describe phenomena or identify if a particular phenomenon exits.

Non-probability sampling may be useful in qualitative work, or for practicality such as in focus group selection. Non-probability sampling is also useful if there is a limited population size, as with very small frames the key statistical properties required to support a probability sample do not exist. E.g., surveying 20 users of a new tool in a specialty clinic. It may also be a useful technique where the frame parameters are uncertain. E.g., sampling street drug users. Techniques for non-probability sampling are summarized as follows:

Non Probability Sampling

Probability Sampling

Probability sampling is more commonly used in quantitative research and aims to use representative samples of a whole. It is based on probability theory and accepted statistical principles to allow the prediction that findings observed in the sample will occur in the whole population.  It requires that every element has the chance of being selected, ideally (but not necessarily) an equal chance. In this type of sampling the probability of selection of an element can be calculated, so a sample element can be weighted as necessary to give it unbiased representation. It also requires that random chance determine selection. In the case of random samples, mathematical theory is available to assess the sampling error. Thus, estimates obtained from random samples can be accompanied by measures of the uncertainty associated with the estimate e.g., standard error or confidence intervals. Examples of probability sampling techniques are summarized as follows:

Probability Sampling

This seems consistent with the literature I have looked at on the subject over the years (such as Lenth, 2001; Campbell, Machin & Walters, 2007; Polit & Beck, 2014). The advantages and disadvantages of both approaches can be summarized as:

Sampling Advantages and Disadvantages

I did write to Stats Canada, asking for an clarification, and even politely suggested a possible correction that would make their description consistent. However, I never heard back, so I guess they don’t have time to answer the blathering of an inquisitive nursing professor.

I then asked a couple of stats savvy colleagues if they could explain the apparent inconsistency. One said, “Err, that doesn’t seem right to me” and another “Well, if Stats Canada say so it must be right!” Therefore I am am none the wiser to their rationale. All I can say is from the good science perspective: never take for granted anything you read (from whatever source, and well, especially on the web)!

If any stats wizards ever read this blog please do pitch in and give us your thoughts.

Onwards and Upwards

Bernie

References

Campbell M.J., Machin D. & Walters S., (2013) Medical Statistics: A Textbook for the Health Sciences. Chichester, John Wiley.

Lenth, R.V. (2001). Some practical guidelines for effective sample size determination. The American Statistician, 55, 187-193.

Polit B.F. & Beck C.T. (2014) Essentials of Nursing Research: Appraising Evidence for Nursing Practice. New York. Wolters Kluwer

 

Bernie’s Philosophy of Science eBook is Alive!

Apologies for the lack of activity on the site for the last few weeks, but I had been putting the final touches to my ebook and Roger is off on his travels. Anyhow, the ebook is now published and so in a bit of unabashed and shameless self-promotion here are the details (cue the fanfare)…

So what is it about, you may well ask? In the book I present a pragmatic approach to the philosophy of health science (particularly nursing) that underpins evidence-based practice (EBP), including an exploration of research methodologies. It covers the history of scientific inquiry and foundational principles underpinning science as applied in the field of professional healthcare. I have used some provocative arguments to challenge some of the constructivist and postmodern approaches to nursing knowledge. There is a chapter exploring some key examples of non-science, pseudoscience, and simply bad-science that continues to pervade modern healthcare research. Basically, if ever you want some good arguments to challenge postmodern or relativist philosophical positions in healthcare, this book should provide you with plenty. It also includes an updated version of our good science detection kit, and an excellent foreword by Roger (although I fear endless claims of “it must be your round” when I am next in the UK as a result of that).

The book is available from a range of ebook sellers and I will update the list as they put it out ( see details on ebook readers at the end of the post) . You can search for it by my name, the title, or the ISBN: 9780991984602.

Sources 

It has been an interesting journey on the road to publication, and the book should be found at around $9 (which was the price I agreed; although, strangely some vendors seem to have added a few dollars, so shop around). I have had some excellent reviews and some dissent with the ideas presented  (but that is always the way with controversial issues). Anyhow, I hope people will keep an open mind while they read it and enjoy the book, finding something of interest. Do let me know your thoughts on the book in the blog. I welcome all constructive criticism, ideas, commentary and feedback. I am still finding the occasional typo now and again too, so apologies, I will amend, but hey it’s only $6!

A publicity flyer PDF for the book is available here.

If anyone has any trouble finding a copy, or format issues (I can supply any ebook format), please e-mail me; likewise any educators who would like an inspection copy.

Regards

Bernie

 P.S. Additional eBook Reader Information

The ebook was developed for the Apple iPad, but I have been testing various free ebook readers with the book and interestingly found some differing results. Despite adhering to all of the latest publishing standards for ebooks it seems full standardization is some way off in the electronic publishing world. I thought the following might be of use (click on the name of the application to download it):

  • Apple iBooks: displays perfectly – available via iTunes – best choice for iPad readers
  • Kindle Reader: displays fine, but loss of coloured fonts, lines and formatting of titles, in most versions. Readers may need to resize text in their Kindle to get optimum page layout.
  • Sony Reader: Displays fine- but the application is a bit slow to run on some PCs .
  • Kobo: displays perfectly The reader is available here: http://www.kobo.com/desktop
  • Lucidor: the book displays perfectly but some Windows users have experienced installation issues with this reader (it also needs Firefox to be installed first)
  • Adobe Digital Editions: Displays fine, but it seems the font size is not resizable in the application, which can mean fine sized print on some hi-resolution monitors
  • Calibre: a fine free ebook reader, and the book displays perfectly in it
  • ePubReader: displays Ok but format display issues with chapter titles (does not follow font size coding). This application also needs Firefox installed and works through it.
  • Aldiko EReader: Not tested on this Android platform but uses epub standards so should be OK. Let me know if anyone has tested it on this reader.

I hope this is helpful.

Bernie