Category Archives: data analysis

Tips on data analysis

Hop over to the QualPage, where Kathryn Roulston has an informative post on strategies and tips for thinking through data analysis.

Becker reminds us that we can only “see” using the ideas that inform our thinking. Since our training as researchers shapes the kinds of ideas we can think with, it is useful to think about how our perceptions are shaped by our individual fields of studies and ways that research is usually done. Becker (1998, p. 18) writes:

… in a strong sense, there aren’t any “facts” independent of the ideas we use to describe them….Recognizing the conceptual shaping of our perceptions, it is still true that not everything our concepts would, in principle, let us see actually turns up in what we look at.

publicity for research brings fame AND scrutiny

Two recent studies have captured the attention of the media and considerable publicity: Alice Goffman’s book On the Run and Michael LaCour’s now retracted Science article. In both cases, the researchers’ work garnered substantial attention in the popular media, and while each also has important possibilities for social change the primary focus has been on how the research was done, including the ethicality of the researchers.

9780226275406Goffman’s work is unusual in academe, research that has so captured the public attention that it has led to a TED talk, a speaking tour, possible TV and movie deals, and a trade paperback reprint. Stunning success for any assistant professor. The University of Chicago Press has had a hand in the promotion of the book, marketing it as one would a trade book including an NPR interview.

Her work has won accolades within academe as well ~ Goffman won the American Sociological Association dissertation award for her work, which is described as a major contribution to the study of U.S. poverty and racial inequality. It has also been favorably reviewed within the discipline ~ here, for example. This is a potentially important study in the wake of recent police violence against Black Americans that especially captured media attention when a white cop killed Michael Brown, a young black man in Ferguson, MO.

LaCour’s study also won the attention of the popular media because of the incredible claim that people’s views about same sex marriage could be changed through a short conversation with someone who supported the position. Ground breaking findings in political science! This study appeared in a peer reviewed highly regarded journal, one that is directed to a very broad audience of scientists rather than a specific discipline.

The reasons for retracting the paper are as follows: (i) Survey incentives were misrepresented. To encourage participation in the survey, respondents were claimed to have been given cash payments to enroll, to refer family and friends, and to complete multiple surveys. In correspondence received from Michael J. LaCour‘s attorney, he confirmed that no such payments were made. (ii) The statement on sponsorship was false. In the Report, LaCour acknowledged funding from the Williams Institute, the Ford Foundation, and the Evelyn and Walter Haas Jr. Fund. Per correspondence from LaCour‘s attorney, this statement was not true. (ii) The statement on sponsorship was false. In the Report,LaCour acknowledged funding from the Williams Institute, the Ford Foundation, and the Evelyn and Walter Haas Jr. Fund. Per correspondence from LaCour‘s attorney, this statement was not true.

Retraction Watch has chronicled the events around the retraction and responses of the two authors.

Research Issues 

It is a good thing when research findings become part of the public discourse about important issues… that is a critical role for researchers and research in contemporary society. With that publicity (whether fostered by a publisher or a researcher) comes a level of scrutiny perhaps with a spoonful of envy from other researchers. In both these cases, it was other researchers who are calling Goffman and LaCour out.

So what might we learn from these two cases. Here are some initial thoughts.

The Data Record ~ Goffman destroyed her data claiming she needed to protect her research participants and LaCour destroyed his claiming that was standard protocol. Neither position is tenable nor standard practice. While protecting research participants is of critical importance (and too often researchers blithely make a promise they may not be able to keep) but Goffman should be prepared to stand behind her promise and retain her data. And, it would seem now that LaCour may never have had data to destroy, he just said he destroyed it.

Triangulation ~ Goffman was either naive or is depending on the naiveté of her readers. Many of the critiques are reasonable requests for either corroboration, explanation or clarification. One example is her claim that, “The officers told me they had come to the hospital with a shooting victim who was in custody, and as was their custom, they ran the names of the men on the visitors’ list” has been challenged on many levels, including the unlikelihood that hospitals would release this information and that no one (and several researchers have tried) has been able to find hospitals, police departments or officers who actually do this. There are several other examples of this sort.

The onus on any researcher is to ask how to provide the best explanation of their data and this requires looking at those data in different ways, often collecting more data to clarify and test veracity.

Ethics ~ There are two contexts for thinking about research ethics: the ethics of the research practice and then ethics within the research practice. Destroying data isn’t acceptable and the ethics of research practice dictate how and for how long data ought to be kept, how to best maintain confidentiality and anonymity of research participants, honesty about sponsorship, and so on. Both Goffman and LaCour made major mistakes, and one can safely assume they acted purposefully and wrongly. In other words, there is no reason to believe they didn’t know better. LaCour’s fabrications are most egregious and his response has served only to confirm his unethical research practice.

But there is also the ethics within research practice, a much murkier gray area. Researchers, especially those doing the kind of fieldwork Goffman conducted, will necessarily encounter ethical dilemmas that arise as a result of being in the field. Her decision to live in the community she studied, and to live with some of her research participants isn’t a wrong choice (for all the critics that claim this is so, they would deny years of ethnographic research??) but it does inevitably challenge the researcher to make difficult choices. Likewise the criticism that she was not like the research participants (young black men) is an old irrelevant criticism ~ as Brian Fay has argued, “you don’t have to be one to know one.”

A number of critiques of Goffman’s work center on the ethics within research practice and are good fodder for discussions about how to do research well, in ways that respect research participants, and that leave room for researchers to explore areas of social life that may involve the illegal, immoral and unconventional fabric of human existence. These discussions are healthy for research, and the issues raised reinforce the importance of educating novice researchers about ethics beyond the ethics of research practice dictated by institutional conventions, like research review boards.

Grounded Theory ~ a brief note

grounded-theory-new-4-638Few research methodologies have been so beguiling as grounded theory. Even when researchers are not working squarely within this methodology, many strategies for data analysis are liberally borrowed from the work of grounded theory methodologists. For those who claim to be doing grounded theory, the ground on which they stand has and continues to shift from the original positions proffered by Glaser & Strauss in the mid 60s. Most especially Glaser’s realist ontology has been left behind.

Certainly, Juliet Corbin pushed that process along in her 9780857029140collaborations with Anselm Strauss and more recently Kathy Charmaz has offered what she calls a constructivist grounded theory approach. On the latter, you can watch this interview with Charmaz that clarifies her view, one that I think captures the emerging contemporary stance on grounded theory.

The use of prior empirical and theoretical literature is a key idea in contemporary grounded theory… the old school notion of avoiding all prior knowledge has been largely set aside in favor of abductive reasoning that sees the researcher traveling back and forth between their empirical data and relevant literature.

Grounded theory has also taken a more narrative turn, emphasizing more literary writing and more acknowledgement of the centrality of the researcher to the analysis. Nonetheless, grounded theory remains unique in the search for a core category, that theoretical idea that integrates the various aspects of the theoretical explanation. (See, for example, this recent article The San Miguel Project.)


Metaphors: how they help us to understand social life (and maybe make positive change)

I’ve written a few posts about metaphors  including their centrality to how knowledge about and action in the social world is constructed [The Power of Metaphors] and how to use a metaphoric lens during data analysis [Making Sense of Metaphors].

People use metaphors often as a short-hand, a way to capture complex ideas and relationships; to direct attention in a particular way; and often to present a moral view. In British Columbia where I live the province is in the midst of a fairly pitched battle between the teachers union and the government (ok so I’ve already tipped my hand in terms of the metaphor I use in talking about these labor relations). A rising chorus of voices have begun to use the metaphor of labor relations as marriage, not surprisingly since both the teachers union and the government claim to have the best interests of children at heart.

The labor relations as marriage works on a number of levels and not on many others. But, it is dominant in the media, the rhetoric of the union and the government, school administrators, students, and analysts. So, it needs to be taken seriously to understand the impasse in negotiations (and indeed the now decade old acrimonious relationship between the two) and using this understanding to both think and then act differently.


Here’s a link to the article:  Does It Help to Say the BC Teachers and the Government are in a Bad Marriage?




Narrative Analysis Tools

TV Tropes is a rich wiki repository of resources described as tricks of the trade for writing fiction, but as social scientists we borrow as needed in trying to understand and explain the social world. The emphasis is on tropes, which are described as:

Tropes are devices and conventions that a writer can reasonably rely on as being present in the audience members’ minds and expectations. On the whole, tropes are not clichés. The word clichéd means “stereotyped and trite.” In other words, dull and uninteresting. We are not looking for dull and uninteresting entries. We are here to recognize tropes and play with them, not to make fun of them… Since a lot of art, especially the popular arts, do their best to reflect life, tropes are likely to show up everywhere.

Narrative analysis can be facilitated by looking at the tropes at the centre of the stories we are investigating, and in the section on narrative devices one can investigate a range of possibilities. Tropes are named (such as “always need what you gave up,” “dead end job,” “dramatic irony,” “fighting for survival,” “human shield,” “I just want to be normal,” “shapeshifting,” “with due respect”), a short explanation is provided and links to many examples are given.

Other sections focus on characters, plot, setting and so on.

The wiki is a rich resource, every changing, including such gems as the Periodic Table of Storytelling.



A classic is back ~ new edition of Miles & Huberman’s Qualitative Data Analysis

With Johnny Saldaña as a new third author to the ghosts of Matt Miles and Michael Huberman, a new edition of this classic text on qualitative data analysis is back. This 3rd edition stays true to Miles’ & Huberman’s original organization and ideas, but is significantly updated with the inclusion of more information on computer based analysis and specific approaches to qualitative research emerging over the past few decades. The new edition ends with a very good list of resources for qualitative researchers.