Category Archives: methods

A very short note on mixed methods

big abstract ideas, methodology, methods

The many guises of mixed methods…

Mixed methods are used when there are two or more types of:

➢ research questions
➢ sampling procedures
➢ data collection procedures
➢ data
➢ data analysis
➢ conclusions

It is relatively easy to mix methods at the methods level, i.e., when the intent is to collect both quantitative and qualitative data. This can be accomplished within a single method. For example, you might use observation as a method and use an inventory to record frequency of behaviours, interactions, and so on. You could also record dialogue or take field notes occurring during this same observation. But this strategy may also mean using a method that generates numbers (likert scale item survey) and a method that generates words (oral history interview).

With mixed methods often one or the other paradigm is primarily emphasized and often there is a sequential use of one then the other paradigm. This chart illustrates possible variations.

Mixed methods research wants to move beyond this simple distinction of types of data and the field makes an effort to elevate the idea to a methodology, even sometimes crossing the epistemological boundaries of objectivism and social constructivism. A research study that uses interviews and participant observation, for example, is now not necessarily considered mixed methods research. Whereas, a research study that uses hermeneutics and times series analysis would be a mixed methods study. These different contexts also suggest the “mixing” can occur at different places within the research process.

Justification for mixed methods…

A primary justification for mixed methods is pragmatism. (Read, for example, David Morgan’s argument for this justification here.) Pragmatism asserts no first or foundational principles and suggests that all human knowledge is empirical, what John Dewey called “empirical metaphysics.” I confess to being unclear how the philosophical position of pragmatism is a justification for mixed methods—how does the primacy of experience lead to any particular methodology or method? And, is this a confusion of pragmatism with being pragmatic?

Related to this pragmatic justification is the triangulation justification, especially more contemporary notions of triangulation that focus on the complementarity and complexity added by multiple data sources, analyses, and so on. See my discussion of triangulation along these lines here.

Feyerabend’s “anarchist epistemology” might also justify mixed methods, either within the same study or across studies of the same or similar phenomena. This is in the big picture a more dialectical approach, working iteratively across paradigms rather than necessarily combining paradigms.

To justify mixed methods, one must at some level reject the incommensurability argument, i.e., the argument that the differences in epistemological theories cannot be overcome. (Note that the incommensurability argument at the level of the unit of measurement is easily overcome.)

In this article, Sommer Herrits’ illustrates two mixed methods (nested analysis and praxeological knowledge) approaches that operate at the paradigmatic level and result in substantially different emphases. I’ve included her summary comparison of the two approaches below.

Here are some additional resources for further exploring mixed methods…

Bergman, M. (2008). (Ed.) Advances in mixed methods research: Theories and applications. Thousand Oaks, CA: Sage.
Creswell, J. W. & Plano-Clark, V. L. (2018). Designing and Conducting Mixed Methods Research (3rd ed.). Thousand Oaks, CA: Sage.
Greene, J.C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.
Journal of Mixed Methods Research
Tashakkori, A., & Teddlie, C. (2010). Handbook of mixed methods research. Thousand Oaks, CA: Sage.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks, CA: Sage.

And here is a good example of a study using mixed methodology…

Castro, F. G., & Coe, K. (2007). Traditions and alcohol use: A mixed-methods analysis. Cultural Diversity and Ethnic Minority Psychology, 13(4), 269–284. https://doi.org/10.1037/1099-9809.13.4.269

Email, RSS Follow
Pin It
Email

“For sale: Baby shoes, never worn.”

methods, narrative inquiry

Six word stories have been around for many years now, instigated by Hemingway’s six word story, the title of this post. (Caution: there is dispute about this authorial  attribution, but I’ll take the story at face value here.) Hemingway’s story contains three elements, a 2-2-2 form. (Another common form is 3 -3 and these forms contribute to the aesthetics of the story.) The first “for sale” calls to mind a recognizable trope, followed by “baby shoes” a recognizable object, but it is the third element that turns this into a story. “Never worn” can invoke many possible meanings, but perhaps most common is the tragedy of child loss. Those baby shoes were never worn, because the baby wasn’t there to wear them.

These examples from reddit show the continued appeal of this genre, and the claim is this is a new genre. A collection of six word stories can be found at Six Word Stories sorted by category. An off-shoot of six word stories is Six Word Memoirs. Most six word stories are published on the internet, rather than in books or journals.

In The Poetics of Six-Word Stories, David Fishelov provides a cogent analysis of six word stories as a genre. He identifies seven (alas not six) features of six word stories:

  1. a chain of events
  2. something implied
  3. punch line structure
  4. poetic/rhythmic structure
  5. realism
  6. contagiousness of the form (you will want to write one too)
  7. predominately a genre in English

Six word stories have only one explicit rule… that they be six words. But looking more analytically at these stories they also seem typically to require something implied, something not explicitly stated in the story… the possible death of the baby in Hemingway’s story. This very short story form has a contemporary appeal in an age of texts, memes, and tweets.

Many six word stories are primarily about the aesthetics, but as the reddit examples illustrate they are also stories filled with meaning and the telling is heart-felt. There are plenty of pedagogical uses of six word stories, but they may be useful in research, especially narrative research. From a research perspective, whether six word stories are literature is less important than whether they provoke reflection, illustrate human experience and meaning, and perhaps whether they are a window through which we can see other stories.

While six word stories stand alone, they might also be prompts for more discussion of experience and meaning… a beginning to a more elaborated story.

Six words, only six words, alas.

 

Email, RSS Follow
Pin It
Email

Illustrated Interviews

image based research, interviews, methods

Interviews are a series of questions and answers and as social scientists we presume both use words. More common in the arts community is the possibility answers can be entirely or partly in the form of drawings, illustrations, images. Thinking about illustrated interviews within social science opens possibilities in forms of thinking and representation that often go untapped in interviews.

Asking interviewees to draw pictures that become part of interviews is a strategy I have used with children. Studying the impact of high stakes testing on children, I asked them to draw a self-portrait whilst taking the test. They also wrote a caption for the self-portrait and these drawings were used to engage the children in an interview about their experience.

In this same study, children kept journals and were encouraged to  write and draw. Not every child drew but most did at least some of the time suggesting the flexibility to chose how they represented their experience enhanced the likelihood of authentic sharing. You can read a report of this research here.

The NYT does a series of illustrated interviews… asking simple questions and having celebrities draw responses, which are then modestly animated resulting in a sense of action. Some interviewees are skilled at drawing, many are not. The interviews share a common set of questions and thus reveal a “life” for each person.

There are examples of drawings that represent concepts or ideas, like Jennifer Burtchen’s drawings of time, her drawing of the “present” shown here. This is one of a series of illustrated interviews in the magazine Ignant.

 

Liana Finck and Amy Kurzweil are both cartoonists and in this interview about a book written by Finck they use drawings in the margins of a more typical Q & A format to amplify the written responses.

Incorporating illustrations in interviews can be done in a number of ways:

  • interviewees respond in drawings, sketches, or even photographs
  • interviewees generate drawings that are used during an interview
  • interviewer drawings are used to enhance, elaborate, accompany questions

There are many examples of using drawings when doing research with children, but this strategy can work with adults as well.

Email, RSS Follow
Pin It
Email

Story Maps

data analysis, image based research, importance of place, methods, research examples, technology

A story map visually displays data in relation to places, location, or geography, and story mapping is the process of finding and analyzing the connections among human experience and place. Story maps can be simple or complex, low or high tech. And, story maps help in analyzing complex social issues such as human rights, climate change, refugee resettlement, student transcience, and community integration.

First, a simple example.

In a study of climate change, researchers worked with Ecuadorian subsistence farmers, used Post-It notes to facilitate a community discussion on climate change. Using a map as the basic reference, farmers in the mountainous central Ecuadorian province of Cotopaxi answered three questions: Has your community changed since you were a child? How has the climate changed since then? Are there any past climate-related events that affected you the most?

Responses were posted to the map, illustrating connections between place and observed human (such as illness) and climate (such as agricultural pests) events.

While this data collection and analysis was part of larger mixed methods approach, it illustrates how mapping human experience enhances our understanding of climate change. (Here is the article.)

More sophisticated examples.

Using existing web based templates (like StoryMapJS) or even Google Maps a story can be build that illustrate events and relationships. These tools rely on a narrative that moves through geographical space and the flow can be in space or time and space. An example of the former might be a subway line or interstate highway along which events, places or people can be placed. This story map of the Green Line train in Minneapolis is a good example. A static map example is of Pioneer Square in Seattle as a center for Queer history in that city. An example that combines movement through time and space is a story map illustrating the shifting population throughout USA history.

Often, story maps use GIS (geographical information systems) software, most commonly ArcGIS. Here is a link to Environmental Systems Research Institute, the most common portal for the use of this software, with illustrations from marketing, social science, and natural sciences. There are lots of examples on this website, but this story map of the experience of Rohingya refugees is a good place to start.

Story maps are often web-based, which facilitates interactivity and reveals movement within the story.

What applications might this have in educational research? Here are just a few ideas.

  1. Student mobility is an issue in the lives of some students, the quality of education received, experiences in school, and the experiences of schools. Mapping student mobility within a district or city combined with interviews, performance data, family characteristics, school environment and so on could provide insight into the experience of transcience and help schools plan better for the inevitability of student mobility.
  2. We know standardized test scores correlate as much with social class as with ability. Social class is closely linked to neighbourhoods and so mapping scores onto neighbourhoods, with additional information about income, types of housing, and cost of housing would reveal this relationship succinctly.
  3.  Generally, one might examine issues of space and educational inequality. Inequality that stems from race, ethnicity, or special needs are geographically unevenly distributed and revealing that distribution in communities, districts, schools and even classrooms could be done with story mapping.
Email, RSS Follow
Pin It
Email

methodology as recipe

big abstract ideas, methodology, methods, research design

A challenging idea for many graduate students learning how to do interpretive and critical research is that one must make a commitment to a research methodology. The two most common errors/scenarios are that students want to begin with research methods (ways of making and analyzing data) or research design features (the most common is to say they want to do a case study, a topic I discuss in some detail here.)

Screen Shot 2014-03-27 at 4.34.21 PMI find Michael Crotty’s definitions and relationships among the elements that frame research helpful. See this image and here, for a short note on this.

A methodology is a framework that contains core ideas reflecting more fundamental epistemological and ontological groundings, but it also provides guidance on the focus of our inquiry, key concepts, values, and often a hint at methods (although this is a separate matter). A methodology contains core definitional foci and cue us to what to expect in a research project. Methodologies also give us a sense of what we might expect the outcome/product of the research to be, even though there is tremendous variation in representational forms within any methodology.

Methodology as Recipeimgres

To help students understand what a methodology is and why we need one, I use the metaphor of methodology as recipe. A recipe is a framework for preparing food: it has a name that reflects what the outcome will be; a list of ingredients, procedural guidelines; tools and techniques; and often a color photo to show us what the food should look like when we are finished. And, different recipes illustrate fundamental differences. For example, a recipe for a chocolate cake and one for beef stew vary on all of these features and are therefore about something quite different. One would never follow a recipe for a chocolate cake and expect to end up with beef stew.

Chocolate Cake                                                            Beef Stew

flour, sugar, cocoa, eggs, butter                            beef, broth, carrots, potatoes, onions

whisking, stirring, mixing                                         chopping, browning, deglazing

mixer, cake pan, whisk                                              knives, heavy pot

baking                                                                               braising, stewing

Screen Shot 2016-05-04 at 4.47.08 PM Screen Shot 2016-05-04 at 4.47.34 PM

A methodology is a framework for doing research: it contains a name that reflects the outcome; what is needed to identify the research as being within that framework; procedures and tools; and we often have a general notion of what the outcome will look like.

Ethnography

culture, language, rituals, artifacts

field work: prolonged engagement, participant observation, interviewing, kinship charts, mapping, SNA

field notes, photographs, transcripts

thick description; analysis with cultural categories; emic perspective

Screen Shot 2016-05-04 at 5.03.37 PM

 

 

 

Novice cooks and researchers are more likely to follow recipes closely, developing the knowledge and skills that will in time free them from a specific recipe while still working within the recipe framework. A good cook might substitute Guinness beer for some broth in the beef stew, but she is still making beef stew. An ethnographer might substitute live field note taking using social media for more traditional field notes, but she is still taking field notes.

Without a methodology a research project is ungrounded, drifting and has a high probability of being atheoretical. With a methodology, with a recipe, the researcher plans on making an ethnography or a narrative analysis or a hermeneutic investigation because the core ideas, the ingredients, the tools are valued and indeed reflect deeper senses of the nature of both the world and knowledge about it.

Email, RSS Follow
Pin It
Email

learning by example

methodology, methods, publications, research design, research examples

If you think you will do interpretive or critical research it will be helpful to see what this kind of research looks like. Reading other studies is a window into the research process as well as the ways researchers represent knowledge from their studies. While many genres of research trade in the peer reviewed journal article, to get the most pedagogical benefit from reading qualitative research look more to book and monograph length works. First, good studies are complex and so it just takes more than 25 ms pages to communicate the findings. Second, increasingly interpretive and critical researchers include a confessional methodological tale in an appendix, a rich source of learning from others.

This is not really a part of the literature review for your study (although it could be) and to avoid conflating reading studies related to your research topic with learning
about research methodology and methods, I require my students and encourage others to read book length works that are not in your area of expertise. You might learn something about which you know little, but more importantly can focus on the research process more easily.

So what do I recommend as teachable/learnable texts, remembering that you are not looking for a perfect study but rather a really well done study that has flaws and features that permit you to see how the research process plays out in real research life.

The lists below hews mostly to education and schooling, but not exclusively.imgres

The books I have most often used in my research classes are:

God’s Choice by Alan Peshkin
Dude You’re a Fag by A. J. Pascoe
Ain’t No Makin’ It by Jay MacLeod
Working Class Without Work by Lois Weiss
Home Advantage by Annette Lareau

But, it’s good to go back to what I consider classic texts:

Boys in White by Blanche Geer, Everett C Hughes, Anselm Strauss, Howard Becker
Street Corner Society by William Foote Whyte
The Man in the Principal’s Office by Harry Wolcott
Deep Play: Notes on a Balinese Cockfight by Clifford Geertz (this is more a monograph, but too good not to list)
Learning to Labour by Paul Willis
Living and Dying in Murray Manor by Jaber F. Gubrium
Contradictions of Control by Linda McNeil
Life in Schools by Peter McLaren
The Polish Peasant in Europe and America by Florian Znaniecki and W. I. Thomas
Asylums by Erving Goffman

And, I can’t resist adding what has to be the most controversial piece of research in quite some time…

On the Run: Fugitive Life in an American City by Alice Goffman

 

Email, RSS Follow
Pin It
Email

case study ~ research design, methodology, method?

big abstract ideas, methodology, methods, research design

Since the early 80s when I did my MA degree I have been steeped in “case study research,” and I have the deepest respect for those who have articulated the importance of focusing on the particular in social science research (Robert Yin, Robert Stake, Sharon Merriam, for example). The work of these individuals is valuable ~ Yin provides a valuable foundation for why we should look at cases, and Stake has added detail about different motivations for looking at cases.

With a background in sociology and cultural anthropology my early exposure to case studies created little confusion ~ I was and remain interested in the particular. So researching and understanding a case make conceptual sense. Being schooled in ethnography as a methodology meant that using the language of case study provided a way to engage the tools of ethnography in a flexible way, focusing on pretty much anything that can be identified as a case, that is, a bounded system. Indeed, Creswell defines case study as “an in-depth exploration of a bounded system (e.g., an activity, event, process, or individuals) based on extensive data collection” and he goes on to clarify what a bounded system is: “the case is separated out for research in terms of time, place, or some physical boundaries.” The coupling of a bounded case in naturalistic settings aligns these methodological ideas with interpretivist and critical perspectives on research, but a case can also be investigated within a post-positivist perspective.

In his 2000 book, The Art of Case Study Research, Bob Stake characterized case studies as intrinsic, instrumental or collective. Intrinsic cases are those that are inherently interesting to a researcher, perhaps because of their uniqueness or peculiarity. Instrumental cases are those that researchers study because they have features connected to bigger concepts and that provide an empirical instance to study a bigger idea. Collective case study is looking at multiple cases often with a desire to compare and understand variation, in other words it is a collection of instrumental cases.

So far, we have case study as the investigation of bounded systems that we are motivated to investigate with three possible intentions. Sounds like a very important feature of research design.

But is it a methodology?

Research methodology is a framework that guides research practice ~ it is the theoretical frame that pulls epistemology forward into a discourse that further articulates the nature of knowledge and that guides our choice of methods. Crotty describes it as: “the strategy, plan of action, process or design lying behind the choice and use of particular methods and linking the choice and methods to the desired [research] outcomes.” Note that methodology is a theoretical framework. An important feature of methodologies is that they have substantive content, notions about what the focus of the research will be. If one does ethnography, for example, some notion of culture (even if adapted substantially from cultural anthropology) is central to the investigation. If one does critical research, some notion of power (and likely inequity) is central to the investigation. If one does narrative research, some notion of storying is central. So methodologies bring together salient, foundational social constructs with features of doing research. One could argue that certain methodologies logically entail the investigation of cases, in which case, case study could be an element of a methodology, but that it is only one among a number of elements.

So where does this leave case study as a methodology? What are the foundational social constructs that are central to it as a methodology? Here is where the logic breaks down. Looking at a case, in a natural setting, doing extensive data collection in natural settings doesn’t begin to hint at any particular foundational social constructs… inevitably researchers must draw on some other methodology for those. There is considerable variation in methodologies that inform what one does when investigating a case, and even when researchers do not articulate their methodology it lurks in the articulation of what those central social constructs are and the means by which we investigate them (reflection on pre-reflected experience in phenomenology; story telling in narrative analysis; culture in ethnography; and so on).

Terminology is inconsistent in discussions of research ~ there is a bewildering, often rolling sea of ideas, concepts, and practices to navigate in learning about research. There is no single ship of understanding, but thoughtful (re)articulation of the ideas underlying the theoretical and practical aspects of research is part of being in a community of social science researchers.

PS.

Further reading: This recently published comparative analysis of Yin, Stake & Merriam‘s take on what case study research is explicates their positions well, and I think still leaves unanswered the fundamental question of what case study is. My view of case study as a feature of research design is unchanged.

Email, RSS Follow
Pin It
Email

crowdsourcing ~ research possibilities

methods, research examples, technology

crowdsourcing |ˈkroudˌsôrsiNG|
noun
the practice whereby an organization enlists a variety of freelancers, paid or unpaid, to work on a specific task or problem

Social scientists have traditionally felt a need to tightly control how they get their data, who should provide the data and how are seen as key features of good research. But, we might take note of the ways other scientists (biologists, astronomers, mathematicians, ornithologists, and geologists, for example) are capitalizing on the lived experiences of people to help them collect and analyze data on natural phenomena.

The Case of Citizen Scientists

Many research projects now involve collecting or analyzing huge amounts of data, and both tasks are sometimes beyond the resources of an individual researcher or research team. Crowdsourcing the research tasks is being used in an increasing number of projects, mobilizing the general citizenry’s interest in science. Screen Shot 2015-06-09 at 1.36.04 PM

There is one web-based platform that has facilitated these research tasks; check out Zooniverse, which creates citizen science websites that allow anyone with an interest to participate in research online. These projects call on citizen scientists to help with data analysis. One recent example is Snapshot Serengeti, a website with photos taken in Serengeti National Park, Tanzania. Anyone with an internet connection can help classify the different animals caught in millions of camera trap images.

Crowdhydrology

Chris Lowry a University of Buffalo assistant professor of geology has developed Crowdhydrology, a project that enlists hikers, fishermen, birdwatchers, school kids and nature-lovers to monitor stream levels in NYS, Michigan, Wisconsin and Iowa. The idea is simple ~ at each site there is a giant measuring stick and a sign explaining how passersby can text water levels and stream locations to researchers. Citizens  in this project participate in data collection, contributing to research on hydrology. The data collected are public and available on the CrowdHydrology website, which describes the project thusly…

“The CrowdHydrology mission is to create freely available data on stream stage in a simple and inexpensive way. We do this through the use of “crowd sourcing”, which means we gather information on stream stage (water levels) from anyone willing to send us a text message of the water levels at their local stream. These data are then available for anyone to then use from Universities to Elementary schools.”

This is the interface that allows anyone to view or download the data.

Crowdsourcing for Social Science Research?

So imagine how social scientists might use crowdsourcing to investigate social issues and human phenomena. One example is a project in Egypt that gives women an opportunity to document rape, harassment and assault ~ Harrassmap tracks incidents of sexual harassment in the Greater Cairo area with the goal of understanding and changing the social acceptability of gender based violence. Social phenomena that are experienced or witnessed (safety, anxiety, happiness, bullying, crime, kindness, road rage, drug use, oppression) are all potentially chronicled, mapped and understood through crowdsourcing.

Also, check out the humanities projects on Zooniverse.

Screen Shot 2015-06-09 at 1.38.25 PM

A key issue in the successful use of crowdsourcing in the social sciences is that scientists do retain some control of the research questions and what counts as useful data, crowdsourcing as data collection ought not to mistake participation in data generation with expertise in the research topic.

Email, RSS Follow
Pin It
Email