Emoji: Imperfectly Filling a Gap in Text-based Communication

Molly Tuttle

University of British Columbia

 

 

 

Introduction

 

In 1982, computer scientist Scott Fahlman proposed the application of the emoticon smile :-), consisting of a colon, hyphen and parenthesis to help reduce miscommunication arising on internet message boards. He proposed that writers insert the smile emoticon to indicate they were joking (Jibril & Abdullah, 2013; Robb, 2014). Since then, other emoticons also have become common in computer-mediated communication. The three most common emoticons are the smile, frown 🙁 and wink ;-). A defining feature of emoticons is that they are strings of two or more standard ASCII (keyboard) text that when viewed sideways resemble expressive faces. (Kelly & Watts, 2015; Jibril & Abdullah, 2013; Walther & D’Addario, 2001)

Emoji, roughly translated as “picture word”, are colourful graphics that were developed in the late 1990s by Shigetaka Kurita, who worked for a Japanese telecom company. His company sought to distinguish themselves from other pager services and appeal to teen consumer market by offering digital cartoon images. Kurita sketched out 176 simple images in an attempt to cover the spectrum of human emotions (Sternberg, 2014). As these images became very popular, other telecom services began creating their own emoji (Blagdon, 2013). When Apple launched the iPhone in 2007, they needed emoji to appeal to the Japanese market, so emoji featuring a range of facial expressions and common objects (Novak et al., 2015) were installed on all iPhones, which could only be accessed by downloading a Japanese language app. Apple then did not anticipate interest from non-Japanese customers. However, after users in North America realized they could access the emoji, usage grew widely (Sternberg, 2014).

The use of emoji in text messaging and social media has exploded in recent years since the icons were added to Unicode Standard in 2009, making them compatible across service providers (Miller et al., 2016). In 2015, Oxford Dictionaries selected “face with tears of joy” emoji as the word of the year due to its popularity and to recognize the prevalence of emoji in online text-based communication (Oxford Blog). Emoji currently appear in half of all Instagram posts (Miller et al., 2016), as well as being used extensively on other social media platforms and in text messaging. Emoji, like their predecessor emoticons, are changing the way we share ideas and convey emotions online and offline as well. The use of emoji has expanded outside of text-based communication into many domains from advertising and public health initiatives (Skiba, 2016) to music videos (Hill, 2014) and literature (Skiba, 2016).

The use of emoji has significantly changed the way we communicate online. Visual icons can enhance personal expression in text-based communication to reduce misunderstandings (Walther & D’Addario, 2001), generate positive affect for recipients (Novak et al., 2015) and enable innovative and creative construction of meaning (Robb, 2014). Though emoji help fill a gap in text-based digital communication, much in the way body language and facial expressions enhance verbal communication, they are too limiting in the expression of emotion, ideas and identity, and are increasingly being co-opted by corporate interests. As emoji fatigue sets in, users will likely seek computer-mediated communication expression through other means. The following will first examine the benefits of emoji, followed by an analysis of the limitations and corporate involvement, and finally conclude by arguing emoji are a replaceable trend.

 

Filling a Gap in Text-based Communication

 

Research indicates that emoticons, and other graphic icons such emoji, can be considered quasi-nonverbal cues that have a significant effect on a receiver’s perceptions of message in computer-mediated communication (Lo, 2008). Emoticons have been found to enhance and complement verbal message interpretation, much in the way that facial behavior operates in F2F (face-to-face) communication (Derks, Bos & von Grumbkow, 2008). Primary motives for emoticon use are to “express emotion”, “strengthen the message”, “regulate the interaction” and “put into perspective” (Derks, Bos & von Grumbkow, 2008). Lo (2008) found that with the addition of emoticons, receivers are able to “correctly understand the level and direction of emotion, attitude and attention expression” in a way that they are unable to do in “pure text” (not containing emoticons) messages (p. 597). Recipients of emoticons consistently view the emoticons as expressing a clear emotional sentiment. Derks, Bos & von Grumbkow’s (2008) research on the impact of emoticon usage on message interpretation found that the addition of positive emoticons in text strengthens the positive text and negative emotions strengthen negative text. On the other hand, in the case of mixed messages, when the emoticon sentiment (positive, negative or neutral) does not match the emotional sentiment of the text, the emoticons influence message interpretation but carry less weight than the sentiment of the text and are interpreted as being more sarcastic than pure messages (matching emoticon and text sentiment). For example, in the following, “I failed my test :), the message would be interpreted as being more negative than positive, but would be viewed as expressing sarcasm (Derks, Bos & von Grumbkow, 2008). Research by Filik et al. (2015) concluded that emoticons have a greater influence on both comprehension and emotional impact than other form of punctuation.

The ability to express emotion and tone in text-based communication is a challenge that first emoticons, and now emoji help address. Emoji, unlike emoticons, often do not express as clear an emotional sentiment but still can soften and enhance messages. To investigate the emotional impact of emoji use, Novak et al. (2015) analyzed 1.6 million Twitter tweets and developed the first emoji sentiment lexicon of 751 commonly used emoji. In the study, participants viewed tweets and rated the perceived emotional attitude of the sender (positive, negative or neutral). Some of the tweets contained emoji and others did not. The individual emoji sentiment score was calculated based on the sentiment of all the tweets in which that particular emoji character occurred. This approach enabled them to investigate the feelings associated with emoji and differences between tweets containing emoji and those without. They found that the emotional perception of the tweets was effected by the presence or absence of emoji. Tweets containing emoji were rated as significantly more positive, and evaluated as expressing stronger emotions.

There is evidence that emoji usage also goes beyond the original goal of simply expressing emotion. According to research done by Kelly & Watts (2015), emoji use can increase reports of perceived of connectedness and intimacy through maintaining a conversational connection (telling someone you are thinking of them when you have nothing to say, acknowledging receipt of text, ending conversation) permitting creative play (choosing emoji based on relevance to recipient and the use of repeated emoji) and creating shared and secret uniqueness (though collaborative storytelling, creating new meaning based on your context and relationship with the recipient). Studies by linguist Tyler Schnoebelen found use is similar in variation to dialects and friends often develop their own meaning and create emoji slang (Robb, 2014). Sternberg (2014) sums up the flexibility and creativity offered by of emoji in the following:

As Wortham explains about her favorite emoji, the Tempura Shrimp, what she loves about it is precisely the fact that it can have many different meanings. Sometimes she uses it to mean a foul or “salty” mood, when she wants to curl up like a shrimp. With some of her friends, the shrimp morphed into a joke that stands in for “Mariah Carey.” (“Something about her complexion and the way she’s always stuffed into a tube-ish dress,” Wortham writes in her shrimp essay.) Others use the shrimp as “quirky filler”-a nod, a wink, an acknowledgment that you’re simply thinking of someone. Tempura Shrimp emoji, she writes, has become “a way to be present when there’s nothing else to say at all.” (Sternberg, 2014)

            The use of emoji may also reflect a desire for positive, kind and supportive expression on social media platforms. The majority of emoji used are positive, and these are the most commonly used emoji. People expressed positive emotion through a wide variety of emoji including happy faces, hearts and celebration graphics, whereas negative emotion was expressed almost exclusively through sad faces (Novak et al., 2015). Some suggest that cartoonish emoji are inherently positive in nature and their popularity has arisen as a response to the negativity of the online commentary, and usage has increased as an “anecdote to the incivility” online. (Sternberg, 2014)

Limitations of Expression

While Emoji can enhance online communication, they are an imperfect replacement for non-verbal F2F communication. Differing interpretations of emoji within and across platforms can lead to miscommunication of meaning and emotional expression. Emoji also are not culturally or politically neutral and can severely limit expression of identity. The following demonstrates the ambiguity of emoji usage:

A few weeks ago, after I said goodbye to a friend who was moving across the country, I texted her an emoji of a crying face. She replied with an image of a chick with its arms outstretched. This exchange might have been heartfelt. It could have been ironic. I’m still not really sure. (Robb, 2014)

Unicode Consortium assigns a code and name to each emoji character, but each platform has a different rendering for the emoji code, so the same emoji code will look different on different platforms. There are at least 17 different platform renderings of each emoji Unicode character (e.g.. Apple, Microsoft, Google). Miller et al. (2016) conducted a study to measure people’s interpretations of emoji sentiment (positive, negative or neutral) and semantics (emoji meaning) within the same platform and across five popular smartphone platforms. They focused on the 25 most common anthropomorphic Unicode emoji characters (faces and people) used by Twitter users. They found that the sentiment of emoji are often interpreted differently both within the same platform and across platforms. About 25% of the time participants who viewed the same emoji rendering (within same platform) did not agree on the sentiment expressed by the emoji. The highest agreement was for Apple’s rendering of the Unicode character “sleeping face” which most participants evaluated as being neutral in sentiment. Microsoft’s rendering of “smiling face with open mouth and tightly closed eyes” had very high levels of sentiment misinterpretation, with about 50% believing it to express a positive emotion and 50% a negative one. Results revealed a wide disagreement of sentiment between one or more platforms rendering of emoji characters. For example the emoji for “sleeping face” was viewed as neutral on four platforms but strongly negative with Microsoft’s rendering. Similarly, in terms of semantics (perceived meaning), emoji were interpreted as having very different meaning for both within platform emoji renderings and across platforms emoji renderings. This is particularly profound across platforms. Participants interpreted Apple’s depiction of “grinning face with smiling eyes” emoji as meaning “ready to fight” while interpreting Google’s depiction of the same emoji as being “blissfully happy.” Within a platform even though the sender and the receiver see exactly the same graphic, interpretation of emotion and meaning may differ. However, emoji use across platforms can lead to even further miscommunication because the sender and the receiver are not viewing the same graphic rendering.

Due to common misinterpretation of sentiment and semantics, research shows that emoji cannot be used effectively in the absence of text. Despite this, a social media site, emojli, that only uses emoji for communication attracted over 50,000 members in two days. A Kickstarter project translated Moby Dick in its entirety into emoji through crowdsourcing volunteers (Robb, 2014). However the strings of emoji are completely unintelligible without the accompanying line of “pure text” below. These two endeavors show a desire to use emoji in new playful and creative ways, while at the same time sacrificing comprehensibility.

Emoji, far from being culturally or politically neutral, contain a strong Japanese and western bias in terms of icons in categories such as food, clothing and transportation. Tokyo Tower and the Statue of Liberty are featured as landmark icons, but few other countries have such representation available. (Unicode, n.d.). Flags have become an area of political contention. The Palestinian flag was absent from the first set of emoji flags, but later added. There is currently no Kurdistan flag, Tibetan flag, or more locally a Quebec flag. (Unicode, n.d.; Shade, 2015) How can one adequately express their experiences using visual icons that fail to represent their local reality and cultural or political identity? “The presence or absence of emoji both hints at and contributes to cultural visibility and erasure” (Shade, 2015). Unicode attempts to deal with issues of underrepresentation, but is unable to meet the high demand for more diverse emoji to express identity and represent what is meaningful to users. In 2015, racial options were made available for people emoji, but critics feel that simply varying the color and not other facial features does not go far enough at equal racial representations (Shade, 2015; Robb, 2014; Sternberg, 2014).

 

Corporatization of Emoji

            Corporations have become increasingly involved in both the development and use of emoji to strengthen brands and encourage customer engagement in ways that may significantly change the relationships people have with emoji. There are three basic avenues for a corporation to develop a desired emoji. First, brands can submit a proposal through the Unicode Consortium, as Taco Bell and Guinness have done in attempts to get a burrito and dark beer emoji (Bradley, 2015; Wohl, 2016). The approval process can take years and brands must demonstrate a significant market demand for the emoji (Unicode, n.d.). The second way is to bypass the Consortium and create their own set of brand emoji that must be downloaded by users as a keyboard app. Ikea, Dove and Eggo have opted to go the faster route of brand emoji keyboard apps available to users if they feel the need to share waffles, curly hair, or furniture on social media (Bradley, 2015; Wohl, 2016). The final way is through a paid social media ad campaigns. Coca Cola became the first paid brand emoji, through an ad deal with Twitter to create a brand specific emoji of two Coke bottles clinking that will appear when users tweet #shareacoke (Nudd, 2015), since then other companies including Dove and Starbucks have paid a million dollars for branded hashtags on Twitter. (Johnson, 2016)

In addition to being involved in the development of new emoji, many companies are also using emoji already in existence extensively for advertising and user engagement through social media. GE’s Emoji Science created a periodic table made up of emoji icons. Users could send an emoji via SnapChat to GE and they would send back a corresponding science video. The three-day campaign resulted in a 260% growth of GE’s SnapChat community (Bradley, 2015). Corporations also use promotions involving emoji to engage the public, such as delivering pizza to users who send a pizza emoji to them (Wohl, 2016).

 

Replaceable Trend & Emoji Fatigue

 

Pavalanathan & Eisenstein (2015) found that emoji compete to fill the same niche that emoticons previously occupied and lead to an overall reduction in orthographic variation on social media. Their study found that Twitter users who adopted emoji subsequently reduced their usage of emoticons. According to Highfield & Leaver (2016) “The contemporary visual social media landscape replete with GIFs, selfies, emoji, and more is the latest iteration of networked communication with a long-running theme: we have always found ways to be visual online” (p. 48). Visual material online is not a new phenomena, it has just changed form and become more sophisticated. Online visual content such as ‘under construction’ icons, basic GIF banners, emoticons and social sites that required the uploading of photos started this trend. Like emoticons have to a large extent been replaced by emoji, another visual representation that reduces some of the miscommunication and ambiguity resulting from emoji, will likely replace them completely in the future.

The increasing development and use of emoji by corporations may change the relationship people have with emoji and lead to emoji fatigue and a decline in organic use. Emoji are increasingly visible offline as part of marketing campaigns as opposed to actually serving to facilitate text-based communication. Currently in supermarkets and convenience stores you can see proprietary emojis looking back at you from Pepsi cans and bottles, occupying the position of the logo. The company is trying to profit from people’s sense of personal connection to these icons, and have attempted to form their own complete emoji language. Currently 600 PepsiMoji icons are available for download in 100 global markets. On their Design & Innovation website, Pepsico states, “the designs incorporate the essence of Pepsi…for a universal language that is proprietary to the brand” (PepsiCo). Similarly the increasing branded emoji on Twitter will likely result in less organic interaction and creative expression on the platform. Efforts by corporations to bypass Unicode’s transparent and seemingly democratic, though lengthy, approval process may serve to negate the feelings of shared creation, ownership, and connection with quasi non-verbal language supplements.

 

References

 

Blagdon, J. (2013, March 4). How emoji conquered the world: The story of the smiley face from the man who invented it. The Verge. Retrieved on July 15, 2016 from http://www.theverge.com/2013/3/4/3966140/how-emoji-conquered-the-world

Bradley, D. (2015, June). Emoji-mania: Not just for kids as brands embrace text tool. PRweek, 18 (6), 14.

Derks, D., Bos, A. E. R., & von Grumbkow, J. (2008). Emoticons and online message interpretation. Social Science Computer Review, 26(3), 379-388.

Filik, R., Turcan, A., Thompson, D., Harvey, N., Davies, H., & Turner, A. (2015) Sarcasm and emoticons: Comprehension and emotional impact. The Quarterly Journal of Experimental Psychology, 1-17.

Highfield, T., & Leaver, T. (2016) Instagrammatics and digital methods: studying visual social media, from selfies and GIFs to memes and emoji. Communication Research and Practice, 2(1), 47-62.

Hill, J. (2014). Beyoncé (feat. Jay-Z) “Drunk in Love” Unofficial Emoji Video. Retrieved from: https://vimeo.com/88073857 on June 15, 2016.

Jibril, T. A., & Abdullah, M. H. (2013). Relevance of emoticons in computer-mediated communication contexts: An overview. Asian Social Science, 9(4), 201.

Johnson, L. (2016, February). Twitter’s Branded Emojis Come with a Million-Dollar Commitment. AdWeek. Retrieved on July 26, 2016 from http://www.adweek.com/news/technology/twitters-branded-emojis-come-million-dollar-commitment-169327

Kelly, R., Watts, L. (2015). Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design.

Lo, S. (2008). The Nonverbal Communication Functions of Emoticons in Computer-Mediated Communication. CyberPsychology & Behavior, 11(5), 595-597.

Luangrath, A., Peck, J. & Barger, V. (2016) Textual paralanguage and its implications for marketing communications. Journal of Consumer Psychology

Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., Hecht, B. (2016). “Blissfully happy” or “ready to fight”: Varying Interpretations of Emoji. Association for the Advancement of Artificial Intelligence. Retrieved from http://www-users.cs.umn.edu/~bhecht/publications/ICWSM2016_emoji.pdf

Novak, P., Smailovic, J., Sluban, B., & Mozetic, I. (2015). Sentiment of emojis. Plos One, 10(12),

Nudd, T. (2015). Twitter Unveils Its First Paid Brand Emoji and It’s for (Who Else?) Coca-Cola. AdWeek. Retrieved from: http://www.adweek.com/adfreak/twitter-unveils-its-first-paid-brand-emoji-and-its-who-else-coca-cola-167019

Oxford Blog. Retrieved on July 20, 2016 from http://blog.oxforddictionaries.com/2015/11/word-of-the-year-2015-emoji/

Pavalanathan, U., Eisenstein, J. (2015) Emoticon vs. Emojis on Twitter: A Causal Inference Approach. Association for the Advancement of Artificial Intelligence. http://arxiv.org/abs/1510.08480v1

PepsiCo. Design & Innovation. Retrieved on July 27, 2016 from http://design.pepsico.com/pepsimoji.php#section5

Robb, A. (2014) How Using Emoji Makes Us Less Emotional. The New Republic. Retrieved from: https://newrepublic.com/article/118562/emoticons-effect-way-we-communicate-linguists-study-effects

Shade, C. (2015) The Emoji Diversity Problem Goes Way Beyond Race. Wired. Retrieved from: http://www.wired.com/2015/11/emoji-diversity-politics-culture/

Skiba, D. J. (2016). Face with tears of joy is word of the year: Are emoji a sign of things to come in health care? Nursing Education Perspectives, 37(1), 56.

Sternberg, A. (2014, November 17) Smile, You’re Speaking Emoji: The rapid evolution of a wordless tongue. New York Magazine.

Unicode. (n.d.) Unicode Emoji Charts. Retrieved on July 10, 2016 from http://unicode.org/emoji/charts/

Walther, J.B. & D’Addario, K.P. (2001) The impacts of emoticons on message interpretation in computer-mediated communication. Social Science Computer Review, 19. P. 323-345

Wohl, J. (2016, April 7). How Marketers Can Win the Great Emoji Arms Race. Advertising Age, 87(7), 26.