THE FIVE EMOJI SKIN TONE OPTIONS DON’T ACCOMMODATE A DIVERSE WORLD

How do you choose which emoji skin tone to use? This week on Why’d You Push That Button, Vox’s Kaitlyn Tiffany and I discuss the five emoji skin tones (not counting the default gold option) and how people decide which color best represents them. The tones debuted in 2015, and now, three years later, people have studied how they’re used and how commonly people opt to change the default option. The choice isn’t as simple as you might think.

We also change the show up this week. Instead of relying on just two users, we wanted to hear about as many experiences with the emoji as possible, so we have lots of guests. Thank you to all of them for coming on the show, including Ben, J., Jordan, Joshua, Rosie, Soco, and Malachi. We also received lots of emails when looking for guests, so thank you for writing to us.

After we hear from everyone, we chat with two expert guests. The first, whose interview is transcribed below, is Alexander Robertson. He’s a PhD candidate at the University of Edinburgh who published a study — called “Self-representation on Twitter using emoji skin color modifiers” — about skin tone emoji and their usage across Twitter and around the world. Then we chat with Zara Rahman, a linguist and writer, who published a piece called, “The problem with emoji skin tones that no one talks about.” She walks us through her story and how her interviewees felt about the tones.

Sure. So in 2015, the Unicode Consortium, which is the body that determines what’s going to become emoji and what’s not, they added the ability for users to change the appearance of some emoji to make them look more human in terms of skin color, and five skin tones were added, and the idea was that this was going to help to represent human diversity.

Initially, when people realized this, they were like, “Well, this is going to go one of two ways: either people won’t use them because there’s already too many emoji to begin with, and this is just multiplying the number of emoji by five in some cases, or, people are going to use them to be abusive on Twitter, especially.” But nobody did any research into this. People were quite sure. They’re like, “No, people are definitely not going to use them, or they’re definitely going to be abusive online.”

So the work that we did here at Edinburgh looked at whether this was true. In the case of the first question of do people use them. I found overwhelmingly, yeah, they’re very, very common. So a vast majority of emoji that can have a modifier are modified for skin color. Then, in terms of whether they’re used abusively, the way that we looked at this was determining whether people use a skin color that matches their own skin tone in real life. What we found is that overwhelmingly, people do pick a skin tone on an emoji that is very close to their own skin tone in real life. Also, people very rarely use emoji that are at the other end of the scale.

We also did a sentiment analysis, though, of these emoji of these tweets that had emoji at the far end of the scale compared to what you would expect based on the user, and they were actually more positive overall. So again, this is just saying overall, one, people do use these modifiers, and two, they’re using them in actually a very positive way. They’re using it to represent themselves, and even when they’re not representing themselves with them, they’re using very positive language.

Kaitlyn Tiffany: Do you have any numbers on how many people use them and what your sample size was?

We started with a sample of something like 0.6 billion tweets. We gathered these over the course of a year from Twitter. From that, we started with 10,000 users at random, and from these random users, we got their Twitter profile pictures, and we annotated them for the skin color of the person in the picture where possible. Not everyone has a profile picture on Twitter, as I’m sure you all know, so we ended up with around 5,000, I think it was, annotated users that we had this demographic information for and their tweets that we could look at.

Kaitlyn: Do you know roughly how many of those people were using emoji with the different skin colors?

Not all of them have got emoji, but let’s say 83 million do have emoji. Of those 83 million, then 30 million of them have an emoji that you can modify for skin color. Of these, half of them had been modified. So, in terms percentages, then we’re looking at, in half of the cases where you could apply a skin tone modifier, people did actually do that.

Kaitlyn: If we could go back to the impetus of the study, why were you as curious about this question specifically? We already care, but why should anybody care about how other people are using emoji skin tone?

I think what really motivated me was that people had very strong opinions about what they thought was happening, but they didn’t have any facts. No research had been done on this. I had no opinion to begin with either way. I wasn’t one of the people saying that, “Oh, yeah, people will be using these modifiers,” or “People won’t be,” or “People will be being racist,” or “People won’t be being racist.” I just wanted to explore this and look at it from a data perspective. That’s really what got me interested in the first case was just answering these questions, which I was sure I could answer from just looking at the data.

Ashley: A classic scientist, loving data. Well, so anecdotally, when Kaitlyn and I had first talked about this episode, I thought what we were going to find was that there were more light-skinned people using darker emoji. You actually found the opposite of that, right?

Absolutely, yeah. First of all, if you go onto Twitter, and say you got all of the emoji that have skin color modifiers, and you code them all up, and you just see what’s more common: do you have more white ones or more dark ones? You’re going to find more light-skinned emoji on Twitter, and that’s simply because there are just more white people on Twitter.

What was interesting about the study that I did was that we got this kind of background knowledge. We were able to control for the sizes of populations on Twitter by finding out, well, which of these users have which color of skin, essentially. Then, once you take that into account, you can see that, although there are more white people and more white emoji altogether, when you look at how often people of particular skin colors use these colored emoji, actually the darker your skin is, the more likely you are to use these emoji.

Ashley: To be more specific, so you grouped people by geography, right? And that’s kind of how you assumed what skin color they were?

No. That’s the first analysis we did. That was the sort of easy one. It was like, yeah, we grouped it by geography. We count up all the five colors, and we see what the distribution is. No matter where you are in the world, you see that it’s always the lightest skin tone is the most common, and the darkest one is the least common. But when you group people by their actual skin tone in real life, even though you’ve got, say, 1,000 people with the lightest skin tone but only 100 people with a darker skin tone, you find that they use them at very different rates. When you can control for different population sizes, you can actually look at their relative usages of these skin tone colors, and that’s where you see that white people essentially use them less often than people with darker skin tones.

Kaitlyn: What’s most interesting to me about this question is the way that people use them in conversation and whether they’re using them differently when they’re referring to themselves or referring to someone else or just based on the skin color of the person they’re talking to. I think maybe that would make a difference. We had an email from a listener who said that he’s mixed race, so he will use a default emoji until someone he’s talking to uses an emoji, and then maybe he’ll change. It’s kind of like he’s just waiting to see how the conversation goes. I guess I’m curious, on Twitter especially, when maybe the emoji people are using don’t refer to them, or to another specific person, but to a general thought or a famous person even. How do you see people using them differently?

That’s really interesting that you ask that because I didn’t cover that in the paper that was published earlier this year, but the paper that I am submitting for review this Friday is actually asking exactly that question. So I can actually give you some answers right now.

Overwhelmingly, people use emoji with skin tone modifiers for self-reference. But in this new study that we’ve done, we looked at cases where people were using a skin tone color for their emoji that didn’t match their actual skin color. Then we looked at the circumstances around this, and we classified them into different types of usage, essentially.

We found that often people will use them to refer to other people or to groups of people. This is especially if they are chatting to someone that they know on Twitter, so if they mention someone in a tweet, and it’s a direct, public message, then if they’re using an emoji to refer to that person, then they’ll also potentially, but not all the time, match the skin tone of the emoji to the person they’re referring to. But I think it’s important to point out that this is just extremely rare, so under 4 percent of millions and millions and millions of emoji usages, most of them do refer to the person who’s writing. But there definitely are people out there, like your friend, who use them to refer to other people, or choose to turn them on based on who they’re talking to.

Kaitlyn: What are the different emoji shades based on?

They’re based on the Fitzpatrick scale, which is a measure of skin color in humans based on levels of pigmentation. This is actually a six-point scale, but we use it in emoji, we reduced it down to five, so skin tones one and two are represented by the first emoji skin tone, and then three, four, five, and six are covered by the rest.

Kaitlyn: When someone’s using an emoji with the skin tone modifier and it doesn’t match their actual skin tone, what’s going on there usually?

In many cases, it can just be a mistake, right? Because you’re on your phone, you’re typing, and you click the wrong icon. It’s very easily done. I do it all the time. You’ve got big thumbs on tiny phones. But occasionally, you’ll be talking with someone that has a different skin color to you. This is very common in the case of celebrities. One tweet that I saw was someone very excited about being pregnant, but especially because they were pregnant at the same time as Beyoncé, and therefore, they used emoji for the women in this tweet that matched Beyoncé’s skin tone rather than the original tweeter’s skin tone. So that’s one you’ll see. That’s a very easy example to figure out what’s going on because they’ve mentioned Beyoncé in this tweet, and they’ve used emoji that matches everyone nicely.

Also occasionally, you’ll find that if people are talking to people that they know, then that’s another situation where they’ll match their friend’s skin tone. Those are examples of direct reference, but we also categorize a second type of that, which is oblique reference. So you don’t mention someone by name or by username on Twitter, but you’re definitely talking about someone that isn’t you, and you’re just kind of like vaguebooking. You want to refer to this person in a roundabout kind of way. In many cases, those tweets will have an emoji where the skin tone doesn’t match the user who’s writing. But it’s kind of clear from the context that this emoji refers to this other person, and they’ve included a skin tone that matches them as well, to some extent.

Ashley: What about when people are not using their own skin tone to refer to themselves? Is there something deeper going on there, like internalized norms or something around race?

Potentially. I know there’s a lot of work being done on racial identity and the idea of skin tone politics in the real world and the idea of what skin colors are essentially desirable based on your ethnic background and so on. You see to some extent this is actually also reflected on Twitter. We have annotators looking at photos of people and agreeing on what that skin tone is for that person, but that person themselves might actually think, “Well, no, I’m one tone lighter or one tone darker.” We haven’t looked into this specifically, why that is, whether it’s to do with social norms or to do with personal identity or pride, but definitely I think that’s work that we will do in the future. One of the problems is that those users are still quite rare. People do tend to use these emoji to match themselves quite closely.

Kaitlyn: I think the weirdest thing for me about skin tone emoji is what to do with the gold default emoji. Do you happen to have any data on is that emoji coded in some social way as white? Because I use it just because I feel like a little bit creepy being like, “I’m going to select the white emoji to emphasize my whiteness.” But I don’t know how other people feel about it. It’s just weird. I don’t know.

I don’t think you’re alone there at all. Also, back when these became available in 2015, another thing that people said was that, “Oh, white people will never use these because they’ll be too embarrassed or too ashamed. Only white supremacists will use them because by putting a white skin tone on your emoji, you’re celebrating your whiteness in some way.” But then, of course, what the data shows is that no, white people are essentially shameless, and they are using these very often but at a lesser rate than people with darker skins. I think that’s essentially because there isn’t that sense of pride that goes alongside being white, potentially, in the same way that, for minorities, taking that minority status and turning it into a badge of honor, that’s one way of dealing with being a minority and being proud of yourself and your background. Therefore, emoji can be a way of doing that.

Kaitlyn: But I also don’t want to claim the gold emoji, the default emoji, as the white person emoji. Perhaps I’ll just opt out. This doesn’t need to be a problem for me.

With the yellow one, though, you’re right. This is the entire reason the Unicode Consortium brought in these new skin tones because the original argument is that, “Oh, yellow doesn’t represent anyone. No one has yellow skin, this bright gold skin.” But when you look at it from the perspective of actual people, the yellow tone is essentially a proxy for whiteness. I think it’s very telling that the original emoji weren’t bright green or bright purple. I think in that case, you maybe have a bit of an argument for saying, “Oh, they’re not pretend white,” essentially.

Kaitlyn: I know you said this is rare, but when a white person is using a brown emoji, why are they doing that?

In most of the cases, it could just be input error. But in some particular cases, it’s very clear that they are being used for stereotypical purposes. It’s white women referring to black men using black emoji. I’m sure you can imagine the other emoji that go alongside these and the sort of messages that are involved. But yeah, it’s referring to these sort of stereotypical things. Whether these are real users or not, or whether they are porn accounts is something that I have not looked into. But I think based on how uncommon this is and how we see it in so few users, I think that it’s likely to be porn accounts on Twitter, promoting certain types of content.

Ashley: We’ve talked a lot about kind of the social aspects here, but to kind of move into more of the product limitations, most recently, Apple introduced bald emoji and red-haired emoji. I think you even mentioned in this paper that a lot of skin tones aren’t represented. I’m wondering if you found that a lot of people just don’t see themselves represented here. The possibilities are truly endless. There was a really good blog post about this, how basically, there’s just like an endless combination of skin tones, hair color, hair type, eye color. I mean, all of these things, and it seems like the emoji themselves have also come with pre-designed combinations that don’t necessarily work.

Going back to 2015 again, the Unicode Consortium says, “We need more representation of human diversity in emoji.” Absolutely brilliant goal, and to some extent, they’ve achieved that. But the five skin tones that they chose are not well distributed in, well, first of all, in the color space. You can take a color. You can represent it as three different values. Then, you can plot these in a little 3D cube.

If you do this for the five skin tone emoji that are available, you see that the first four from lightest to second less darkest, they form a nice spread-out pattern in three dimensions. But then when you get to that fifth-darkest emoji, it’s way off in its own little corner. The difference between skin tone four and five for emoji is much more different compared to between two and three or three and four. This means that there’s a whole load of people who probably fall somewhere between four and five who aren’t being represented. Essentially, five skin tones for emoji just isn’t enough.

But then, this brings back to what you were saying about the practicalities of inputting these on devices. It kind of explodes the possible combinations that you can have. You can imagine your emoji keyboard and have to scroll through it for six days just to get to the exact right combination of hair length, hair color, hairstyle, and then you’ll add your skin tone and maybe your eye color as well. So, I understand why they have to limit this, but I think the way that they’ve chosen those five limited options hasn’t been done very evenly.

Kaitlyn: They should just make a little version of Bitmoji that’s for your emoji keyboard. Why isn’t that a thing already?

Behind all those little pictures that you see, there’s fixed code points, and there’s a finite amount of those, essentially. But the main issue is how do you render them on those little, tiny keyboards? In terms of human-computer interaction and user interface, that’s a whole other problem. Regardless of whether you’ve chosen the right five skin tones, it’s like, well, how are you actually going to put any more on screens?

Ashley: Honestly, do you think this is something that can be solved? It kind of sounds overwhelming. I mean, the blog post I’m referencing, Jeremy Burge, he’s a big deal in the emoji world. He points out that our menus are already so stacked with emoji and options. If you tried to create that pull-down menu for skin tone and hair color any bigger, you’re just in a whole ‘nother world. Do you see a solution for this? Have you thought through what the future could be?

I think in terms of skin tone, it’s maybe slightly easier than it is for hair. Because hair is discrete. You’ve got straight. You’ve got curly. You’ve got bald. There’s a very small number of combinations there. I don’t think people will be too upset that there isn’t an emoji which perfectly represents their hairstyle as it is that day. But in terms of skin color, because you can represent this along a continuous spectrum, what I can imagine is that in the future for these keyboards, you don’t have the five colors shown to you in a little list. Maybe just at the beginning, you pick one color from a wheel or some other interface. I don’t know. I’m not an interface designer. I’ll tell you that right now. But people choose this, and it just gets set. They choose the color that’s best for them, and then this is what gets used for everything.

But then, this also brings problems for the users that we’ve talked about before who might use them to refer to other people and / or also want to turn it on and off, depending on the circumstances. So yeah, I think we could definitely make it easier to pick a color which better reflects people’s identities, but whether we could make it convenient every time we wanted to change that is another question.