Discrimination, as it turns out, is hardwired within everyone. Most people probably think individuals belonging to a particular race all look the same. But, what is the difference between an honest mistake and intentional racism? The Cross Race Effect, first introduced by Gustave Feingold in 1914, states that people can better identify differences in the faces of people from their own races, and less clearly so among those from other races. This does allow room for honest switch-ups, such as mistaking one person for another — but where does one draw the line?

A new study, titled “Neural Adaptation to Faces Reveals Racial Outgroup Homogeneity Effects in Early Perception,” published in the U.S.-based journal Proceedings of the National Academy of Sciences, states that this difference in processing originates in the visual cortex — one of the earliest cognitive processes that happen when you encounter something new. The study involved using a functional MRI paradigm to measure the most familiar faces for 17 white people. Blocks of white and black faces that looked quite different from each other were shown to the participants. Researchers found the participants showed increased sensitivity in how they adapted to the physical difference among white faces i.e. from their own race, rather than from a different race.

Related on The Swaddle:

All the Arguments You Need: To Convince People That Some Dating ‘Preferences’ are Discriminatory

Having your face mistaken for another member of the same race multiple times is experiencing a microaggression, i.e. when people’s biases against a particular marginalized group reveal themselves in a way that makes the marginalized individual feel uncomfortable or less than. This is because it is perceived as insulting when people believe there’s no diversity in a particular race and that everyone from that particular race is interchangeable for another.

The problem is, while there’s always room to make honest mistakes — there’s a disproportionate difference between who makes these mistakes and who suffers the microaggression. Brent Hughes, lead author on the study said, “Members of minority groups wind up being exposed to more members of majority groups than majority members get exposed to minority members.” Since majority groups aren’t as exposed to minority faces, they’re more likely to confuse two different races in day-to-day life — like at a party, at the workplace, and if you’re particularly unlucky, by police who mistakes you for a criminal.

The latter and the most terrifying instance occurred when a white New York City policeman arrested biracial tennis star James Blake, mistaking him for another black man suspected of credit card fraud. While science can explain away the policeman’s ‘mistake,’ the phantom terror of potentially worse situations (such as permanent or fatal aggression stemming from misidentification) remains in the background.

While it would be a stretch to blame an individual exclusively for behaving as they were conditioned to by their surroundings, there ought to be a line drawn between an honest mistake and racist intent. Where one can draw this particular line is tricky; while a cop mistaking a person for another criminal of the same race definitely falls on the irresponsible and dangerous end, the forgetful old boss who keeps jumbling names and faces may not deserve the same flak, even though the behavior still causes mental distress and irritation, which cannot be ignored.

A probable way to differentiate between racism and an honest mistake could be to factor in other racially tinged behavior from that individual. A one-off is more likely honest ignorance rather than callousness. But, one-offs can also prove fatal. Police can shoot the wrong person, an eyewitness could identify an innocent individual as a perpetrator, a doctor could bungle treatments for patients — the possibilities are endless and scary.

Drawing the line in these situations becomes further strained because — what if the honest mistake is a function of deep-seated bias? In the study, Hughes writes that these early differences in racial perception might affect beliefs and behaviors as a person grows older.”We are much more likely to generalize negative experiences if we see individuals as similar or interchangeable parts of a broad social group,” he said.

The only way to draw the line between an honest mistake and racist intent is how clearly an individual owns up to their error and takes the responsibility to do better. The doing better bit? That’s got to be action: conscious unlearning and immersing oneself in different cultures, rather than just lip service. Researchers told the New York Times that people can improve cross-racial perceptions if there is a strong need to do so. “[For] a white woman, relocating to Accra, Ghana, for instance, would heighten her ability to distinguish between black faces, just as a black man living in Shanghai would enhance his ability to recognize various [Chinese] faces.”

Hughes also believes that these effects are controllable. “These race biases in perception are malleable and subject to individual motivations and goals. In this sense, attitudes, motives and goals can be shaping visual perceptual processes.”