Online Hate Ecosystem Primer
To better understand online hate, it is important to illustrate its ecosystem, the web of connections between bad actors, victims, institutions, behaviors, causes, and outcomes. These connections often overlap and influence one another. The ecosystem is complex and involves the behaviors of various types of actors across multiple ideologies and biases expressing themselves in ways ranging from subtle harassment to explicit calls for violence.
By visualizing the online hate ecosystem, we can create a shared vocabulary around how online hate arises—the reasons why people engage in hate, the targets of that abuse, the forms the abuse takes, the tools and platforms used to commit or mitigate abuse, and the damaging consequences of hate.
As a starting point, we use three diagrams to depict the ecosystem of online hate.
The Perpetrators / Targets / Mitigators / Amplifiers diagram (pdf) identifies and examines the four primary contexts and roles within the online hate ecosystem. The Perpetrators section examines who perpetrates online hate, what types of hate are happening, and what compels people to post hateful content. The Targets section explores why and how people are targeted, as well as the outcomes of being targeted by online hate. These two sections can mirror each other: Religious bigotry is linked to targeting Jews, Muslims, Christians, Mormons and other faiths, for example. Outcomes for targets can be subtle (hurt feelings) to devastating (lost jobs, reputations and lives).
The Amplifiers and Mitigators sections explore the people, places and tactics across social spaces through which hate spreads and rhetoric is broadcast and amplified, or is mitigated and blocked. What are the activities that occur in gaming, social platforms, and media that spread online hate? Who is working to stop online hate? Looking at the Mitigator section, we see system actors—governments, lawmakers, media—as well as individualș—teachers, parents, celebrities and bystanders—all working in various ways to educate, intervene and develop systems for stopping online hate.
The outer ring of the diagram lists possible reasons why someone might hold a specific ideology or participate in online hate; what might happen as a result of being targeted by online hate; the types of mitigation efforts and policies that can be used to fight online hate; and the areas within social platforms where we most often see online hate amplified.
By breaking down the higher-level classifications into discrete categories, actions, and outcomes, we can begin to conceive of the breadth of each of these contexts, see where there are overlaps between areas (the media as a perpetrator, mitigator and amplifier of hate) and strategize solutions based on causalities and other relationships. We also create a shared vocabulary and start to understand the nuances between the contexts.
The Relationships Between Actors diagram (pdf) looks at the problem from a different lens, examining the deeper relationships between the actors, specific places and opportunities to intervene and make changes in the system. Read from left to right, it walks us through who posts online hate (inputs), where they post and what happens when they participate in hate against their targets (outputs). What are the relationships between the parts, what paths connect them? Where are the perpetrators, and where are the mitigators? It also includes an option to follow along the path of those who fight hate and where they intersect with the inputs and outputs. For example, a path through the diagram might trace how nonprofits train law enforcement which needs to recognize hate speech which is often racist hate speech that incites offline violence and terror. This is a very high-level visualization of the landscape and is intended to be an overview to prompt discussions and deeper dives into more granular details around each area.
The Corporate Value Chain diagram (pdf) depicts companies that provide infrastructure to online businesses. Who makes money from online hate and by what means? How is online hate supported through site visitors and advertisers? The complexity of the number and types of infrastructure companies is often hidden from the general public. When tech companies choose to not address hate on a platform they have built and managed, it becomes important for infrastructure companies to evaluate whether to provide services to that company. These companies have tended to stay neutral when considering online hate content beyond what is strictly prohibited by law, but hopefully through awareness and pressure, more of them can be compelled to reject supporting online businesses that profit off hate.
These diagrams are not exhaustive or all-inclusive. We reviewed these across multiple disciplines, both inside and outside of ADL, with subject matter experts across the technology industry and academia. We worked with information architects and user experience design experts to develop these visualizations and refine them through dozens of review sessions.