Three Messenger screens showing a message removed with no context, the same message replaced with a tombstone, and a disabled chat in the chat list

Meta · Messenger · Community Chats

Community Chat Takedowns

An end-to-end design solution for a 0‑to‑1 product in Facebook Messenger, turning silent content removals into moments that explain themselves.

My roleLead product designer, end to end
Timeline6 months, 2022
TeamProduct, Engineering, Content, Ops
SurfaceMessenger × Facebook Groups
OutcomeMVP launched June 2022

Background

A new front door for communities

Community Chats is the Messenger product that lets communities connect over chat, audio and video with an audience that actually matters to them: the group they already belong to. It was brand new, and I joined as the lead designer to take enforcement from nothing to launch.

Community Chats directory for Miss Frizzle's AP Chemistry with a join prompt for Ice Cream PartyJoining a chat inside a Facebook Group
An active Community Chat thread with messages and reactionsA live community thread

Public surface, public standards

Community Chats integrate with Facebook Groups: chat threads are analogous to posts, and messages behave like comments. That inheritance carries a consequence. Community Chats use the open feed enforcement model and are treated as a public surface. Anything that violates Community Standards must be enforced upon and taken down.

A chat message reading I'll add in some nudes too
Message takedownsIf a message (text or media) is found violating through detection or human review, it can be removed.
A chat header with a violating chat name
Chat takedownsIf a chat contains violating messages, or its own metadata violates, the whole chat can be removed.
A Facebook Group header for Miss Frizzle's AP Chemistry
Group takedownsIf the violator was an admin, the Group takes a strike. Past the strike threshold, the entire Group comes down.
DetectedViolations can be caught by MMS, a scanning service that matches illegal images against a known bank of images.
ReportedMembers can also report content, which flags it for review by Operations representatives.

The problem

Content vanished, and no one knew why

Enforcement worked, but the experience around it did not. When a message or a chat was found to be violating, it was simply deleted, with no notice, no explanation and no trace. Participants lost the thread of a conversation they had been part of, the person who violated learned nothing, and Meta's Community Standards went entirely unspoken at the exact moment they were being enforced.

Design challenge

How might we help users understand that content has been removed due to a violation of Community Standards?

Before: removal with no context

Both takedown types behaved the same way: the content was there, and then it was not.

Message takedown · before
Messenger thread containing a violating messageViolating message detected
The same thread with the violating message silently deletedPost-action: message deleted
Chat takedown · before
Chat list containing a violating chatViolating chat detected
The same chat list with the violating chat silently deletedPost-action: chat deleted

Approach

Three goals to design against

InformTell users a message or chat was removed, so thread participants know something was previously there and keep contextual continuity.
Norm Community StandardsEducate both actors and bystanders that a set of Community Standards exists, and that we act when they are violated.
Educate respectfullyInform without shaming the actor or creating pile-ons, and clearly differentiate removals by Meta from removals by admins.

Six months, no blueprint

A 0-to-1 product meant no precedent to follow and a moving target underneath me. The work ran across four overlapping teams, so the process was as much about anticipating other roadmaps as it was about drawing screens.

Month 1Onboard & understandMeet the new team, onboard to the new product, understand the users and the problem space.
Month 2Ideate, diverge, convergeExplore ways to solve the problem, meet overlapping teams to understand constraints, share and iterate on feedback.
Month 3Propose & reviewPropose the recommended solution to cross-functional and leadership teams, and surface any gaps or improvements needed.
Month 4Iterate and buildIterate on partner and leadership feedback, propose the modified solution, hand off and collaborate with engineers.
Month 5Build and iterateClose collaboration with Engineering and Product on technical constraints and support through the build.
Month 6LaunchSupport edge cases, test, bug bash, then deliver and launch the MVP of the new Community Chats product.

Explorations

Early iterations for the message takedown treatment, ranging from a plain admin note to a full anonymized tombstone carrying the violation reason

Exploration showing removal communicated as plain admin textAdmin textInline, easy to miss
Exploration showing a quick promotion banner with an anonymized tombstoneQuick PromotionWith anonymized tombstone
Exploration showing a tombstone with no sender profileTombstoneWith no sender profile
Exploration showing an anonymized tombstone including the violation detailAnonymized tombstoneWith violation detail

Considerations

Audit before invention

We audited every existing tombstone and admin treatment already shipping across the app before deciding on a visual direction for removed messages. The end solution deliberately builds on top of patterns users had already seen, and optimises for visual consistency rather than a new language of its own.

Existing tombstone / admin treatments today

Admin Text treatment in a Messenger thread
Admin TextThe view CTA opens transaction details.
Failed loading Tombstone treatment in a Messenger thread
Failed loading TombstoneWith CTA below error
Update required Tombstone treatment in a Messenger thread
Update required TombstoneWith CTA below error
Attachment unavailable treatment in a Messenger thread
Attachment unavailableViewer has no permission to view or link doesn’t work
Unsent message Tombstone treatment in a Messenger thread
Unsent message TombstoneBasic tombstone
Tombstone with Mustache treatment in a Messenger thread
Tombstone with MustacheBasic tombstone with timestamp mustache
Message unavailable treatment in a Messenger thread
Message unavailableSeverely violating messages removed
Message no longer available Tombstone treatment in a Messenger thread
Message no longer available TombstoneSender account disabled, or no profile pic

Solution

Don't just remove it, tombstone it

A violating message is still removed. But in its place sits a tombstone: a quiet, anonymized marker that says something was here, it went against Community Standards, and it was acted on.

Before
Messenger thread containing a violating messageViolating message detected
The message silently deleted, leaving no traceDeleted, no trace
Proposed
Messenger thread containing a violating messageViolating message detected
The message replaced with a tombstone reading This message was removed because it went against Community StandardsPost-action:
message tombstoned
Close view of the message tombstone in a Messenger thread

Message tombstone

Tombstone treatmentReuses an existing Facebook pattern and maintains continuity of the thread. Replacing a deleted message with a tombstone reinforces the norms of Community Standards.
Anonymized profileThe violator's profile image is swapped for the default avatar, protecting their privacy and preventing pile-ons.
Voice standardsWhen the user takes action, the subject of the sentence is the user. When Meta takes action, the subject of the sentence is the content.

Chat takedowns, same principle

A removed chat leaves a trace in the list rather than disappearing from it, and tapping that trace opens the policy language behind the decision.

Chat list containing a violating chatViolating chat detected
The chat replaced with an inactive row reading This chat was disabled, tap for detailsPost-action: chat tombstoned
A dialog explaining the chat was disabled with policy snippet language and a Learn more buttonPolicy snippet on tap for details
The Transparency Center policy page for adult sexual exploitationLearn more in Transparency Center
Close view of a disabled chat tombstone in the chat list

Chat tombstone

Tombstone treatmentA removed chat is replaced with “This chat was disabled” and becomes inactive, promoting norms while letting users keep context of a thread they were active in.
Tombstoned contentTo future-proof the solution, we had to inform users why their chat was disabled in a way that still scales once the chat preview line disappears.
Policy detailsTapping a disabled chat opens a dialog with policy snippet language explaining the violation, helping norm Community Standards.

Results & learnings

MVP launch

Meta Newsroom article announcing new features for Facebook Groups, June 2022

The team worked tirelessly to launch the MVP for enforcement experiences in Community Chats, which shipped in June 2022.

The product is a continuing focus and investment for the company as a way to increase engagement within Facebook.

Proactive communication with overlapping teamsBecause Community Chats crossed multiple products and teams, this project was complex and challenging. It required me to stay tuned into every overlapping team's future vision and upcoming changes in order to anticipate the long-term scalability of my design solutions.
Agile collaboration while buildingLaunching a 0-to-1 product meant there was no blueprint or precedent to follow. As the lead designer, it was necessary to be agile and quick to adjust design solutions as new edge cases and engineering constraints arose.