Mzinga moderators spend much of their shifts putting an end to flame wars, banning trolls, handling customer complaints, and keeping the peace.  As Mzinga’s Director of Moderation Services, I ensure that the team works in an environment that encourages and practices civil interaction as well.

To produce harmonious workplace conditions, the consulting firm Civility Partners has established the following guidelines for teams, whether they work together in an office or virtually.  Teams should especially avoid:

—  Aggressive Communication (includes insults or offensive remarks, angry outbursts, avoidance,  offensive written communications, and blaming someone for issues not their fault or they have no control over)

— Humiliation (includes ridiculing or teasing, spreading gossip, taunting (in person or writing), publicly pointing out mistakes or mistakes that have been corrected, and snubbing for having a different interpretation of a company policy or management style)

— Manipulation of Work (includes subverting tasks associated with a person’s job responsibilities, unmanageable workloads and impossible deadlines, making general statements about poor performance without offering assistance to correct it, and leaving a person out of the correspondence and meeting loop)

Behaviors that contribute to workplace civility are respect, support, encouragement, politeness, openness, appreciation, trust, sensitivity, sincerity, having a positive attitude, taking pride in what you do, and being a good example.

Each company should have an established Company and Management Commitment to Civility that ensures their workers are free from negative, aggressive, and inappropriate behaviors and that the workplace will provide an atmosphere of respect, collaboration, openness, safety, and equality; where complaints about negative workplace behaviors are taken seriously and followed-through to resolution.  Every employee, from the CEO to the intern, is given a copy and a signed copy is part of their employee file.  Larger companies will have a training module as part of new employee orientation.

Online moderators keep their client’s sites free from risks that run from bad publicity to legal liability.  As a result, the burnout rate is high (see my blog entry from a couple of weeks ago about two Microsoft moderators who say they are permanently disabled from moderating disturbing images).  At Mzinga, the moderation team is able promote civil interaction because it is practiced in their workplace.

The post Online Moderators Keep it Civil, But What About Where They Work? appeared first on Online Moderation.

More than ten years ago, I complimented Facebook for encouraging its members to send in a report if they saw a post by a member or friend saying they were serious about harming themselves.  If a report was received, Facebook contacted the member with a message of concern, along with a list of resources for getting support.

Recently, Facebook announced updated resource tools, as well as the use of artificial technology (AI), to offer a more rapid assist to those who may be contemplating suicide.  In addition to an improved support system that is now available on Facebook Live, text and videos are analyzed for content indicating that the member may be considering suicide.

If the software reads triggering words and phrases that indicate a member is at risk, the Facebook Community Operations team is notified.  They will send a message of support and suggest ways the member can seek help if they need it.

Have we seen the last suicide on Facebook Live?  Probably not, but instead of using AI to combat online trolling behavior (i.e. Google’s Perspective), saving lives is a much more real-world and effective use of the technology.

The post Facebook Adds AI to Suicide Prevention Arsenal appeared first on Online Moderation.

Late last year, I commented on Google’s Jigsaw software, created to apply machine learning to detect and remove harassment and abusive content in areas where users interact online.  At the time, I said that no matter how much Jigsaw learned, it would never be smart enough to replace human moderators.

Recently, Google’s Counter Abuse Technology Team released Perspective, the newest Jigsaw tool.  It’s an API that allows users to tap Jigsaw’s library of millions of words and phrases and determine a message’s “toxicity.”  Perspective scans each message and produces a toxicity rating as a percent, based on what panels of users have thought of similar comments.  Each comment, for example, is rated as “8 percent similar to phrases people said were “toxic.””

While several sites, such as the New York Times, are giving it a try (you can try it as well at the Perspective Demo Site), there are many that believe that, while helpful, it will never be used more than as a first pass to flag content for subsequent human review.  Perspective advances the learning curve of using Artificial Intelligence to combat online trolling, but it also further illustrates the continuing value of human moderators.

The post Google’s Jigsaw Gets a New Perspective: Learning Curve Still a Hairpin appeared first on Online Moderation.

Twitter used to brag that it was “the free speech wing of the free speech party,” but recently the party was crashed by the company’s security team, who launched a protocol aimed to decrease users’ exposure to abusive content.  It’s both a step forward and a step back.  Forward because it’s effective.  Back because it’s a tactic that’s been around since the 90s.

If a user begins tweeting abusive messages, triggering the protocol will cause them to be seen only by those following them.  If the follower retweets the message, it won’t be visible to those who follow that user.  In the 90s, making posted content only visible to the poster was called “throwing the Bozo flag” or being sent to “banned camp.”  Several of my larger clients still successfully use that tactic.

At Twitter, the restriction is temporary, usually lasting 12 hours, and, unlike Bozo flags of the past, the affected user is notified with a “We’ve temporarily limited some of your account features” because of “potentially abusive behavior.”

While there is room for improvement (such as an appeals process for those who feel they weren’t breaking the rules), a sizable “free speech” backlash, and accusations that the company is making the changes to become a more lucrative to potential buyers, I applaud Twitter’s efforts to reduce trolling.  As Twitter’s vice president of engineering Ed Ho said, this is one step toward effectively reducing abuse and the campaign will continue until there is “a significant impact that people can feel.”  I’m feeling it.

The post Twitter Throws the Bozo Flag on Trolls appeared first on Online Moderation.

A new bill introduced in the Texas legislature seeks to criminalize cyber-bullying of children in educational settings.  The bill, called “David’s Law” (named after a 16-year-old victim of cyber-bullying who killed himself – there were no charges filed against those accused) would give school district officials more power to discipline, expel, and expose the identities of online harassment suspects.

The bill aims to protect students from communications that infringe upon their rights, but it does not define those rights or how they might be violated.   If a single email “infringes on the rights of the victim at school,” the sender could be disciplined.  If that email results in the recipient’s suicide, the sender could be expelled.

The worst provision, however, is the unmasking of the sender if they are accused of harassment.  The bill authorizes subpoenas to investigate injury claims before a lawsuit is filed.  As a result, if it is determined that no injury took place, the sender is still stamped with the stigma of a harasser or cyber-bully.

Bullying on social media is on the rise and a cause for concern, but anti-harassment policies must be very limited in scope (and rights and remedies narrowly defined) so they do not jeopardize the First Amendment rights of those who engage in lawful interaction.

The post Texas Anti-Harassment Legislation Threatens Lawful Online Interaction appeared first on Online Moderation.

Lego Life’s concerns for the safety of its users are evident in many ways: users under 13 must have their parents provide email permission, profiles are avatars that users create from a list of traits, usernames are randomly generated from a three-word sequence (i.e. ChairmanWilyDolphin), all user-generated content is premoderated, no photos are allowed that contain human faces, and most responses are either controlled by using emojis from a special keyboard or selected from a list of phrases.  Users are allowed to use their own words when responding to official Lego content.  A version for the web is in development.

Lego Life is a win-win for both the company and its customers.  The app increases brand loyalty and keeps kids safe. When they aren’t using it, they are presumably building new Lego creations.  That’s just fine with Lego Group’s senior director Rob Lowe, who says of kids who use the app, “One of its core purposes is to put their iPhone down and go do something else.”

The post Lego Life App Combats Trolls and Bullies appeared first on Online Moderation.

The two, members of the Online Safety Team, allege they could not transfer to another division for 18 months, they were not adequately trained, were not provided with the level of counseling they needed, and were denied workman’s compensation for medical leave time they took in order to take a break and reduce their PTSD-related symptoms.  The compensation requests were denied because OSHA said their conditions were not an occupational disease.

When contacted by McClatchy, Microsoft responded to the allegations by saying, in part, “The health and safety of our employees who do this difficult work is a top priority. Microsoft works with the input of our employees, mental health professionals and the latest research on robust wellness and resilience programs to ensure those who handle this material have the resources and support they need, including an individual wellness plan.”  The company also pointed to programs such as the Compassion Fatigue Referral Project and its mandatory support sessions for members of the Online Safety Team and the Digital Crimes Unit.

As Microsoft says in its response to the charges, “This work is difficult, but critically important to a safer and more trusted internet. The health and safety of our employees who do this difficult work is a top priority…”  Mzinga moderators know they may encounter offensive content at any time, and they are trained in the proper responses.  Here are a few of our best practices:

— Our moderators are warned about and can handle the content they may see

— Our moderators are trained to rapidly escalate offensive content to the proper law enforcement authorities

— As soon as a text, picture, or video tips the scales, it is removed immediately without having to look at it any further

— If they feel overwhelmed, they can trade shifts with another moderator, consult with the lead moderator or the director about how to handle their feelings, or they can work on a different project.

Removing offensive content is part of keeping the Internet safe.  It is Mzinga’s goal to keep our moderators safe as well.

The post Microsoft Online Moderators Allege Viewing Explicit Images Gave Them PTSD appeared first on Online Moderation.

In 1989, my master’s thesis, in part, argued that the Electronic Communications Privacy Act of 1986 should protect emails and other communications stored on a server in perpetuity, not just those stored for less than 180 days.  I also stressed that stored communications should not be accessed by any law enforcement organizations unless a criminal warrant (and not just a subpoena) is secured.  Unfortunately, though there has been progress, the recommendations have not been adopted.

The recently-introduced Email Privacy Act goes far in bringing these standards into effect, especially since so much data is now stored on YouTube, Dropbox, and Facebook. The Email Privacy Act, which requires a probable cause warrant for all digital communications held on cloud servers no matter how old they are, was passed by the House of Representatives by an overwhelming majority last year, but stalled in the Senate and was eventually withdrawn when amendments to weaken it were introduced.

While some groups, such as the Electronic Frontier Foundation, would like to have the government secure a warrant for obtaining geolocation information, I believe that this provision would thwart law enforcement from stopping trolls who post threats using mobile devices.  Obtaining a subpoena is a sufficient safeguard.

In 2017, it is time to bring the privacy of electronic communications into the present, offering protection that is in line with fundamental and current needs.

The post Papers in Your Desk Have More Protection than Those in Your Inbox appeared first on Online Moderation.

Over a year ago, I wrote about GitHub’s issues with bullying and discrimination, which came to a head when a female developer quit the collaborative coding hub – a victim of gender-based harassment by white male managers and co-workers.  The highly-publicized move eventually led to the resignation of GitHub’s CEO.

As a result, GitHub has made several major changes.  First, they hired Nicole Sanchez as the company’s VP of Social Impact.  Sanchez formalized GitHub’s organization (previously there had been no designated managers or job titles), made it easier for employee issues to be addressed, and announced that workplace diversity would be acknowledged and celebrated.

Sanchez also hired two transgender community managers: February Keeney as head of the Community and Safety team tasked with eliminating workplace harassment, and Coraline Ada Ehmke, a senior engineer and creator of the “Contributor Covenant,” a code of conduct that had been loosely adopted by several project teams.

Sanchez, Keeney, and Ehmke found, however, that their institutional changes weren’t welcome at all levels.  Several groups, advocates of free speech, resisted being told that the terms of the Contributor Covenant were being applied more widely.  They retaliated by using the software’s tagging feature to connect Ehmke with fake projects that had racist names, a tactic they had also used on the developer who had initially exposed the harassment in 2014.

One of the first tasks of the Community and Safety Team was to build “consent and intent” into the software.  Now, you cannot tag a coder on a project without their approval.  And late last year, they updated the Contributor Covenant to include guidelines for conduct that prohibited doxxing, bullying, and discrimination, as well as a wider range of moderation tools.

While there is still resistance from a few groups, Sanchez and her team have moved the company in the right direction: making the platform less prone to abuse, developing a policy that is explicit and embraces civility, diversity, and inclusion, and involving the community members in its enforcement.  As she said at a recent conference, ““Diversity is coming to your party despite my bad experiences at other parties. Inclusion is being glad I came.”

The post GitHub’s “Contributor Covenant” Makes Waves; Curbs Online Abuse appeared first on Online Moderation.

Riot Games, producer of the PC-based multiplayer League of Legends (LoL), has been selected as Inc. magazine’s Company of the Year.  One of the reasons they are the most popular PC game in North America and Europe is due to their success at handling abuse, through its Tribunal system of rule enforcement, which not only penalizes toxic players but rewards those who have a pattern of positive behavior.

The Riot Games Tribunal is made up of players with exemplary conduct who review and vote on reported abuse of the “Summoner’s Code” of Conduct.  If the conduct warrants it, a case is opened.  Tribunal members then vote on the appropriate action against the user, either to “Punish” them, to “Pardon” them, or to “Skip” the case.  When twenty votes are received, the case is closed and the member receives a detailed report on the decision.  Punishment ranges from a warning, to a one-day ban, to permanent expulsion.

The most positive aspects of the Tribunal system are that a player is notified within hours of its decision, and in the report, they are told what infraction triggered the opening of a case.  As reported by Christine Porath in her book “Mastering Civility,” after 100 million votes, verbal abuse is 40% lower and 91.6% of reported members never receive another violation report.

The takeaway for community managers and moderators is to empower your community members: allow members to report the abuse they witness, allow an empowered team of members (The Tribunal) to quickly vote on the violation report and let the member know the outcome, and let the member what, if anything, they did wrong.

Of course, this system only works on platforms with an extremely large member database.  Smaller ones still need a team of experienced moderators to vet abuse reports and take action against toxic community members.

The post Riot Games’ Tribunal System Reduces Abuse on Its Platform appeared first on Online Moderation.