It’s no secret that the internet has its fair share of users who put others down, share unhelpful comments, and in some cases, are outright abusive. Google subsidiary Jigsaw on Monday released new tools to further address negative online dialogue.
Jigsaw’s Perspective API, which can be integrated into existing comment systems, websites, apps, and other platforms, uses AI to sniff out toxic commentary and elevate comments from users that are helpful, positive, and generally add to the conversation. The company has now improved its API by adding “experimental bridging attributes.” These include a host of new evaluation points, including whether a comment has nuance or shows curiosity. The model can also analyze comments to determine if they tell a personal story, try reasoning to understand something, or “show deference or appreciation to others.”
Also: Facebook’s approach to content moderation slammed by EU commissioners
When Perspective ingests a comment, the tool evaluates the content across a range of attributes, including its toxicity level, whether it’s insulting, whether the comment has profanity, whether it’s sexually explicit, and now, if it elicits curiosity or has nuance, among others. Perspective then assigns a score to each of those attributes to determine whether a person would find a comment toxic, profane, or abusive.
“For example, a comment like ‘You are an idiot’ may receive a probability score of 0.8 for attribute toxicity, indicating that 8 out of 10 people would perceive that comment as toxic,” Jigsaw wrote.
Once the comment has been assigned a score, users of the API have to determine what to do with it. Jigsaw envisions a scenario in which moderators elevate or downgrade comments based on their scores. Jigsaw’s Perspective API also lets users create score thresholds to automatically filter comments that the AI deems abusive.
Websites, including social media platforms, often devolve into cesspools of users engaging in ugly debates, profane-laced commentary, and more. While many services have tried for years to address the issue with tools like user upvoting, filters, or even turning comments off, there’s been little progress in reducing online harassment.
Whether Jigsaw’s new tools can help make progress in this area is unclear. While Jigsaw’s researchers have long touted their technology, trouble persists in online discourse. What’s more, Jigsaw acknowledges in its documentation that its tool still lacks the ability to understand context. While Perspective can objectively look at a comment to determine if it’s abusive, in many cases, context matters more.
“In the future, we hope to use this to improve the analysis of the comment,” Jigsaw wrote. “Currently, our attributes are not making use of context, so sending context won’t change an attribute’s scores.”
The new attributes added to the Perspective API are available now to developers.
