Mod Assist App
    • Dark
      Light
    • PDF

    Mod Assist App

    • Dark
      Light
    • PDF

    Article summary

    The Mod Assist app is a content moderation app that automates the message reporting process using OpenAI API. This offers an efficient solution when manual reporting becomes unfeasible.

    It detects harmful content based on predefined rules, flags and reports these messages for moderator review, and enriches reports with contextual details like sender information, room location, and timestamps. This helps moderators make informed decisions. Learn how to install and use the app to successfully leverage its features for a safe workspace.

    Install Mod Assist app

    The app will only be available to workspaces running version 6.3.0 and higher.

    To install the Mod Assist app,

    • Go to Administration > Apps > Marketplace.

    • Search for the Mod Assist app and click it.

    • Click Install and accept the permissions needed to install it.

    Get an OpenAI API key

    The Mod Assist app requires API keys to authenticate your OpenAI account on Rocket.Chat and make requests.

    Your OpenAI account must have active billing with credits. For a more detailed guide on billing, please refer to OpenAI's billing questions.

    To get your OpenAI API key,

    • Log in to your OpenAI account and click on the dashboard via the menu in the navigation bar

    • In the dashboard, select API Keys from the menu on the left

    • If you don’t have any API keys, click the “+ Create new secret key” button to generate one

    • Copy the API key to your clipboard. You'll need it to configure the app in your workspace

    Configure the Mod Assist app

    To configure the Mod Assist app after installing it,

    • On the Mod Assist App Info screen, navigate to Settings.

    • Update the required fields:

      • AI Moderated Rooms: Select the rooms where the moderation app should automatically screen for inappropriate content.

      • Exclude Roles: Select and whitelist trusted roles whose messages can bypass moderation. It does not support custom roles.

      • Moderate accounts created in the past (number in days): This setting enables the app to moderate messages from accounts created within a specific number of days. For example, if you set a 30-day period, the app will moderate the message if the sender's account was created within the last 30 days. Messages from accounts created prior to this timeframe will bypass moderation.

      • OpenAI API Key: Paste the Open AI API key you copied earlier.

      • Moderation Categories: Select the types of inappropriate content you want the app to moderate. Natural language processing techniques are used to detect all types of inappropriate content such as profanity, toxicity, etc.

        • Toxicity: Any message that is harsh, disrespectful, negative, malicious, harmful, or hateful is toxic in nature.

        • Profanity: Any message where the language is bad, abusive, foul, vulgar, or obscene is deemed as profanity type.

        • Financial risk: Scam messages sent by fraudsters and imposters tricking users into divulging sensitive personal information or financial details, which can lead to identity theft, financial loss, or unauthorized access.

        • Flirtation: Messages that are indecent, intimate, unwanted, and inappropriate can be deemed as a flirtatious content type.

    • Click Save Changes.

    The API key used when configuring the app is applied to the entire workspace. Users in your workspace don't need to get their own API keys.

    Now that you've successfully installed and configured your Mod Assist app, proceed to explore Using the Mod Assist App.

    Note on private room access

    Due to privacy considerations, the app will not report messages or operate in private rooms by default. If you want to enable content moderation in private rooms, follow these steps:

    1. Add the modassist.bot app user to the private room that you want to moderate. Use the slash command /invite modassist.bot

    2. From the installed Mod Assist App, navigate to Settings > AI Moderated Rooms and select the private room

    This gives the app access to private rooms.


    Was this article helpful?

    ESC

    Eddy AI, facilitating knowledge discovery through conversational intelligence