Rocket.Chat AI (RAI) is a privacy-first, enterprise AI framework that brings AI-powered search, summarization, and conversational capabilities to your workspace while keeping your data within your infrastructure.
RAI uses a Retrieval-Augmented Generation (RAG) pipeline to retrieve relevant information from your knowledge base and generate context-aware responses from large language models (LLMs). This enables users to search conversations, summarize discussions, and ask questions about organizational knowledge while enforcing Rocket.Chat's existing access controls.
This guide introduces the core concepts of RAI, including its architecture, applications, and how the RAG pipeline processes data securely.
Key features
Sovereign AI: Deploy AI models and supporting services within your own infrastructure instead of routing requests through third-party AI providers. This keeps proprietary and sensitive data under your control and reduces vendor lock-in.
Knowledge lifecycle management: RAI continuously versions, expires, and reindexes content to keep indexed knowledge current as your workspace data changes.
Access controls and guardrails: RAI enforces Rocket.Chat role-based access controls (RBAC) and data classification policies during retrieval. It also evaluates prompts for unsafe or unauthorized requests to help prevent sensitive data exposure.
Reliable grounded insights: Responses are generated from indexed workspace content and include references to the original messages or documents, allowing users to verify the source of the information.
Rocket.Chat AI apps
The Rocket.Chat AI pipeline exposes its capabilities to end users through two distinct applications available on the Rocket.Chat Marketplace:
Intelligent Search App: Intelligent Search uses semantic search to find relevant messages in your workspace rooms based on intent rather than exact keyword matches. For example, a search for “internet issues” can return discussions containing phrases such as “Wi-Fi” is down.
Rocket.Chat AI: Rocket.Chat AI provides an AI assistant for workspace conversations. Users can interact with the AI bot directly or mention it in a channel to ask questions about indexed organizational knowledge. The app can also summarize conversations and threads, highlighting key discussion points and action items.
Rocket.Chat AI architecture
Rocket.Chat AI apps do not access workspace data directly. Instead, workspace data passes through a pipeline that processes, indexes, and stores information for retrieval by these AI applications. The AI pipeline architecture consists of the following components:
1. Rocket.Chat AI pipeline server
The Rocket.Chat AI pipeline server is a Retrieval-Augmented Generation (RAG) server that runs in your infrastructure. It provides APIs for document processing, embedding generation, vector storage, retrieval, and AI agents. It indexes workspace content to power Rocket.Chat AI and Intelligent Search.
2. Mongo connector
The Mongo connector is the control plane and data synchronization service that connects your Rocket.Chat workspace database to the AI pipeline server. It handles historical data backfills and live synchronization, enabling the RAG pipeline to index workspace data for context-aware AI responses. Administrators can also control which rooms are indexed.
Once the Rocket.Chat AI pipeline server and Mongo connector are configured, Rocket.Chat AI applications can retrieve indexed workspace data. Rocket.Chat continues to enforce existing permissions and access controls, so users can access only the data they're authorized to view.