Configure Rocket.Chat with Post-Quantum Resistant TLS

Quantum computing poses a growing threat to traditional public-key cryptography algorithms such as RSA and ECC. A key risk is the "Store Now, Decrypt Later" (SNDL) attack model, where encrypted data is collected and stored today for decryption by future quantum computers.

Self-managed Rocket.Chat deployments can mitigate this risk at the TLS layer. By placing NGINX in front of Rocket.Chat as a reverse proxy and enabling NIST-approved post-quantum cryptography (PQC) at the proxy, your workspace gains hybrid post-quantum key exchange for client connections without any code changes to the Rocket.Chat application.

This guide assumes you already have a working self-managed Rocket.Chat workspace with Docker. It explains how to configure NGINX to use hybrid post-quantum key exchange for client connections to your Rocket.Chat workspace.

How post-quantum protection works in Rocket.Chat

Rocket.Chat can be deployed behind a reverse proxy, such as NGINX or Traefik. Post-quantum cryptography is implemented at the reverse proxy layer rather than within the Rocket.Chat application itself.

This guide focuses on implementing post-quantum-resistant TLS using NGINX for Rocket.Chat. In this deployment model, NGINX handles TLS termination and is responsible for negotiating secure connections with clients before forwarding traffic to Rocket.Chat.

During the TLS 1.3 handshake, the client and NGINX negotiate a key exchange mechanism that is used to establish a shared encryption key for the session. In a post-quantum deployment, NGINX is configured to use a hybrid key exchange group that combines traditional elliptic-curve cryptography (X25519) with the NIST-standardized ML-KEM-768 algorithm. The standardized name for this combined group is X25519MLKEM768.

This hybrid approach provides protection against both current and future cryptographic threats. Even if large-scale quantum computers become capable of breaking traditional public-key cryptography, the ML-KEM component helps protect the confidentiality of the TLS session. Because these cryptographic enhancements are applied at the reverse proxy layer, no changes to the Rocket.Chat application are required.

Prerequisites

Standard NGINX and OpenSSL binaries provided by most Linux distributions do not currently include post-quantum cryptography support. To enable hybrid post-quantum key exchange, you must compile a custom stack.

Before proceeding, ensure you have:

1. Ubuntu 22.04 LTS+ or Debian 12+ with root access.

   Some steps in this guide may vary depending on your distribution and version.

2. A Rocket.Chat workspace deployed with Docker and accessible on 127.0.0.1:3000. If you have not yet deployed Rocket.Chat, visit Deploy with Docker and Docker Compose.

3. A domain name that resolves to your server's public IP address.

4. A valid TLS certificate for your domain. Standard RSA or ECC certificates can still be used because post-quantum cryptography is applied during key exchange, not certificate validation.

5. The following build dependencies installed

   sudo apt-get install -y build-essential cmake ninja-build git libpcre3-dev zlib1g-dev libssl-dev

Step 1: Complile the post-quantum stack

To enable post-quantum cryptography, install the Open Quantum Safe (OQS) libraries that extend OpenSSL with support for quantum-resistant algorithms.

1. Build liboqs: liboqs is an open-source C library for quantum-safe cryptographic algorithms. Clone the repository and build the library:

   git clone -b main https://github.com/open-quantum-safe/liboqs.git
   cd liboqs
   mkdir build && cd build
   cmake -GNinja -DOQS_USE_OPENSSL=OFF ..
   ninja
   sudo ninja install
   cd ../..

2. Build oqs-provider for OpenSSL 3: The OQS provider acts as a plug-in for OpenSSL 3 to inject post-quantum algorithms. Clone and build the provider:

   git clone https://github.com/open-quantum-safe/oqs-provider.git
   cd oqs-provider
   cmake -DOPENSSL_ROOT_DIR=/usr -DCMAKE_BUILD_TYPE=Release -GNinja -S . -B _build
   cmake --build _build
   sudo cp _build/lib/oqsprovider.so /usr/lib/x86_64-linux-gnu/ossl-modules/
   cd ..

3. Configure OpenSSL to load OQS: After installing the OQS provider, configure OpenSSL to load it automatically.

   1. Create a backup of your OpenSSL configuration file:

      sudo cp /etc/ssl/openssl.cnf /etc/ssl/openssl.cnf.bak

   2. Open your OpenSSL configuration file:

      sudo nano /etc/ssl/openssl.cnf

   3. Add or update the following sections:

      [openssl_init]
      providers = provider_sect
      
      [provider_sect]
      default = default_sect
      oqsprovider = oqsprovider_sect
      
      [default_sect]
      activate = 1
      
      [oqsprovider_sect]
      activate = 1

   4. Verify the provider loads successfully:

      openssl list -providers

      You should see both default and oqsprovider listed.

Step 2: Compile NGINX with OpenSSL 3

Most Linux distributions do not currently ship NGINX builds with post-quantum cryptography support enabled. To use hybrid post-quantum key exchange, compile NGINX against your OpenSSL 3 installation.

1. Download and extract NGINX:

   wget https://nginx.org/download/nginx-1.25.4.tar.gz
   tar -xzvf nginx-1.25.4.tar.gz
   cd nginx-1.25.4

2. Configure and build NGINX:

   ./configure \
     --prefix=/etc/nginx \
     --sbin-path=/usr/sbin/nginx \
     --with-http_ssl_module \
     --with-http_v2_module \
     --with-stream_ssl_module
   make
   sudo make install

3. Verify that NGINX was installed successfully:

   nginx -v

Step 3: Configure post-quantum TLS 1.3 in NGINX

Once NGINX is installed and the OQS provider is active, configure NGINX to use TLS 1.3 and advertise a hybrid post-quantum key exchange group.

Hybrid key exchange is highly recommended because it combines traditional cryptography with post-quantum cryptography (e.g., X25519 combined with ML-KEM-768). This ensures that the connection remains at least as secure as current standards, even if a vulnerability is discovered in the new PQC algorithm.

1. Add an include directive to the main NGINX configuration so Rocket.Chat config can be stored in a separate file. Open the NGINX configuration:

   sudo nano /etc/nginx/conf/nginx.conf

   In the http {} block, add the following directive:

   include /etc/nginx/conf.d/*.conf;

   Save the file and exit the editor. No other changes to nginx.conf are required.

2. Create or open the Rocket.Chat NGINX configuration:

   sudo mkdir /etc/nginx/conf.d
   sudo nano /etc/nginx/conf.d/rocketchat.conf

3. Add the following configuration, replacing <yourdomain.com> with your domain:

   server {
       listen 80;
       server_name <yourdomain.com>;
       return 301 https://$host$request_uri;
   }
   
   server {
       listen 443 ssl; 
       http2 on;
       server_name <yourdomain.com>;
   
       # Standard RSA or ECC certificate
       ssl_certificate     /etc/letsencrypt/live/<yourdomain.com>/fullchain.pem;
       ssl_certificate_key /etc/letsencrypt/live/<yourdomain.com>/privkey.pem;
   
       # Enforce TLS 1.3 only
       ssl_protocols TLSv1.3;
   
       # Hybrid post-quantum key exchange: X25519 + ML-KEM-768
       ssl_ecdh_curve X25519MLKEM768;
   
       # Security headers
       add_header Strict-Transport-Security "max-age=63072000" always;
       add_header X-Frame-Options SAMEORIGIN;
       add_header X-Content-Type-Options nosniff;
   
       # Proxy to Rocket.Chat
       location / {
           proxy_pass http://127.0.0.1:3000;
           proxy_http_version 1.1;
   
           # Required for Rocket.Chat WebSocket connections
           proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
   
           # Forward client information
           proxy_set_header Host               $http_host;
           proxy_set_header X-Real-IP          $remote_addr;
           proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;
           proxy_set_header X-Forwarded-Proto  https;
           proxy_set_header X-Forwarded-Host   $http_host;
       }
   }

4. Test and start the configuration:

   sudo nginx -t
   sudo nginx

Step 4: Update Rocket.Chat workspace URL

1. Open the .env file used by your Docker Compose deployment and verify that the ROOT_URL points to the HTTPS endpoint exposed by NGINX:

   ROOT_URL=https://<yourdomain.com>

   This ensures Rocket.Chat generates secure URLs for attachments, avatars, OAuth callbacks, and API endpoints.

2. Apply the change by restarting Rocket.Chat:

   docker compose -f compose.yml up -d

After the containers restart, verify that you can access your workspace through the HTTPS URL before proceeding.

Step 5: Verify the post-quantum connection

To confirm that the server is using the hybrid post-quantum key exchange group, run the following command from any machine with OpenSSL installed:

openssl s_client -connect <yourdomain.com>:443 -groups X25519MLKEM768

Replace <yourdomain.com> with your workspace domain.

The -groups X25519MLKEM768 option forces the client to use only this hybrid post-quantum key exchange group during the TLS handshake. If the server does not support it, the handshake fails with an error such as no shared cipher or handshake failure. If the connection succeeds, it confirms that the server supports X25519MLKEM768 and allows the TLS handshake to complete using TLS 1.3.

You should see output similar to:

Protocol  : TLSv1.3
Verify return code: 0 (ok)

Note that openssl s_client does not always explicitly display the negotiated key exchange group.

If the hybrid key exchange group is successfully negotiated, client connections are using post-quantum-resistant key exchange and gain protection against the Store Now, Decrypt Later threat model.

Client compatibility

For the post-quantum handshake to succeed, both the server (NGINX) and the client (browser/app) must support the hybrid key exchange group. The following clients are compatible:

If a client does not support the hybrid PQC group, the TLS handshake falls back to standard TLS 1.3 and the connection remains secure, but without post-quantum protection.

Post-quantum cryptography is an evolving field, and future standards, algorithms, and client implementations may require updates to your deployment. By deploying Rocket.Chat behind a post-quantum-enabled NGINX reverse proxy, you can begin protecting communications against future quantum threats while maintaining compatibility with existing infrastructure.

Related topics

• Deploy Rocket.Chat with Docker and Docker Compose

• NIST Post-Quantum Cryptography Standards

• Open Quantum Safe Project