MongoDB mmap to wiredTiger migration
Migrate your existing mmap based MongoDB instance into a wiredTiger one without with as little downtime as possible.
Starting with the major release 4.X.Y of Rocket.Chat, MongoDB has to be setup with a WiredTiger storage engine rather than the deprecated mmap one. This is mandatory, if you plan to upgrade to one of the future Rocket.Chat versions and has to be prepared before initiating the application upgrade.
This project/repository aims to help out people migrating their existing dockerized, mmap based MongoDB installation into a WiredTiger one.

Requirements

  • Docker-based Rocket.Chat deployment
  • MongoDB instance with mmap storage engine

Quick usage

  1. 1.
    Make sure you have met all requirements listed above.
  2. 2.
    Create a database dump to ensure a backup is in place in case anything goes south!
  3. 3.
    Stop running Rocket.Chat system.
  4. 4.
    Update docker-compose.yml to include the migration container/logic.
  5. 5.
    Start up containers, wait for the migration to be completed.
  6. 6.
    Once finished, Rocket.Chat should be up again - just running with a WiredTiger MongoDB storage engine now.

Detailed usage

Assumptions

  • Your existing docker-compose.yml file is located within /opt/rocketchat
  • Your "mongo" container has a /data/db volume mapped, that holds the existing (mmap) data files of MongoDB
  • Your "mongo" container has a /dump volume mapped, that is accessible from the host and can be used to store the backup/dump

Step-by-step guide

  1. 1.
    Create a database dump of your existing MongoDB, save it on the host to have a backup just in case:
    1
    cd /opt/rocketchat
    2
    docker-compose exec mongo mongodump --archive=/dump/mmap --gzip
    3
    cp /data/dump/mmap ~/mongo-mmap-dump.gzip
    Copied!
  2. 2.
    Stop your existing Rocket.Chat system including all its services (especially MongoDB).
    1
    docker-compose stop
    Copied!
  3. 3.
    Download the repository or clone it using git and extract it to /opt/rocketchat-migration:
    1
    git clone https://github.com/RocketChat/docker-mmap-to-wiredtiger-migration /opt/rocketchat-migration
    Copied!
  4. 4.
    Copy the docker/ folder that holds the Dockerfile of the custom migrator image into your existing compose folder:
    1
    cp -r /opt/rocketchat-migration/docker /opt/rocketchat/docker
    Copied!
  5. 5.
    Backup and rename your current docker-compose.yml file to docker-compose.mmap.yml:
    1
    mv /opt/rocketchat/docker-compose.yml /opt/rocketchat/docker-compose.mmap.yml
    Copied!
  6. 6.
    Copy the new docker-compose.yml file into your compose folder:
    1
    cp /opt/rocketchat-migration/docker-compose.yml /opt/rocketchat/docker-compose.yml
    Copied!
  7. 7.
    Apply any possible customizations that you had in your old docker-compose.mmap.yml file to the new one (volume, port mappings, service names, etc.)
  8. 8.
    Run diff and make sure it looks (more or less) like the output below when you compare your old and new docker-compose.yml files:
    1
    diff /opt/rocketchat/docker-compose.mmap.yml /opt/rocketchat/docker-compose.yml
    Copied!
    1
    8c8
    2
    < "for i in `seq 1 30`; do
    3
    ---
    4
    > "for (( ; ; )); do
    5
    11c11
    6
    < echo \"Tried $i times. Waiting 5 secs...\";
    7
    ---
    8
    > echo \"Could not start Rocket.Chat. Waiting 5 secs...\";
    9
    37c37,45
    10
    < command: mongod --smallfiles --oplogSize 128 --replSet rs0 --storageEngine=mmapv1
    11
    ---
    12
    > command: >
    13
    > bash -c
    14
    > "while [ ! -f /data/db/WiredTiger ]; do
    15
    > echo \"wiredTiger migration hasn't started yet. Waiting 30 secs...\";
    16
    > sleep 30;
    17
    > done;
    18
    > docker-entrypoint.sh mongod --oplogSize 128 --replSet rs0 --storageEngine=wiredTiger;"
    19
    > depends_on:
    20
    > - migrator
    21
    40a49,53
    22
    > migrator:
    23
    > build: ./docker/
    24
    > volumes:
    25
    > - ./data/db:/data/db
    26
    >
    27
    45c58
    28
    < "for i in `seq 1 30`; do
    29
    ---
    30
    > "for (( ; ; )); do
    31
    51c64
    32
    < echo \"Tried $i times. Waiting 5 secs...\";
    33
    ---
    34
    > echo \"Could not reach MongoDB. Waiting 5 secs ...\";
    Copied!
    All changes above, summarized:
    • for loops slightly adjusted for both "rocketchat" and "mongo-init-replica" to run them endlessly
    • Adjusted command property for "mongo" service:
      • while loop to check for initiated WiredTiger migration (in $MONGO_DATA_DIR/WiredTiger)
      • Use custom entrypoint that applies the migration
      • Use --storageEngine=wiredTiger switch instead of --storageEngine=mmapv1
    • Added "migrator" service
  9. 9.
    Build the "migrator" image and start up the containers again:
    1
    docker-compose up --build -d
    Copied!
  10. 10.
    Wait for the migration to be completed - optionally check logs of "migrator" and "mongo" containers:
    1
    docker-compose logs -f migrator # once that one has completed the migration ...
    2
    docker-compose logs -f mongo # ... check if the mongo one already took over
    Copied!

Help and support

If you encounter any problems during the migration or in case you have general feedback or improvement ideas, feel free to create an issue in the GitHub repository.
Last modified 1yr ago