MongoDB MMAP to WiredTiger Migration

Migrate your existing MMAP based MongoDB instance into a WiredTiger one without with as little downtime as possible.
Starting with the major release 4.X.Y of Rocket.Chat, MongoDB has to be setup with a WiredTiger storage engine rather than the deprecated MMAP one. This is mandatory, if you plan to upgrade to one of the future Rocket.Chat versions and has to be prepared before initiating the application upgrade.
​This project/repository aims to help out people migrating their existing dockerized, MMAP based MongoDB installation into a WiredTiger one.
Requirements
Docker-based Rocket.Chat deployment
MongoDB instance with MMAP storage engine
Quick usage
1.Make sure you have met all requirements listed above.
2.Create a database dump to ensure a backup is in place in case anything goes south!
3.Stop running Rocket.Chat system.
4.Update docker-compose.yml to include the migration container/logic.
5.Start up containers, wait for the migration to be completed.
6.Once finished, Rocket.Chat should be up again - just running with a WiredTiger MongoDB storage engine now.
Detailed usage
Assumptions
Your existing docker-compose.yml file is located within /opt/rocketchat
Your "mongo" container has a /data/db volume mapped, that holds the existing (mmap) data files of MongoDB
Your "mongo" container has a /dump volume mapped, that is accessible from the host and can be used to store the backup/dump
Step-by-step guide
1.Create a database dump of your existing MongoDB, save it on the host to have a backup just in case:
 cd /opt/rocketchat
 docker-compose exec mongo mongodump --archive=/dump/mmap --gzip
 cp /data/dump/mmap ~/mongo-mmap-dump.gzip
2.Stop your existing Rocket.Chat system including all its services (especially MongoDB).
 docker-compose stop
3.​Download the repository or clone it using git and extract it to /opt/rocketchat-migration:
 git clone https://github.com/RocketChat/docker-mmap-to-wiredtiger-migration /opt/rocketchat-migration
4.Copy the docker/ folder that holds the Dockerfile of the custom migrator image into your existing compose folder:
 cp -r /opt/rocketchat-migration/docker /opt/rocketchat/docker
5.Backup and rename your current docker-compose.yml file to docker-compose.mmap.yml:
 mv /opt/rocketchat/docker-compose.yml /opt/rocketchat/docker-compose.mmap.yml
6.Copy the new docker-compose.yml file into your compose folder:
 cp /opt/rocketchat-migration/docker-compose.yml /opt/rocketchat/docker-compose.yml
7.Apply any possible customizations that you had in your old docker-compose.mmap.yml file to the new one (volume, port mappings, service names, etc.)
8.Run diff and make sure it looks (more or less) like the output below when you compare your old and new docker-compose.yml files:
 diff /opt/rocketchat/docker-compose.mmap.yml /opt/rocketchat/docker-compose.yml
 8c8
 <         "for i in `seq 1 30`; do
 ---
 >         "for (( ; ; )); do
 11c11
 <           echo \"Tried $$i times. Waiting 5 secs...\";
 ---
 >           echo \"Could not start Rocket.Chat. Waiting 5 secs...\";
 37c37,45
 <     command: mongod --smallfiles --oplogSize 128 --replSet rs0 --storageEngine=mmapv1
 ---
 >     command: >
 >       bash -c
 >         "while [ ! -f /data/db/WiredTiger ]; do
 >           echo \"wiredTiger migration hasn't started yet. Waiting 30 secs...\";
 >           sleep 30;
 >         done;
 >         docker-entrypoint.sh mongod --oplogSize 128 --replSet rs0 --storageEngine=wiredTiger;"
 >     depends_on:
 >       - migrator
 40a49,53
 >   migrator:
 >     build: ./docker/
 >     volumes:
 >       - ./data/db:/data/db
 >
 45c58
 <         "for i in `seq 1 30`; do
 ---
 >         "for (( ; ; )); do
 51c64
 <           echo \"Tried $$i times. Waiting 5 secs...\";
 ---
 >           echo \"Could not reach MongoDB. Waiting 5 secs ...\";
All changes above, summarized:
for loops slightly adjusted for both "rocketchat" and "mongo-init-replica" to run them endlessly
Adjusted command property for "mongo" service:
while loop to check for initiated WiredTiger migration (in $MONGO_DATA_DIR/WiredTiger)
Use custom Entrypoint that applies the migration
Use --storageEngine=wiredTiger switch instead of --storageEngine=mmapv1
Added "migrator" service
9.Build the "migrator" image and start up the containers again:
 docker-compose up --build -d
10.Wait for the migration to be completed - optionally check logs of "migrator" and "mongo" containers:
docker-compose logs -f migrator # once that one has completed the migration ...
docker-compose logs -f mongo # ... check if the mongo one already took over
Help and Support
If you encounter any problems during the migration or in case you have general feedback or improvement ideas, feel free to create an issue in the GitHub repository.
High Availability
Snaps
Last modified 5mo ago