MongoDB MMAP to WiredTiger Migration

Starting with the major release 4.X.Y of Rocket.Chat, MongoDB has to be setup with a WiredTiger storage engine rather than the deprecated MMAP one. This is mandatory, if you plan to upgrade to one of the future Rocket.Chat versions and has to be prepared before initiating the application upgrade.
​This project/repository aims to help out people migrating their existing dockerized, MMAP based MongoDB installation into a WiredTiger one.
Requirements
Docker-based Rocket.Chat deployment
MongoDB instance with MMAP storage engine
Quick usage
1.Make sure you have met all requirements listed above.
2.Create a database dump to ensure a backup is in place in case anything goes south!
3.Stop running Rocket.Chat system.
4.Update docker-compose.yml to include the migration container/logic.
5.Start up containers, wait for the migration to be completed.
6.Once finished, Rocket.Chat should be up again - just running with a WiredTiger MongoDB storage engine now.
Detailed usage
Assumptions
Your existing docker-compose.yml file is located within /opt/rocketchat
Your "mongo" container has a /data/db volume mapped, that holds the existing (mmap) data files of MongoDB
Your "mongo" container has a /dump volume mapped, that is accessible from the host and can be used to store the backup/dump
Step-by-step guide
1.Create a database dump of your existing MongoDB, save it on the host to have a backup just in case:
1
 cd /opt/rocketchat
2
 docker-compose exec mongo mongodump --archive=/dump/mmap --gzip
3
 cp /data/dump/mmap ~/mongo-mmap-dump.gzip
Copied!
2.Stop your existing Rocket.Chat system including all its services (especially MongoDB).
1
 docker-compose stop
Copied!
3.​Download the repository or clone it using git and extract it to /opt/rocketchat-migration:
1
 git clone https://github.com/RocketChat/docker-mmap-to-wiredtiger-migration /opt/rocketchat-migration
Copied!
4.Copy the docker/ folder that holds the Dockerfile of the custom migrator image into your existing compose folder:
1
 cp -r /opt/rocketchat-migration/docker /opt/rocketchat/docker
Copied!
5.Backup and rename your current docker-compose.yml file to docker-compose.mmap.yml:
1
 mv /opt/rocketchat/docker-compose.yml /opt/rocketchat/docker-compose.mmap.yml
Copied!
6.Copy the new docker-compose.yml file into your compose folder:
1
 cp /opt/rocketchat-migration/docker-compose.yml /opt/rocketchat/docker-compose.yml
Copied!
7.Apply any possible customizations that you had in your old docker-compose.mmap.yml file to the new one (volume, port mappings, service names, etc.)
8.Run diff and make sure it looks (more or less) like the output below when you compare your old and new docker-compose.yml files:
1
 diff /opt/rocketchat/docker-compose.mmap.yml /opt/rocketchat/docker-compose.yml
Copied!
1
 8c8
2
 <         "for i in `seq 1 30`; do
3
 ---
4
 >         "for (( ; ; )); do
5
 11c11
6
 <           echo \"Tried $i times. Waiting 5 secs...\";
7
 ---
8
 >           echo \"Could not start Rocket.Chat. Waiting 5 secs...\";
9
 37c37,45
10
 <     command: mongod --smallfiles --oplogSize 128 --replSet rs0 --storageEngine=mmapv1
11
 ---
12
 >     command: >
13
 >       bash -c
14
 >         "while [ ! -f /data/db/WiredTiger ]; do
15
 >           echo \"wiredTiger migration hasn't started yet. Waiting 30 secs...\";
16
 >           sleep 30;
17
 >         done;
18
 >         docker-entrypoint.sh mongod --oplogSize 128 --replSet rs0 --storageEngine=wiredTiger;"
19
 >     depends_on:
20
 >       - migrator
21
 40a49,53
22
 >   migrator:
23
 >     build: ./docker/
24
 >     volumes:
25
 >       - ./data/db:/data/db
26
 >
27
 45c58
28
 <         "for i in `seq 1 30`; do
29
 ---
30
 >         "for (( ; ; )); do
31
 51c64
32
 <           echo \"Tried $i times. Waiting 5 secs...\";
33
 ---
34
 >           echo \"Could not reach MongoDB. Waiting 5 secs ...\";
Copied!
All changes above, summarized:
for loops slightly adjusted for both "rocketchat" and "mongo-init-replica" to run them endlessly
Adjusted command property for "mongo" service:
while loop to check for initiated WiredTiger migration (in $MONGO_DATA_DIR/WiredTiger)
Use custom Entrypoint that applies the migration
Use --storageEngine=wiredTiger switch instead of --storageEngine=mmapv1
Added "migrator" service
9.Build the "migrator" image and start up the containers again:
1
 docker-compose up --build -d
Copied!
10.Wait for the migration to be completed - optionally check logs of "migrator" and "mongo" containers:
1
docker-compose logs -f migrator # once that one has completed the migration ...
2
docker-compose logs -f mongo # ... check if the mongo one already took over
Copied!
Help and Support
If you encounter any problems during the migration or in case you have general feedback or improvement ideas, feel free to create an issue in the GitHub repository.

High Availability

Snaps

Last modified 4d ago

Export as PDF

Copy link

Contents