In running my projects, I've moved hosting providers 4 times now and experienced no downtime. I don't think this is super remarkable but thought I'd share how I do it in case anyone is facing move #1 and feeling anxious.

Prerequisites

First off, as the title suggests this is focused on moving a MongoDB replica set between providers. This approach would also be handy if you're moving between geo regions within the same company.

If you're not running your MongoDB setup in a replica set, MongoDB has a good tutorial on getting started. Spinning up a replica set is a great idea to add some peace of mind (if not some performance benefits if secondary reads are your jam) as it adds redudancy to your architecture and makes tasks like migrating to a new provider super easy.

I usually run my replicasets in the same datacenter, so all mongod instances are only open on their private IPs. Part of this transition involves opening those up to their public IPs temporarily which is definitely less secure. I'd recommend enabling access control in your database, using a key file, and taking a look at MongoDB's security checklist before proceeding. (These are all good protections to keep in place after the move too!)

We'll be throwing your entire database across the network to your new provider, so this approach won't work well if your database is large and will blow through your existing inbound/outbound bandwidth limits!

The general idea here is to add a new replica set member in your new provider, get it up to sync, then slowly shut down your old servers. A database set of Theseus, as it were: same replica set and data in the end but completely different servers.

1. Boot up the new server

In your new provider location, boot up a new server that will run MongoDB for your new architecture. Get it ready to run Mongo and communicate with your existing database server(s):

1. Install MongoDB
2. Modify the Mongo conf file (/etc/mongod.conf for Linux machines) to...
3. Use your existing replica set
4. Use your security keyfile and other settins
5. Bind to the new server's external IP address at the very least
6. Upload your keyfile, if need be
7. Update the device's hosts file, if need be, so it knows which external IPs to use to reach your currently running MongoDB servers
8. This is usually /etc/hosts on Linux machines, and the structure is "IP_ADDRESS HOSTNAME"
9. Ensure your target MongoDB port is open on the new server (27017 by default)

And then start up MongoDB on your new server. It'll try reaching out to your existing servers to become a part of the replia set but will likely be unable to connect.

2. Modify your existing servers

Head into your existing servers and make the following modifications:

1. Modify the Mongo conf file to bind the server's external IP address
2. Modify your device's hosts file, if need be, so it knows which external IPs to use to reach your new server

Then, starting with your secondary MongoDB instance, restart mongo to start using the new settings. Move to your primary MongoDB instance and make it step down to secondary (issue a rs.stepDown() command), then restart it as well. See how easy these things are when you have a replica set?

3. Add the new replica set member

At this point, the old servers and new server should be able to communicate with one another over the port your Mongo instance uses. All that's left is to add the new server to the replica set.

Open up a new mongo shell connection in your primary replica set member and issue the following command:

rs.add( { host: "new_host_name:27017", priority: 0, votes: 0 } )

Be sure to update the port number if you don't run on the default 27017. Note the priority and votes args: this is because the new member will immediately count as a normal secondary instance when it comes to voting for a new primary server "even though it cannot serve reads nor become primary because its data is not yet consistent." We'll update this later!

You should then be able to run rs.status() and see the new server listed. It may be in a "STARTUP2" state which is fine - as long as you don't see any connection errors we should be in business. You'll likely see a line showing which current mongod instance the new server is syncing from too.

4. And now we wait

At this point all your database is sending all data to the new server to get it up to speed with the other replica set members. Depending on how large your database is this will take minutes to hours - I usually let it chug along for a day or two to be safe.

The new server will remain in a "STARTUP2" state as long as its doing the initial data sync. It then moves to a "RECOVERING" state and finally to "SECONDARY" when the new member has "enough data to guarantee a consistent view of the data for client reads."

5. Final steps

Once the new server is in a "SECONDARY" state, you can update your replica set config to give the new member full voting rights and a normal priority. This new server will likely be a primary very soon after all!

MongoDB has a good tutorial on doing this, but essentially in the mongo shell you'll want to grab the current replica set config with rs.conf(), edit the target member's priority and votes values, then apply the modified config with rs.reconfig(cfg).

At this point your new server is a fully-functional member of your replica set. If you have a new web server or other client that will interact with your database within the new hosting provider, you should be able to connect to the new database server and read data as normal (this may be slow if your setup is configured to read only from primaries and the primary is still back in the old hosting provider).

Once you've confirmed everything is working as you would expect, you can make the new server the replia set primary by issuing rs.stepDown() commands in your existing primary until the new server is elected as the new primary. All existing connections to your database should continue working -- if a little slowly -- as its now piping database traffic between hosting providers.

If you're running a web server, you should now be able to update DNS entries or take whatever other steps to shift web traffic to your new hosting infastructure.

6. Remove old members

You now have things running completely in your new provider! As a bonus, your existing infastructure is keeping up to date with new database changes from the new infastructure so if anything goes wrong or needs to be rolled back, you can transition back to your old provider pretty quickly and without data loss.

I like to keep things running in this dual-provider setup (all traffic going to new provider, databases in old provider still running and keeping in sync) for a couple days at least just to be safe. Nothing worse than realizing you forgot a script or some important process or file on a server that was deleted too soon!

You should next spin up new replica set members in your new provider, mimicing your old replica set to ensure you'll have the same safe redudancy when you shut off the older instances.

Once your new replia set is ready and you're comfortable the new hosting setup is working as expected, you can remove the old servers from your replicaset by issuing this command from the new primary:

rs.remove("old_member_hostname:27017")

Checking rs.status() afterwards should confirm the old member is no longer part of the set. You can then safely stop and decomission the old server.

When all servers from the old hosting provider are removed from the replica set, it's a good idea to update your new servers' mongo conf files to bind only to private / internal IP addresses if possible.

Celebrate!

And that's about it! You've shifted your data and mongo infastructure over to your new provider without any down time and with data redudancy the whole time.