Mongodb atlas migration

By using our site, you acknowledge that you have read and understand our Cookie PolicyPrivacy Policyand our Terms of Service.

The dark mode beta is finally here. Change your preferences any time. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

I've recently had a lot of weird usage between my MongoDB instance hosted on Atlas and my AWS instance hosting the application that uses the db. Since then i've tried to find tutorials on how to migrate my data from Atlas to my own setup, but haven't found anything but tutorials to migrate from something else to Atlas which is obv not what i want.

Is there some standard way im just not seeing, being fairly inexperienced with the low level MongoDB stuff, or are they actively keeping this quiet? I'd really appreciate some help because i'm hosting a hobby project and the most costs are created from network costs for moving data between my AWS instance and Atlas.

In addition to the hourly hosting cost, MongoDB Atlas also passed on the cloud provider charges for data transfer traffic between cluster nodes. The data transfer includes: replica set monitoring, heartbeats, automation, and back-ups. Note that this tier has lower specifications and some limitations compared to the paid tier.

To answer your question, you can export data from MongoDB Atlas using mongodump. See also:. Learn more. Asked 1 year, 9 months ago. Active 1 year, 9 months ago. Viewed 2k times. Since then i've tried to find tutorials on how to migrate my data from Atlas to my own setup, but haven't found anything but tutorials to migrate from something else to Atlas which is obv not what i want Is there some standard way im just not seeing, being fairly inexperienced with the low level MongoDB stuff, or are they actively keeping this quiet?

Is it not as simple as running mongodump to get a dump file of the Atlas database, then running mongorestore to load that data into a DB you have setup on AWS? Although, if you are uncomfortable or unfamiliar with setting up an EC2 server and managing backups of your DB yourself, I would stick with Atlas or try MLab instead, since both of those run on EC2. Thanks, i honestly didnt know mongodump. The costs from Atlas, as ive found out, seem to be costs for their internal communication between clusters, back ups and stat retrieval.

Active Oldest Votes. Wan Bachtiar Wan Bachtiar To your pricing Standpoint, it seems they are just plain lying, because others have this happen to them too. I also already usw the free tier, which is why i really don't want to get billed for it. Thanks for the mongodump and Backup links, i will probably host it myself anyways.

SebastianDiLuzio, I've edited the answer to reflect the correct pricing including data transfers after more research. Sign up or log in Sign up using Google.

Sign up using Facebook.It works by connecting to your existing MongoDB database and synchronizing it with a cluster running in Atlas all while your application continues to function normally.

Once the data between the two clusters has been synchronized, you can simply update the database connection string in your application to cut over to your cluster in Atlas. Atlas supports the latest versions of MongoDB: 3. Update your MongoDB drivers and make any necessary code changes at the application level to ensure compatibility. If your deployment is currently a standalone instance, you must first convert it to a replica set. The migration process requires that authentication is enabled on your source cluster in AWS.

See Enable Auth for instructions on enabling authentication. You can verify that authentication is enabled on your source cluster using the mongo command:. The database user from your source cluster on AWS that you will use to perform the migration has the required MongoDB roles. The user must have the clusterMonitor and backup roles. To verify that the database user that you intend to use for migration has the appropriate roles, run the db.

Your target cluster must use M10 or larger instance nodes. For development or staging environments, deploy a cluster with M10 or M20 instance nodes. For production workloads, select M30 or larger instance nodes. On the Overview page of your new target cluster, click the ellipsis … button and select Migrate Data to this Cluster.

At the top of the Migrate Data to Cluster modal, Atlas displays the IP address ranges that must be accessible from your source cluster. The address ranges displayed depend on the location of your target cluster and can change, so verify that you enter the address ranges as displayed in the modal.

To whitelist new IP address ranges, either create a new Security Group, or modify your existing Security Group to permit inbound network access from the displayed IP address ranges. If you create a new Security Group, you must associate it with the EC2 instances running your source cluster. On the Migrate Data to Cluster modal, enter the hostname and port number of the primary node in your source AWS source cluster that Atlas will use to perform the data migration.

The address must be resolvable over the public internet, so do not use the private IP address of the node. A countdown timer in a progress bar indicates how much time remains before your target cluster is ready to migrate data from your source cluster.

Wait until the countdown timer and the Prepare to Cutover button are green before proceeding to the next step. When Atlas detects that the source and destination clusters are nearly in sync, it starts an extendable 72 hour timer to begin the cutover procedure.

If the 72 hour period passes, Atlas stops synchronizing with the source cluster. If you have any questions regarding migration beyond what is covered in this documentation, or if you encounter an error during migration, please see the Atlas documentation on requesting support. Your current deployment is a MongoDB replica set or sharded cluster.

Optional Enabled authentication on your source deployment.See replica set live migration documentation for complete prerequisites and instructions. To migrate from a MongoDB sharded cluster to an Atlas sharded cluster. Atlas global clusters are not valid destinations for Live Migration. See sharded cluster live migration documentation for complete prerequisites and instructions.

See the tutorial Migrate with mongomirror. See the tutorial Seed with mongorestore. See the tutorial Load File with mongoimport. You can also restore from an Atlas cluster backup data to another Atlas cluster.

If have a source cluster with authentication and you wish to use an import strategy which includes using mongorestore with the --oplogReplay option, you must delete the admin directory from the dump directory created by mongodump. The admin directory contains database user information which you cannot add to an Atlas cluster with mongorestore. Use Live Migrate. You must provide the hostname of the primary to the Live Migrate tool.

If the cluster is running a MongoDB version earlier than 2. If the cluster runs with authentication, you must have credentials that provide read access to the primary and its oplog.

See Pre-Migration Validation for the full list of migration requirements. If your application can tolerate a short period of downtime, use Live Migrate. The source cluster must run MongoDB 3. If the sharded cluster is not running MongoDB 3.

If the source cluster enforces authentication, specify a user to Atlas that exists on every shard and the config server replica set. The user must have permission to:. If near-continuous uptime is a requirement, please contact MongoDB. From the Atlas UI, click Support. Fill out a Support ticket, noting your uptime requirements and cluster configuration in the More details text entry.

If some downtime is OK and your sharded cluster does not run a supported MongoDB version, you may use mongodump and mongorestore. You must enable the oplog proxy. You must specify the hostname and credentials for the oplog proxy when connecting from Live Migrate.

Convert the standalone MongoDB node to a single-node replica set and then use the Live Migration service. If the standalone is running a MongoDB version earlier than 2.

If you are running with authentication enabled, you must have credentials that provide read access to the primary and its oplog. If you cannot convert the standalone to a replica set, use mongodump and mongorestore. See Seed with mongorestore. Use mongodump and mongorestore. If you are running with authentication enabled, you must have credentials that provide read access to the primary.Before the Live Migration process begins, Atlas performs a validation check to ensure that all the necessary form fields and parameters are functional and correct.

If any parameters are invalid, Atlas returns an error and Live Migration does not proceed. Listed below are some common Live Migration validation errors and suggestions for what to check if you encounter them. Ensure that the correct subnet ranges are whitelisted on the source cluster. The four required subnet ranges can be found in the Live Migration modal window. Confirm that the hostname provided is resolvable to a public IP address.

At a command prompt, use one of the following commands:. Confirm your credentials in the mongo shell with the following commands:. Renaming a collection on the source cluster during Live migration may trigger this error. At a command prompt, use one of the following commands: copy. If the source is a standalone instance, convert it to a replica set before proceeding with the migration. Confirm that the source cluster has a readable oplog.

On some hosted services, such as Compose. If you still cannot access the oplog, use mongorestore instead to import your data into Atlas. Could not determine if —host is a replica set: error connecting to db server: no reachable servers Ensure that every IP address the Live Migration service requires is whitelisted for your source cluster. Error applying oplog entries during initial sync: renameCollection command encountered during initial sync. Please restart mongomirror.

Ensure that no users or applications rename any collections while the Live Migration is taking place. Unsupported index error Certain types and configurations of indexes which were allowable in earlier versions of MongoDB are no longer supported in more recent versions. Check the release notes for the MongoDB version on your destination cluster for possible conflicts. If necessary, drop any indexes which cause errors and rebuild them after migration is complete.

No IP address was found for the given hostname. Confirm that the given hostname is correct and publicly accessible. Confirm that the Is SSL enabled? If you are not using SSL: Check your connection string and confirm that the ssl query parameter is not present.

MongoDB cluster migration with zero downtime

If --ssl is part of your connection string, your cluster requires an SSL connection. Confirm your credentials in the mongo shell with the following commands: copy use admin db.

In order to perform the Live Migration procedure, the MongoDB user must have sufficient system privileges.

How to migrate a Mongo database to Cosmos DB - One Dev Question: Laurent Bugnion

See Source Cluster Security for details. Different Atlas service tiers have different amounts of disk space available. Ensure that your Atlas cluster has enough disk space for all the data on your source cluster. See Create a New Cluster for details on cluster sizing. Your source deployment must be a MongoDB replica set. If your source deployment is currently a standalone node, convert it to a single-node replica set before performing Live Migration.

Confirm that your CA file is complete and correctly pasted into the Live Migration modal window. If the source is a replica set, make sure you have read access on the local database.

Migrating to MongoDB Atlas

Could not determine if —host is a replica set: error connecting to db server: no reachable servers. Ensure that every IP address the Live Migration service requires is whitelisted for your source cluster.

Certain types and configurations of indexes which were allowable in earlier versions of MongoDB are no longer supported in more recent versions.Sign In. Already have an account? Sign in. Learn more. Atlas Fully managed cloud database. Pricing Simple, flexible, pay as you go. Atlas Search Simple, integrated search capabilities beta. Stitch Serverless platform. Charts Native visualization for MongoDB data. Community Server A free and open document database. Enterprise Server Advanced features and security.

Connectors Easy integrations to your data estate. What is MongoDB? Easy integrations to your data estate. University Free online courses from MongoDB. Blog Updates, tutorials, people. Developer Hub Developer best practices, trends, insights. Resources Webinars, white papers, datasheets, and more.

Training Instructor-led sessions on your schedule. Events Worldwide community events. Customers Who uses MongoDB. Consulting Accelerate success with MongoDB. Partners Find or become a partner. Cloud Atlas, Stitch, and more. Server The database.

mongodb atlas migration

Drivers Language APIs. Tools Compass, Charts, Connectors, and more. How to Guides Get started in minutes. Sign In Try Free. Contact Sign In Try Free. MongoDB Atlas is the global cloud database service for modern applications. Best-in-class automation and proven practices guarantee availability, scalability, and compliance with the most demanding data security and privacy standards.It works by connecting to your existing MongoDB database and synchronizing it with a cluster running in Atlas all while your application continues to function normally.

Migrating Data to MongoDB Atlas

Once the data between the two clusters has been synchronized, you can simply update the database connection string in your application to cut over to your cluster in Atlas. Atlas supports the latest versions of MongoDB: 3. Update your MongoDB drivers and make any necessary code changes at the application level to ensure compatibility.

If your deployment is currently a standalone instance, you must first convert it to a replica set. Live migration of data from sharded clusters is not supported. The migration process requires that authentication is enabled on your source cluster. See Enable Auth for instructions on enabling authentication. You can verify that authentication is enabled on your source cluster using the mongo command:.

The database user from your source cluster that you will use to perform the migration has the required MongoDB roles. The user must have the clusterMonitor and backup roles. To verify that the database user that you intend to use for migration has the appropriate roles, run the db. Log in to your Compose account and navigate to the deployment you want to migrate to Atlas.

mongodb atlas migration

It will be helpful during the migration process to keep one browser window open on your Compose deployment console and one window open on your Atlas console. To perform the migration process, you need a database user with permission to read the oplog on your admin database. Click the Add-ons link in the left-side navigation.

If you already have the Oplog Access add-on, click Configure to see the oplog user username and password. Navigate to your Atlas cluster. Click the ellipsis … button and select Migrate Data to this Cluster. On your Compose deployment dashboard, click the Security link in the left-side navigation. Add the IP address range which is listed at the top of the Atlas Migration process dialog window.

Copy them to the Atlas Live Migration dialog. Enter the username and password for oploguser in the Atlas Live Migration dialog window. Click the Validate button to check that all your form fields are valid and your clusters are ready for migration.

A countdown timer in a progress bar indicates how much time remains before your target cluster is ready to migrate data from your source cluster. Wait until the countdown timer and the Prepare to Cutover button are green before proceeding to the next step.

When Atlas detects that the source and destination clusters are nearly in sync, it starts an extendable 72 hour timer to begin the cutover procedure. If the 72 hour period passes, Atlas stops synchronizing with the source cluster. If you have any questions regarding migration beyond what is covered in this documentation, or if you encounter an error during migration, please see the Atlas documentation on requesting support. You created an Atlas cluster, migrated data from your old Compose cluster, and updated your applications to use the new Atlas Cluster.

Congratulations and welcome to MongoDB Atlas! Your current deployment is a MongoDB replica set or sharded cluster. Optional Enabled authentication on your source deployment. You can verify that authentication is enabled on your source cluster using the mongo command: copy. Note It will be helpful during the migration process to keep one browser window open on your Compose deployment console and one window open on your Atlas console.

Note Your Atlas migration IP address ranges may be different from those shown here. Click Start Migration. Click Prepare to Cutover.

mongodb atlas migration

Perform the cutover. Once you are prepared to cut your applications over to the destination Atlas cluster, click Prepare to Cutover.This page provides a list of the prerequisites for migrating to Atlas and a step-by-step guide to the migration process. Ensure that all recurring query patterns are well-indexed and that your deployment is running healthy on mLab:. If you will be using Atlas on behalf of a company, note that you will only need a single Atlas organization.

If you have multiple mLab accounts that belong to the same company, note that you can first connect the target Atlas organization to one source mLab account. Then at any point you can disconnect and then connect the same target Atlas organization to a different source mLab account. There are no restrictions to the number of times you can disconnect and connect to different source mLab accounts.

This view makes it easier to invite your mLab account users to your Atlas organization and to migrate your mLab deployment s to Atlas. In order to create a for-pay Atlas cluster M2 or aboveyou will either need to configure a payment method or apply prepaid credits from an annual contract.

Unlike with mLab, a given email address and username on Atlas can be associated with many Atlas organizations. After reviewing the migration prerequisites detailed above and completing the pre-migration setup stepsperform the following steps to migrate a specific mLab deployment. The steps in the migration wizard are different depending on whether you are migrating to the Atlas shared tier M0, M2, or M5 or the Atlas dedicated tier M10 or above. This is because there are two different migration processes.

Below is an example of what the migration wizard looks like. Note that the order and tasks will vary depending on the characteristics of the mLab deployment being migrated. By default Atlas organizations include only a Basic Support plan which does not include database support or a response time SLA. Unlike on mLab, in order to get support for your database you need to purchase an Atlas support plan separately. You will not be able to open a ticket with Atlas Support unless you have a support plan.

When you migrate from mLab to any for-pay Atlas cluster, you will be presented with the option to select a support plan. As such, when you migrate your source mLab deployment to Atlas, the migration tool will resent you with an option to select a support plan. As part of the migration process many customers will be offered an Atlas support plan at a significant discount. However, if you change your support plan or change the way you pay Atlas e.

If you are in development or are running a non-critical application, the Atlas Developer plan is a great choice. This plan has been designed to provide in-depth technical support but with slower response times. This plan has been designed to provide high-touch, in-depth technical support for advanced issues such as performance tuning, as well as rapid response times for emergencies, 24x7.

If you have questions please email support mlab. Atlas performs maintenance periodically. To ensure that maintenance on Atlas is seamless for your application s :. At that point you have the option to begin the maintenance immediately or defer the maintenance for one week.

You can defer a single project maintenance event up to two times. All nodes in the Atlas cluster could be restarted over a short time window during maintenance. Also some urgent maintenance activities e. Unlike with mLab, different Atlas clusters can share the same database users and network configuration.

Specifically, be aware that Atlas clusters within a given Atlas project share the same database users and whitelisted IP addresses. To access the Atlas cluster, you must authenticate using a MongoDB database user that has access to the desired database s on your Atlas cluster.

On Atlas, database users are separate from Atlas users just as on mLab, database users are separate from mLab users. View Atlas documentation on configuring MongoDB users.

Database users for Atlas clusters cannot be managed via database commands sent to the database directly e. Unlike with mLab, different Atlas clusters can share the same network configuration and database users.


0 thoughts on “Mongodb atlas migration

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>