|
| 1 | +# MongoDB Stream RS |
| 2 | + |
| 3 | +Mongodb-stream-rs is a tool that copies MongoDB databases or single collections from one replicaset or standalone to another MongoDB instance. This tool was created specifically to upload databases from a standalone MongoDB instance into an Atlas db, since mongomirror required a replicaset to function properly, and mongodump/mongorestore is too slow. |
| 4 | + |
| 5 | +### Installation |
| 6 | + |
| 7 | +Once rust has been [installed](https://www.rust-lang.org/tools/install), simply run: |
| 8 | +``` |
| 9 | +cargo install --git https://github.com/findelabs/monogdb-stream-rs.git |
| 10 | +``` |
| 11 | + |
| 12 | +### Technical |
| 13 | + |
| 14 | +This tool is written in rust and leverages the tokio runtime in order to send multiple collection to the destination database at once. By default mongodb-stream-rs will upload four collections in parellel. By default, uploads are transmitted in batches of 2000 docs, but this option can be changed with the `--bulk` flag. You can override this default with the `--nobulk` flag in order to have this tool upload one doc at a time. |
| 15 | + |
| 16 | +If only a database name is passed to the app, then this tool will upload all collections within the db. However, you can specify a single collection to upload with `--collection`. |
| 17 | + |
| 18 | +### Arguments |
| 19 | + |
| 20 | +``` |
| 21 | +USAGE: |
| 22 | + mongodb-stream-rs [FLAGS] [OPTIONS] --db <MONGODB_DB> --destination_uri <STREAM_DEST> --source_uri <STREAM_SOURCE> |
| 23 | +
|
| 24 | +FLAGS: |
| 25 | + -c, --continue Restart streaming at the newest document |
| 26 | + -h, --help Prints help information |
| 27 | + -n, --nobulk Do not upload docs in batches |
| 28 | + --validate Validate docs in destination |
| 29 | + -V, --version Prints version information |
| 30 | +
|
| 31 | +OPTIONS: |
| 32 | + -b, --bulk <STREAM_BULK> Bulk stream documents [env: STREAM_BULK=] |
| 33 | + -c, --collection <MONGODB_COLLECTION> MongoDB Collection [env: MONGODB_COLLECTION=] |
| 34 | + -d, --db <MONGODB_DB> MongoDB Database [env: MONGODB_DB=] |
| 35 | + --destination_uri <STREAM_DEST> Destination MongoDB URI [env: STREAM_DEST=] |
| 36 | + --source_uri <STREAM_SOURCE> Source MongoDB URI [env: STREAM_SOURCE=] |
| 37 | + -t, --threads <STREAM_THREADS> Concurrent collections to transfer [env: STREAM_THREADS=] |
| 38 | +``` |
| 39 | + |
| 40 | +### Future |
| 41 | + |
| 42 | +We plan to utilize watch(), once the mongodb rust driver supports opening a watch on a database. This will effectively remove the need for `--continue`, especially if updates are being done on the source db. |
0 commit comments