Move Your Data to Oracle Cloud Infrastructure

After your configuration is complete, you can start the transfer. Depending on the amount of data and the connection speed, this transfer can take a very long time (days or weeks).

To monitor progress, we recommend starting the transfer in verbose mode from tmux, so you can run the process in the background and watch its progress as needed.

Move the Data

Start the transfer in verbose mode from GNU Screen, so you can run the process in the background and watch its progress as needed.

Start a tmux session and give the session a name:
```
$ tmux new -s<session name>
 
```
For example:
```
$ tmux new -srclone
```
In the tmux terminal window, start Rclone in verbose mode. Replace <destination_bucket> as needed (a bucket will be created automatically if one doesn't already exist).
```
$ rclone --verbose --cache-workers 64 --transfers 64 --retries 32 copy $SOURCE oci:<destination_bucket>
```
While the transfer is ongoing, detach from tmux by pressing Ctrl+b d
To get back to tmux session to monitor the Rclone, enter:
```
$ tmux attach -d -t <session name>
 
```
For example:
```
$ tmux attach -d -t rclone
```
If you want to detach the tmux again, press Ctrl+b d.

Note:

The optimal number of cache-workers and transfers depends on your specific use case, available bandwidth, resources available on the workstation or server, and throttling limits applied by your cloud provider. In general, the higher the number of transfers and cache-workers, the faster the transfer will complete. But beyond a certain number, you might start getting errors. We recommend that you start with 64 concurrent transfers and cache-workers and adjust the number as necessary,

When the transfer is complete, verify that no errors occurred. If you see errors, try decreasing the number of cache-workers and transfers and try again.

Verify the Transfer

Independent of how data is transferred to Object Storage (whether from a local file system or another cloud provider, or by some other means), you can use rclone to verify whether the data copied to Object Storage is identical to its source. You can verify the data quickly or more thoroughly.

Use the following method to perform a relatively quick verification:

$ rclone --verbose --cache-workers 64 --transfers 64 --retries 32 check $SOURCE oci:<destination_bucket>

This method detects differences in the source and destination buckets by verifying file sizes and MD5 hashes. However, MD5 hashes can’t be used for objects that are uploaded by using multipart uploads. As a result, large objects are verified only by comparing their file sizes, which provides only a cursory verification.

To perform a more thorough verification, you can instruct rclone to download files and compare them:

$ rclone --verbose --cache-workers 64 --transfers 64 --retries 32 check --download $SOURCE oci:<destination_bucket>

Although this method performs a more thorough comparison, it requires downloading copies from both object stores.

Note:

When you perform a verification with downloads, depending on where your job is run, you might incur additional data transfer fees from your existing cloud provider, Oracle, or both.