In my first introductory rsync post, How To Synchronize Directories with Rsync, I've shown you the most basic approach to syncing two directories up. Today, I'd like to show you another useful thing you can do with rsync.
Just to remind you all, rsync is a remote synchronization tool. This means that its primary use lies in the filed of synchronizing files and directories between remote Unix systems, but I feel that it's really important for you to understand the basics before moving on to more advanced stuff.
The beauty of rsync is that its syntax and command line options are exactly the same for local and remote directories synchronizations, so it's not like you'll have to learn it all from scratch when I show you the real world usage of rsync in my next post.
rsync experiments setup
Just use the following commands to re-create your rsync playground for this post:
ubuntu$ rm -rf /tmp/dir1 /tmp/dir2 ubuntu$ mkdir /tmp/dir1 /tmp/dir2 ubuntu$ cd /tmp ubuntu$ echo "original file 1" > dir1/file1 ubuntu$ echo "original file 2" > dir1/file2 ubuntu$ echo "original file 3" > dir1/file3 ubuntu$ cp dir1/file1 dir2
It will probably be more convenient for you to just download the same commands in form of a script though, so here's the link: rsync-setup.sh
Tracking rsync progress
The larger the directories you're synchronizing with rsync, the longer it will take for the command to finish. Depending on the scale of your task, it can be minutes, long hours or even days before you get the synchronization complete. With this in mind, the importance of progress tracking with rsync should be obvious to you.
Here's how you make rsync report its progress: just use the –progress command line option (in addition to the command line I introduced you to last time):
ubuntu$ rsync -avz --stats --progress /tmp/dir1/ /tmp/dir2 building file list ... 4 files to consider file2 16 100% 0.00kB/s 0:00:00 (xfer#1, to-check=1/4) file3 16 100% 15.62kB/s 0:00:00 (xfer#2, to-check=0/4)
Number of files: 4 Number of files transferred: 2 Total file size: 48 bytes Total transferred file size: 32 bytes Literal data: 32 bytes Matched data: 0 bytes File list size: 59 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 193 Total bytes received: 64
sent 193 bytes received 64 bytes 514.00 bytes/sec total size is 48 speedup is 0.19
Why you should track the progress of rsyncs
The key differences are that you first get a regularly updated message about building a file list. When you run into hundreds of thousands of files, this becomes quite useful:
building file list ... 4 files to consider
Another way to track progress is to see the transfer progress for each of the involved files. For thousands tiny files it won't be all that impressive a feature, but if you're transferring huge files between remote systems, a dynamic transfer progress for each file will be exactly what you need:
file2 16 100% 0.00kB/s 0:00:00 (xfer#1, to-check=1/4) file3 16 100% 15.62kB/s 0:00:00 (xfer#2, to-check=0/4)
That's it, hope you like this new option. I promise to tell you more some other time!
