Phenquestions
The Rsync (remote sync) command is a Linux/Unix utility used to synchronize and copy files and directories either locally or remotely. Most Linux geeks use Rsync to mirror, backup or migrate data across folders, across disks and networks. One notable feature with the Rsync command is that it uses the “delta transfer algorithm.”
Delta Transfer algorithm works by updating the destination directory with the contents of the source destination. If a change or a new file is created on the source directory, only the particular change will be copied to the destination directory when you run the Rsync command. To synchronize files and directories between the local and a remote machine, Rsync makes use of SSH.
Installing Rsync in Linux
Rsync command comes pre-installed in most Linux operating systems. However, this might not be the case for some minimal installations. You can install Rsync by running the below commands in a terminal.
On CentOS & RHEL
yum install rsync -y
On Ubuntu and other Debian distributions
sudo apt install rsync -y
Rsync Basic syntax
rsync options source destination
Some of the standard options/parameters used with Rsync command:
-v: -verbose Verbose output
-r: copies data recursively
-z: compress file data
-h: Gives output in a Human Readable format
-a: archive files and directory while synchronizing
-progress Shows the progress of the Rsync tasks currently running.
You can see all the options available for the Rsync command using the “-help” option.
$ rsync --help
Top 20 Rsync Examples in Linux
With that detailed information in mind, let's get started with 20 useful Rsync example commands you can use with Linux systems. Our operating system of choice is Ubuntu 20.04 LTS; however, the syntax should also work for other Linux distributions with Rsync installed.
1. Copy/Sync files locally with -v (verbose) option
That is the most basic Rsync command, even for a newbie. In this example, we will copy files between the 'Original' directory on the Desktop to the 'Backup' directory in the '/etc.' folder. We will include the -v (Verbose) option so that rsync can provide information on what is going on.
rsync -v Original/
One thing to note with Rsync is that, if the destination directory doesn't exist, it will automatically create it. Let's see the example below where we want to copy files to the directory “Foss” that doesn't exist.
2. Sync/Copy files and directories recursively with -r option
In the above command, If there was a directory present in the 'Original' folder, it would be skipped. That is illustrated in the image below.
To overcome this, use the -r (recursive) option.
3. Sync/Copy files between the local machine and remote machine
Before carrying out this task, there are several parameters that you need to know about the remote server/machine: the IP-address, the username, and the user password.
The basic syntax that we will use is:
rsync [options] [files] [remote-username]@[ip-address]:/[destination]
Let's copy files from our 'Original' directory on the Desktop to a remote server.
4. Sync/Copy files and directories from a remote server to your local PC
Just like the previous command, we will need to know the IP-address of the remote server. In this example, we will sync files in the 'Backup' folder in the server's home directory to our Local PC.
rsync -rv [email protected]:/home/tuts/Backup /home/tuts/Desktop/Original
5. Use Rsync over SSH with the -e option
To ensure the security of files and folders, we will use the Rsync over Secure Shell Protocol (SSH). Additionally, when providing the root/user password - SSH will provide encryption services, ensuring that they are safe.
To use SSH, we will add the -e option that specifies the protocol that we want to use.
rsync -vre ssh Original/* [email protected]:/home/tuts
6. Show progress with Rsync command
In situations where you copy multiple files or a huge file, knowing the progress would be efficient. Fortunately, rsync has the provision of this option -the '-progress' option.
sudo rsync -rv --progress Original/* /etc/Foss
7. Use Rsync with the '-include' option
There are situations where you only want to sync particular files. With Rsync command, you cause the '-include' option to carry out the task. In this example, we will only synchronize files starting with 'I' letter.
sudo rsync -vr --include 'I*' Original/ /etc/Backup/
8. Use Rsync with '-exclude' option to ignore particular files
With the Rsync '-exclude' option, you can exclude files that you don't want to sync/copy. In this example, we want to ignore all files starting with the 'I' letter.
sudo rsync -vr --exclude 'I*' Original/ /etc/Backup/
Alternatively, you can use both options in one command. See the example below.
sudo rsync -vr --exclude '*' --include 'I*' Original/ /etc/Backup/
We are excluding all files apart from those starting with the letter 'I.'
9. Use Rsync with -delete command
There are situations where there are files present in the destination directory, but not in the source folder. For synchronization, it would be best to first remove such files from the destination directory. Fortunately, Rsync comes with the '-delete' option, which does this automatically.
If a file is present in the destination directory but no in the source, it will delete it.
rsync -vr --delete /etc/Backup/ [email protected]:/home/tuts/Backup
10. Set the Maximum Size of Files to transfer with Rsync
If you are concerned with storage space or bandwidth for remote file synchronization, you need to use the '-max-size' option with the Rsync command. This option enables you to set the maximum size of a file that can be copied.
For example, a '-max-size=100k' will only transfer data equal to or smaller than 100 kilobytes.
rsync -vr --max-size='200k' /etc/Backup/ [email protected]:/home/tuts/Backup
11. Delete source files automatically after a successful transfer
Take a situation where you have a remote backup server and a backup directory on your PC. You back up data to the backup folder on your PC before syncing it with the Backup-server. After every synchronization, you will need to delete the data in the backup directory.
Fortunately, you can do this automatically with the '--remove-source-files' option.
By running an 'ls' command on the source folder, we confirm that indeed the files were deleted.
12. Perform a dry-run with Rsync
For newbies who are not sure with the Rsync command, you can mess your destination directory contents, and undoing could be much as tedious. With the '-dry-run' option, the Rsync command will give you an output of what will be performed, but it won't do it.
Therefore, you can look at this output if its what you expect before going o to remove the '-dry-run' option.
rsync -vr --dry-run Original/* [email protected]:/home/tuts/Backup
13. Set Bandwith Limit required to Transfer files.
If you are on a shared network or running several programs in need of an internet connection, it would be efficient to set a bandwidth limit required to Sync/copy files remotely. We can do this with the Rsync '-bwlimit' option.
This rate is calculated in kilobytes. Therefore, when '-bwlimit=1000' means that only 1000Kb can be transferred per second.
rsync -vr --bwlimit=1000 Original/* [email protected]:/home/tuts/Backup
14. Sync the whole files with Rsync
By default, Rsync only synchronizes the modified blocks and bytes. Therefore, if you had synced a text file before and later added some texts to the source file when you sync, only the inserted text will be copied. If you want to sync the entire file again, you will need to use the '-W' option.
rsync -vrW Original/* [email protected]:/home/tuts/Backup
15. Do not Sync/Copy Modified files in the destination directory
There are situations where you have made modifications to files present in the destination folder. If you run a Rsync command, these modifications will be overwritten by those in the source file. To avoid such, use the '-u' option.
rsync -vu Original/* [email protected]:/home/tuts/Backup
16. Use Rsync with -i option to view the difference in files between source and destination
If you wish to know what new changes will be made to the destination directory, use the '-i' option, which will show the difference in files between the source and destination directory.
rsync -avzi Original/ Backup/
Form the output in the image above, and we see a file called 'heloo.py' not present in the destination directory. Let's look at some of these parameters.
d: shows a change in the destination file
f: represents a file
t: shows a change in timestamps
s: indicates a change in the size of a file
17. Use Rsync to Copy Directory Structure only
You can use Rsync to sync only the directory structure if you are not interested in the files. We will need to use the parameters -f”+ */” -f”- *” before the source directory.
rsync -av -f"+ */" -f"- *" /home/tuts/Desktop/Original/ /home/tuts/Documents/
18. Add Date Stamp to Directory Name
If you want to track when transfers took place without opening directory properties, you can easily add a date to a directory name. That will add a date stamp to all synchronizations you do with Rsync.
To do so, we will append $(date +\\%Y-\\%m-\\%d) to the destination directory.
sudo rsync -rv Original/ /etc/$(date +\\%Y-\\%m-\\%d)
19. Copy a single file locally
To sync/copy a single file with Rsync, you will need to specify the file path followed by the destination directory path.
rsync -v Original/heloo.py Backup/
20. Copying multiple files remotely
To copy multiple files simultaneously, you will need to provide the path to all of them. This method is efficient for only a small number of files.
rsync -vr /home/tuts/Desktop/Original/heloo.py /home/tuts/Desktop/Backup/index.html [email protected]:/home/tuts/Backup
That's all about the top 20 useful Rsync examples that will enable you to sync files and directories both locally and remotely. Feel free to leave your feedback and comments below.
