How to Use rsync to Exclude Files and Directories in Data Transfer

March 3, 2020

Introduction

Rsync is a useful Linux command-line tool that syncs and copies files and directories. You can use the tool to synchronize data locally between directories and drives, or between two remote machines.

The basic rsync commands in Linux sync everything from a location you specify. In many backup scenarios, you may want to exclude specific files, directories, or file types.

Follow the examples in this guide to learn how to exclude files or directories with rsync. We will list the most common exclude use-cases to cover your day-to-day rsync usage.

Guide on how to use rsynic to exclude Files and Directories.

Prerequisites

  • User with sudo or root privileges
  • Access to a command line/terminal window
  • Rsync installed on your system

How rsync Exclude Option Works

The --exclude option with the rsync command uses relative paths to the source directory. Append the --exclude option to the rsync command, followed by the relative path to a directory or file(s).

The basic syntax for the rsync exclude option looks like this:

rsync [OPTIONS] --exclude 'file_or_directory' source/ destination/

Replace source/ with the directory name you want to use as a source for data transfer to another location.

Replace destination/ with the directory name rsync will use as the target location for your data. If the directory does not exist, rsync creates one for you and transfers the files to that directory.

To back up files to a remote location, follow our other guide to learn how to copy or transfer files with rsync over SSH.

Note: The trailing slash (/) on the source directory modifies the behavior of the rsync command.

  • If you do not use a trailing slash, the source directory is copied to the destination directory, and then the contents of the directory.
  • When you do use the trailing slash, rsync only copies the content of the source without creating an additional directory level.

Exclude a Specific File

In the rsync exclude examples below, we will use the -a (archive) and -v (verbose) options (-av). The -a option syncs directories recursively while keeping permissions, symbolic links, ownership, and group settings. The -v flag is optional and prints the progress and status of the rsync command.

To exclude a file while transferring the contents of a folder with rsync, specify the file and the relative path.

For example:

rsync -av --exclude 'testfile1.txt' sourcedir/ destinationdir/

This command allows you to copy the files to destinationdir from sourcedir but exclude testfile1.txt.

Rsync command in the terminal showing how to exclude a specific file.

As you can see in the output, testfile1.txt is not on the list.

Note: If you want to test all the examples in the guide, please remove the destination directory before trying a new example.

To do so, run this command:

sudo rm -rf destinationdir

Exclude a Specific Directory

Now that you know how to use the rsync --exclude file command, use the tool in the same fashion to exclude a directory.

Specify a directory you want to exclude (instead of a file name):

rsync -av --exclude 'dir1' sourcedir/ destinationdir/
Terminal output for the rsync exclude directory command.

This command copied the contents of sourcedir into destinationdir and excluded dir1, as seen in the output.

Exclude Files or Directories Based on a Pattern

Use an asterisk * (wildcard) when defining a file or directory name to exclude everything that matches the pattern.

To exclude files that start with test, run this command:

rsync -av --exclude 'test*' sourcedir/ destinationdir/

Every file and directory that matches this pattern will be excluded from the transfer.

You can also use the wildcard in a similar matter to exclude all directories that end with a specific pattern.

Run this command to exclude all directories that end with number 3:

rsync -av --exclude '*3' sourcedir/ destinationdir/

You can use an asterisk before and after a pattern to additionally refine the --exclude criterion.

Exclude a Specific File Type

The rsync tool allows you to exclude certain file types when synchronizing data. Use an asterisk * followed by the extension of the file type you want to exclude.

For example, you may want to back up a directory that contains many .iso files that you do not need to back up.

To exclude a specific file type, in this case .iso, run this command:

rsync -av --exclude '*.iso' sourcedir/ destinationdir/
Terminal output for rsync command to exclude a certain file type from your backup.

The output shows that rsync did not transfer .iso files. We added the contents of the source directory in the image for comparison.

Exclude Files by Size

During file transfers, you can specify the minimum or the maximum size of files you want to exclude. If your source directory contains many large files that you do not want to back up, use the --max-size=size_in_mb_or_gb option. Replace the size_in_mb_or_gb with the desired size.

For example, to exclude all files larger than 500MB, run this command:

rsync -av --max-size=500m sourcedir/ destinationdir/

On the other hand, to exclude files smaller than a specific size, use the --min-size=size_in_mb_or_gb option.

For example, you want to transfer a directory with images, but there are many thumbnail files. If all your images are larger than 1MB, then you can exclude all files smaller than that size.

To do so, run this command:

rsync -av --min-size=1m sourcedir/ destinationdir/
Terminal output for rsync command to exclude files with minimum file size

Do not use the --exclude rsync option with when defining the minimum or maximum file size.

Exclude Multiple Files or Directories

Add multiple --exclude options to exclude multiple files or directories. You can combine any rsync --exclude folder and rsync --exclude file(s) commands to transfer only the data you need. Any of the commands we previously talked about can be used in one line.

The following command will exclude all files with the .txt extension, as well as the dir3 and dir4 directories:

rsync -av --exclude '*.txt' --exclude 'dir3' --exclude 'dir4' sourcedir/ destinationdir/

You can add as many --exclude entries as you need. To keep the command tidy, you can specify files and directories in the --exclude option using curly brackets. Separate the patterns using a comma.

For example, we can shorten the above command in this fashion:

rsync -av --exclude={'*.txt','dir3','dir4'} sourcedir/ destinationdir/
Terminal ouptut for rsync exlcude multiple files and directories

The output shows that the listed files and directories are excluded from the transfer.

Exclude Files and Directories from a List

When you need to exclude a large number of different files and directories, you can use the rsync --exclude-from flag. To do so, create a text file with the name of the files and directories you want to exclude. Then, pass the name of the file to the --exlude-from option.

The command looks like this:

rsync -av --exclude-from={'list.txt'} sourcedir/ destinationdir/

The rsync tool skips all files and directories you list in the file. You can add any pattern we used in this guide.

The example file we used in the above command contains the following patterns:

A list of files to exclude from a text file.

Conclusion

This guide showed you all rsync exclude examples you need when transferring data with this tool. You can combine the patterns to customize the command and exclude multiple files and directories with rsync.

Was this article helpful?
YesNo
Goran Jevtic
Goran combines his leadership skills and passion for research, writing, and technology as a Technical Writing Team Lead at phoenixNAP. Working with multiple departments and on various projects, he has developed an extraordinary understanding of cloud and virtualization technology trends and best practices.
Next you should read
Rsync Command: 20 Helpful Examples in Linux
March 23, 2020

This tutorial provides 20 most commonly used rsync commands for all data synchronization scenarios. Master...
Read more
How to Transfer Files with Rsync over SSH
January 31, 2020

Rsync is a Linux tool that allows you to transfer data over SSH to a remote server securely. Use the options...
Read more
How to Copy Files and Directories in Linux
May 21, 2019

Want to learn how to copy files in Linux OS? This guide will show you how to use the Linux commands to copy...
Read more
Linux SCP Command: Securely Copy & Transfer Files
March 3, 2020

Tutorial on securely transferring files between Unix or Linux systems using the SCP command. The SCP or...
Read more