How to Compare the Contents of Two Folders and Synchronize them

Let’s assume you made a backup task XCopy, SyncToy or any other tool to mirror copy a certain directory to a different location. If you’re using Microsoft SyncToy, it’s possible that sometimes the files or folders may be missing in the destination, or the destination folder has extra files or folders that are not in the source folder.

compare two folders and files list

This article tells you how to compare the directory structure (along with files) in two different directory paths to make sure you have an exact replica or mirror copy of the source. Some of the methods below also discuss how to synchronize items in two folders after comparing.

There are 12 folder comparison methods (for Windows) discussed in this article. Choose the one which suits you best. If you need to check file integrity after synchronization, use one of the following hash checksum verification procedures listed.

Compare the contents of two folders using Microsoft’s WinDiff

WinDiff is a graphical file comparison tool from Microsoft which was first released in the year 1992 and subsequently updated. Please hold on! Don’t be taken aback by the year of the initial release.

WinDiff is still awesome and it works perfectly fine in Windows 10, but no Unicode support. It has enormous potential. WinDiff can compare ASCII & Binary files, compare two directories, and synchronize the folders (left folder → right folder, or vice-versa).

WinDiff download link (local mirror). The last updated WinDiff version is 5.2.3790.0 with the Date modified stamp showing 4-7-2016.

You can also download WinDiff as part of the Windows XP Support Tools package, and extract its contents using 7-Zip.

To do a directory comparison using WinDiff:

  1. Start Windiff.exe.
  2. On the File menu, click Compare Directories.
  3. In the Select Directories dialog box, type the two folder names that you want to compare in the Dir1 and Dir2 boxes. If you want to compare files in those folders recursively, enable the Include subdirectories checkbox.
    compare the contents of two folders in windows

The results or the outline window by default shows identical files as well as the differences. You can hide identical files by disabling Show Identical Files from the Options menu.

compare the contents of two folders in windows

As you can see, WinDiff is not just comparing file names but also compares file contents in both locations and highlights changes if the file contents are different.

compare the contents of two folders in windows

Double-clicking on the “different” item opens the comparison view with color-coding. The color-coded results indicate what the file differences are.

WinDiff can do ASCII and binary comparison. This means, WinDiff doesn’t rely upon modified date but compares the actual contents of the files.

For example, I modified a character in a text file in the destination folder (Right folder). I replaced an exclamation mark with a colon, and the file size is exactly the same (1,127 bytes) in both locations. WinDiff picked it up and showed that the files are different.

compare the contents of two folders in windows

To save the directory comparison differences to a file, click Save File List in the File Menu. Type a file name with complete path without double-quotes.

compare the contents of two folders in windows

Uncheck Identical files and Include Checksums options, and click OK. The output file will look like this:

-- D:\Websites\Winhelponline\blog\wp-content\uploads\2019 : E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 -- includes left-only,right-only,differing files
.\accesschk.exe	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\compare-folders.txt	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\fta-1803-w10.txt	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\ssh-commands-useful.txt	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\05\dell-supportassist.png	only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019
.\05\disable-theme-sync-ramesh.png	different (E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 is more recent)
.\05\file-explorer-not-highlight-files.png	only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019
.\05\firefox-forget-about-this-site-4.png	different (E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 is more recent)
.\05\msconfig-disable-services-non-microsoft.png	only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019
.\05\windows-store-error-0x800706d9.png	only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\notepad-bing-search.png	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\rundll32-refresh-commands.txt	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\archives\404chk.vbs	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\archives\actxprxy-missing-issues.txt	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\archives\apps.png	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\archives\articles.vbs	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
.\06\archives\bash-sh.vbs	only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019
-- 17 files listed

Since the output file is in tab-delimited format, you should be able to import into Microsoft Office Excel or Google Sheets easily.

compare the contents of two folders in windows - windiff

Note that you can start WinDiff directly with source and destination paths as additional switches for directory comparison. For example, launching windiff.exe "c:\source" "d:\destination" would launch the tool and straight away compare those two folder paths.

WinDiff command-line switches

Here is the complete list of WinDiff command-line switches.

Usage:

	windiff [options] path1 [path2]

Options:

-D	Compare one directory only.
-F[flags] savefile	Save composite file to 'savefile'.  The 'flags' may consist of one or more of I (identical), L (left), R (right), F (moved leFt), G (moved riGht), S (Similar left), A (similiAr right), X (exit after saving list).
(e.g. -FLF saves list of Left or moved-leFt lines).
-I file	Reads list of files to compare, from the specified input file.  Each line can contain one or two filenames, space delimited (with quoting, if filenames contain spaces).  Use "-" as the filename to read from stdin.  If a line contains only one filename, the file is compared to itself.
-N name	NET SEND notification to 'name' at end of comparison.
-O	Outline view (no automatic expansion).
-P	Perverse comparison: breaks lines on punctuation.
-S[flags] savefile	Save list of files to 'savefile'.  The 'flags' may consist of one or more of S (same), L (left), R (right), D (different), X (exit after saving list).
(e.g. -SLD saves list of Left or Different files).
-T	Compare whole subtree.

To save the comparison report to a file using the -S switch. Examples are below:

Save the list of identical files to diff.txt:

-SS D:\output\diff.txt c:\folder1 d:\folder2

Save the list of different files to diff.txt:

-SD D:\output\diff.txt c:\folder1 d:\folder2

Save the list of left-only files to diff.txt:

-SL D:\output\diff.txt c:\folder1 d:\folder2

Saves the list of right-only files to diff.txt:

-SR D:\output\diff.txt c:\folder1 d:\folder2

Saves the list of left-only AND right-only files to diff.txt:

-SLR D:\output\diff.txt c:\folder1 d:\folder2

To exit WinDiff automatically after saving the comparison report, add the X switch, as below:

Save the list of right-only files to diff.txt and exit WinDiff

-SRX D:\output\diff.txt c:\folder1 d:\folder2

Save the list of left-only AND right-only files to diff.txt and exit WinDiff

-SLRX D:\output\diff.txt c:\folder1 d:\folder2

To generate all 5 individual “log” files you would have to run WinDiff 5 times.

Example:

windiff -SSX identical.txt c:\folder1 d:\folder2
windiff -SDX different.txt c:\folder1 d:\folder2
windiff -SLX left-only.txt c:\folder1 d:\folder2
windiff -SRX right-only.txt c:\folder1 d:\folder2
windiff -SLRX left-right-only.txt c:\folder1 d:\folder2

See also WinDiff Help File Contents (windiff.hlp) in HTML format.

Sync folders with WinDiff

What’s more? WinDiff can also copy the missing or different files to the left (Folder 1) or to the right (Folder 2) location. To do so,

  1. From the File menu, select Copy Files
  2. Type the folder path where you want to copy the missing/different files.

    Important: Don’t use double-quotes around the folder path even if the folder path contains spaces. WinDiff fails to copy files if you adding double-quotes to the path.

  3. Choose the Copy from location (Left tree to Right tree, or vice-versa)
  4. Uncheck Identical Files, and click OK.

    compare the contents of two folders in windows

    WinDiff now syncs the folder by copying the missing and changed files recursively to the specified location (Left → Right).

    compare the contents of two folders in windows

Despite the excellent ASCII & binary comparison and basic sync capabilities, WinDiff is often an underrated tool among IT folks. That’s because it sports an outdated user interface and has no Unicode support. But WinDiff is enough for most of us, especially those who use the English language OS.

Microsoft has a knowledgebase article on WinDiff, titled How to Use the Windiff.exe Utility which you may want to check out.



Compare the contents of two folders using WinMerge

Here is how to compare two folders along with sub-folders, using WinMerge.

winmerge - logoWinMerge is an Open Source differencing and merging tool for Windows. It can compare both folders and files, presenting differences in a visual text format that is easy to understand and handle. Let’s use it to compare the contents of two folders along with their sub-folders (i.e., recursively).

  1. Start WinMerge. From the File menu, click Openwinmerge compare 1st folder 2nd folder open
  2. Browse to select the 1st folder and 2nd folder for comparing.
  3. Make sure that the filter is set to *.* so that all files are compared.The “Include Subfolders” (compares recursively) option is enabled by default. If you want to compare files in the top-level directory only, uncheck the option.
  4. Click Compare

    WinMerge generates folder compare results, highlighting the differences in yellow color.
    winmerge compare results main

  5. From the Tools menu, click Customize Columns…. Choose the Columns you want, such as Left size, Right size, Left date, Right date from the numerous other column options provided.
    winmerge columns
  6. Expand the “Folders are different” entry by double-clicking on it. It shows the list of files that are different, “left only“, “right only” and also the files that are identical.
    winmerge compare results expandedNote: You can choose to hide the identical files and folders by unchecking “Show Identical Items” from the View menu in WinMerge.
    winmerge view options
The advantage of using WinMerge is that it offers five different file compare methods to choose from:
  1. Full Contents (Default): Full comparison of files by content, with all the bells and whistles. This method invokes plugins and uses the diffutils engine for fully accurate differencing and moved block detection. This is the most complete and recommended method.
  2. Quick Contents: Slimmed-down comparison of files by content. This method uses streamlined file comparison code which skips plugins and moved block detection. This method is faster than Full Contents, because it does not load the files. Its drawback is that line filters are not applied when comparing. For example, this method sees file as different even if line filters are set to ignore all differences in the files.
  3. Modified Date: Compares only the modification dates on the files, so it is far faster than either of the contents methods. But obviously, it is only as accurate as the modification dates.
  4. Modified Date and Size: Similar to Modified Date, but also checks file sizes when dates are identical.
  5. Size: Compares only file sizes, so it is fast but not as accurate as the contents methods.

WinMerge supports a lot of command-line switches. Check out the WinMerge Manual for the entire list of command-line switches supported.



Sync folders with WinMerge

If you have unchecked the Show Identical Items from the View menu in WinMerge, it only shows you the list of modified, left-only and right-only files. It’s easy to synchronize the items from left → right, or vice versa.

winmerge compare folders and synchronize

All you need to do is select the files (or select all), right-click on the selection and click Copy. In the Copy sub-menu, choose one of the options:

  1. Left to right: Copy selected files from the left folder to the right.
  2. Left to …: Copy files from the left folder to some other folder.
  3. Right to left: Copy selected files from the right folder to the left.
  4. Right to … : Copy files from the right folder to some other folder.

winmerge compare folders and synchronize


Method 3: How to Compare Files in Two Folders Using FreeFileSync

freefilesync iconFreeFileSync is an open Source software folder comparison and synchronization software that creates and manages backup copies of all your important files.

  1. Download FreeFileSync and install it.
  2. To compare folders by file content instead of file size or date modified, choose File content in the Compare options. Comparing by file size will be a lot quicker and less resource-intensive, though.
  3. Select the left folder and the right folder.
  4. Click on the Compare (File content) button.

    It compares items in the two folders recursively and shows the list of new, and changed files.

    compare the content of two folders - freefilesync

    The identical files are hidden in the output. It shows the left-only files, different files, and the right-only files by default, and the options can be toggled using the buttons at the bottom of the FreeFileSync window.



Synchronize folders using FreeFileSync

FreeFileSync offers the following synchronize options similar to Microsoft SyncToy.

compare the content of two folders - freefilesync

  1. Two way sync: Identify and apply changes on both sides. Deletions, moves, and conflicts are detected using a database.
  2. Mirror: Mirror is Left → Right synchronization. Right-folder will become a mirror copy of the left folder. It’s the same as Microsoft SyncToy’s Echo option, and robocopy /mir command-line. Extra files and folders on the right folder are removed and different files are replaced.
  3. Update: This sync option does a Left folder → Right folder synchronization similar to the Mirror option. However, this option doesn’t delete the extra files and folders in the right folder tree. It’s similar to Microsoft SyncToy’s Contribute option.
  4. Custom: In addition to the above three preset options, the Custom option lets you choose your default action on how to deal with a left-only file, changed file, or the right-only file. For instance, you can configure it to delete identical files if your requirement is to have only unique files (eliminate duplicates) in the left and right folders. In most cases, you don’t need to choose the Custom setting unless you have a special requirement.

By default, it does 2-way sync. Personally, I prefer the Mirror backup for my website projects and Update backup option for documents. Choose one of the synchronize options accordingly.

In the comparison results, all the files are selected by default. You can exclude a file from being synchronized by unchecking the checkbox adjacent to the file name, or via the right-click menu.

compare the content of two folders - freefilesync

  • To synchronize all the items displayed on the list, click Synchronize button at the top.
  • To synchronize a particular file or a set of files, Exclude all items (temporarily) via the right-click menu, then enable the checkbox for selected files, right-click on the selection, and click Synchronize selection, and click Start.

That’s it! Your folders are now synchronized and the copy/update/delete operations are logged by FreeFileSync.

compare the content of two folders - freefilesync

Compare & Sync a Local folder Vs. FTP folder recursively

FreeFileSync can also compare a local folder versus Google Drive, or a local folder versus a folder your FTP server recursively using FTP or SFTP connection. Here is how to compare and synchronize a local folder and subfolders with a folder on your FTP server.

All you need to do is click on the Access online storage button next to the Browse button on the left pane or the right side. Clicking that button opens the following dialog:

compare and sync local folder and ftp using freefillesync

Input your FTP or SFTP connection information, credentials and the FTP directory path to compare against your local folder.

Once done, you’re now back at the Comparison window.

Chose File size or File time and size comparison method. The latter comparison method is more reliable.

Select the synchronization variant. I’ve set mine to 2-way sync (instead of Mirror) for this purpose.

Once configured, click Compare. You’ll see the list of missing and different files in the left as well as the right folder.

compare and sync local folder and ftp using freefillesync

Click on the Synchronize button to copy files both ways, as 2-way sync has been selected. If you want to update only the FTP location, choose Mirror sync instead.

Run FreeFileSync as a batch job | Command-line synchronization

You can save the above configuration to a batch job and run it via command-line or Scheduler. To save the configuration and run it using command-line, follow these steps:

  1. Click Save as batch job… option from the File menu.

    compare the content of two folders - freefilesync

  2. You can configure the batch job to run minimized and automatically close the window when the synchronization is complete.
    compare the content of two folders - freefilesync
  3. Save the batch job file to a folder of your choice, say d:\BatchRun.ffs_batch
  4. Then, to sync the folders using command-line, use the following command-line syntax:
    C:\Program Files\FreeFileSync\FreeFileSync.exe d:\BatchRun.ffs_batch

    You can create a Windows batch file, or run the above command via Task Scheduler at periodic intervals as desired.

Tip: FreeFileSync can also sync folders against Google Drive.


Method 4: How to Compare Files in Two Folders Using TotalCommander

total commander - logoTotal Commander is a file manager for Windows that lets you copy, move or delete files. Total Commander can do much more than that. It can pack and unpack files, access FTP servers, compare files by content, etc.

compare the contents of two folders - total commander

  1. Start Total Commander and select the left folder and the right folder to compare.
  2. From the Commands menu, click Synchronize dirs…

    With the Synchronize dirs dialog, you can compare two directories including subdirectories with each other, and then copy the files with differences to the target directory, or any other directory.

    compare the contents of two folders - total commander

    If you choose the Asymmetric option, it makes the right folder a mirror of the left — similar to RoboCopy /MIR command-line or Microsoft SyncToy’s Echo option. That is, the option copies missing or different files to the right folder, and any extra files and folders on the right are deleted.

    You can choose to show identical files or only the different files in the output.

    compare the contents of two folders - total commander

    Total Commander’s compare/synchronize options explained below.

    Asymmetric If this option is checked, it is assumed that a copy of the left side should be created on the right side. Files which do not exist on the left side will be marked for deletion on the right side. This option is meant for backups.
    Subdirs Also compares the subdirectories of the two chosen directories.
    by content Compares the content of files which have the same size and date. It checks that the files also have the same content.
  3. Select the compare options “Subdirs”, “by content”.

    Important: If the ignore date checkbox is enabled, Total Commander fails to find the different/changed files even though the file sizes are different. It seems to be a bug in Total Commander, and I can reproduce it every time.

  4. In the Show section, disable Equal files (denoted by the = symbol), and enable the other three buttons.
  5. Enable the Singles and Duplicates options.
  6. Press the Compare button
    compare the contents of two folders - total commander

Total Commander – Folder compare command-line arguments

To start “Synchronize dirs”, you can use the following command-line syntax:

TOTALCMD64.EXE /S=S d:\folder_1 d:\folder_2

Then, configure the options and press the Compare button.

To start “Synchronize dirs” and compare folders right away, use this syntax:

TOTALCMD64.EXE /S=S:= d:\folder_1 d:\folder_2

(the comparison will start with the passed directories and last used options.)

Optionally, you can also type in a settings name in the form /S=S:SettingsName so that the comparison will start immediately based on that previously saved setting.



Synchronize folders using TotalCommander

  1. In the compare window, select the files you want to copy. This is done by a click on the checkbox between the two comparison columns, or by selecting one of the options in the right-click menu.

    By default, the function will select all newer files for copying. To deselect a file to prevent it from being copied or synchronized, double-click on the arrow. To deselect multiple files, select the files, right-click on the item(s) and choose Remove Selection

    Important: If the Asymmetric option is not enabled, if a file on the right folder is newer than the one in the left folder, synchronization will run from Right → Left for that file. To change it the other way, click on the arrow mark next to the file, which toggles the direction of sync.

    compare the contents of two folders - total commander

    Alternately, you can right-click on the file and choose Reverse copy direction to change the sync direction.

  2. Click on Synchronize. This will open a dialog box to copy the selected files.
    compare the contents of two folders - total commander

    By default, it synchronizes both ways. If you want to synchronize one way – eg., left folder → right folder, then deselect the Right to left checkbox and vice versa.

Tip: Using Total Commander, you can even synchronize a local directory and a directory on an FTP server.


Method 5: Compare & Synchronize folders in Real-time using DSynchronize

synchronize folders realtimeDSynchronize is a nice tool using which you can synchronize two folders manually or automatically in real-time. This tool provides a lot of options than any other folder sync tools for you to configure how files are synchronized.

  1. Launch DSynchronize and configure the source and destination folders.
  2. You can assign custom job name (for later automation) by clicking on the JOBS button. Let’s set the job name as Uploads
  3. For mirror sync, make sure that the Bidirectional Sync is disabled.
  4. To do a compare (only) without synchronizing the two folders, enable the option Preview sync under the General section.
    synchronize folders in real-time - dsync
  5. Click on the Synchronize button to view the list of new, updated and missing files in both locations.


Synchronize folders

To synchronize folders, uncheck the Preview sync option under the General section, and click Synchronize. It does a Mirror mode sync by default. Mirror mode has been discussed in this article earlier. Mirror mode ensures that the right folder is exactly the same as the left folder. Any extra directories and files in the destination are removed automatically.

Real-time synchronization

To synchronize two folders in real-time:

  1. Click on the Realtime sync under the Timer section.
  2. Configure the real-time sync settings as desired. I used the default settings.

    synchronize folders in real-time - dsync

    By default, it automatically syncs folders after a 5 seconds lapse. You can configure it to defer further, or schedule it to run at specified intervals.

  3. Enable the Monitor subfolders checkbox to synchronize folders recursively.

I copied some files to the left folder (source), and within 5 seconds, the items were synchronized to the right folder automatically. You can see that in this animated GIF:

synchronize folders in real-time - dsync

And, the changes (file or folder additions & deletions) are logged in the main window.

synchronize folders in real-time - dsync

When real-time synchronize is enabled, it monitors for file changes and the program sits in the notification area waiting for changes. It doesn’t take much CPU resources or memory. Last I checked, it used only around 10 MB of memory.

synchronize folders in real-time - dsync

Here is the list of salient features of DSynchronize:

  • Standard and timer sync.
  • RealTime sync.
  • Bidirectional sync.
  • Transactional sync (by the journal of NTFS).
  • Selective filter (you can exclude single files or folders).
  • Start as a Service (the program don’t need to log in).
  • Supports Symbolic Links and Hard Links.
  • Supports Junctions and Mount Points.
  • Supports ACL.
  • Log sent by Email.
  • Backup of the previous version of replaced files.
  • Byte to byte compare.
  • Supports multiple Jobs, each with its settings and sync paths.
  • Minimum resource consumption (RAM and CPU).

Command-line support

You can use the following command-line to automate DSynchronize:

/START = Start synch with last saved options.
/STARTJOB "JobName" = Start synch with selected Job options).
/MINIMIZE = Starts minimized in the Tray Bar.
/HIDDEN = Do not display icon in Tray Bar.
/NOSECURITYCHECK = Don’t makes security check before sync.
/NOMONITORCHECK = Do not check monitor resolution.
/AUTOREALTIME = Starts minimized, do a standard sync and activate RealTime.
/SOMEFILE.INI = Starts using selected INI file.

The job name can be customized by clicking on the JOBS button at the top of the program’s main window.

For example, to automatically sync two folders (previously saved job named “Uploads”), I’d run this command:

"D:\dsynchronize\DSynchronize.exe" /STARTJOB "Uploads" /MINIMIZE

To synchronize the two folders (job named “Uploads) and activate real-time sync, run:

"D:\dsynchronize\DSynchronize.exe" /STARTJOB "Uploads" /AUTOREALTIME

The above command synchronizes the two folders named in the job, and activates real-time monitoring automatically. You can see the real-time monitoring icon in the Notification area.

That’s it!




Method 6: Compare Two Folders with HashMyFiles utility using File Hash Checksum

HashMyFiles is a small utility calculates hash checksums for your files. It can compute the hash for files in a folder and its subfolders recursively. HashMyFiles lets you save the results to a file. It can also compute hash checksum via the right-click menu.

get file hash via the right-click menu - hashmyfiles

If PowerShell is not your cup of tea, use the HashMyFiles utility from Nirsoft.net to compare file hashes.

You can save the file names and the corresponding hash output for the source folder tree to source.txt. Similarly, save the entries for the destination folder tree as destination.txt.

In this example, we’re going to compare files in the following folders:

D:\Websites\Winhelponline\blog\wp-content\uploads\2019 
E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019

Follow these steps to generate file hashes using HashMyFiles and compare them:

  1. Download HashMyFiles and run it.
  2. From the Options menu, click Hash types, and select SHA-256
    compare files in two folders - hashmyfiles
  3. From the View menu, click Choose Columns

    compare files in two folders - hashmyfiles

  4. In the Column Settings, enable the options SHA-256, Full path, and File size. Click OK.
  5. Click the File menu and select Add Folder
  6. Select the source folder for comparison.
    compare files in two folders - hashmyfiles
  7. Click to select Add files in subfolders, and click OK.
  8. Sort the results by Full Path folumn, and select all entries.
  9. From the File menu, click Save selected items
  10. Save the listing to a file named source.txt
  11. Clear the results by selecting Clear All from the File menu.
  12. Repeat the above steps, and this time select the destination folder, save the listing to destination.txt
  13. Now, edit the files source.txt and destination.txt using Notepad and remove the base paths. Using the Find option in Notepad, find every occurrence of the base path below and replace with an empty string.
    D:\Websites\Winhelponline\blog\wp-content\
  14. Similarly in the destination.txt file, remove this base path:
    E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\

    You may also like to find and remove the = symbols used as separators.

    This is how the files now look like:

    compare files in two folders - hashmyfiles

  15. Now, fire up WinDiff or your favorite file compare tool to see the differences.
    compare files in two folders - hashmyfiles

You’ll see the list of different files, extra files, and missing files.




Method 7: How to Compare Files in Two Folders Using Robocopy

Let’s now see how to use the built-in Robocopy console utility to compare the contents of two folders.

Robocopy (“Robust File Copy for Windows”) is an excellent file copy tool built-in to Windows that has enormous capabilities. Robocopy’s capabilities are above and beyond the built-in Windows copy and XCopy commands. RoboCopy is the most preferable of all the command-line methods because Robocopy compares not only the file names but also the last modified time. It reports the list of modified files and new files/folders in the source and destination paths.

Among its excellent features include “mirror” mode copy (similar to Microsoft SyncToy’s “Echo” option), ability to retry copying in case of network interruptions, multi-threaded copying, etc. Other than copying files, we can use Robocopy to just compare the contents of two folders (recursively) and log the differences without copying anything, like a dry run.

Scenario

My website’s files are synced to an external hard drive (E:\) regularly. For the purpose of illustration, let’s check if these two folders are identical. In this example, we’re going to compare the following folders, recursively (i.e. along with sub-folders, a deep comparison):

D:\Websites\Winhelponline\blog\wp-content\uploads
and
E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads
  1. Open a Command Prompt window.
  2. Run the following command and press ENTER:
    robocopy "D:\Websites\Winhelponline\blog\wp-content\uploads" "E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads" /L /TS /MIR /LOG:D:\Diff.txt

    robocopy command list differences folders

    The results of the above command is logged to D:\Diff.txt file.

  3. Exit the Command Prompt window.

Details about the Robocopy command-line switches used

  • /L :: List only – don’t copy, timestamp or delete any files. This option is used for logging purposes only; for a dry-run.
  • /MIR :: MIRror a directory tree (equivalent to /E plus /PURGE). Since we’re using it with /L switch, no file/folder will be actually copied.
  • /TS :: include source file Time Stamps in the output.
  • /LOG:file :: output status to LOG file (overwrite existing log).


In this example, we’re writing the output to D:\Diff.txt. Here is what the log look like:


-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Monday, April 8, 2019 5:06:10 PM
   Source : D:\Websites\Winhelponline\blog\wp-content\uploads\
     Dest : E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\

    Files : *.*
	    
  Options : *.* /TS /L /S /E /DCOPY:DA /COPY:DAT /PURGE /MIR /R:1000000 /W:30 

------------------------------------------------------------------------------

                 583	D:\Websites\Winhelponline\blog\wp-content\uploads\
*EXTRA Dir        -1	E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019\videos\
  *EXTRA File 		  821886 2019/04/07 07:19:24	tiktok-superman-dance.mp4
                   3	D:\Websites\Winhelponline\blog\wp-content\uploads\2019\Scripts\
    Older     		     210 2019/04/08 10:54:59	0x80071A91.vbs
    Newer     		     211 2019/04/08 11:35:32	ErrCodeFinder.vbs
    New File  		     211 2019/04/08 11:35:32	Sample.vbs
                 211	D:\Websites\Winhelponline\blog\wp-content\uploads\h1-10\
                  83	D:\Websites\Winhelponline\blog\wp-content\uploads\h1-11\
                   1	D:\Websites\Winhelponline\blog\wp-content\uploads\reg\
                   9	D:\Websites\Winhelponline\blog\wp-content\uploads\temp\

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :       113         0       113         0         0         1
   Files :      4125         3      4122         0         0         1
   Bytes :   59.96 m       632   59.95 m         0         0   802.6 k
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00
   Ended : Monday, April 8, 2019 5:06:10 PM

You can clearly see the extra directories and files in the source as well as the destination folders. Also, it compares the timestamp and tells you if files are older or newer (compared to the same file in your destination).



robocopy log file list differences folders

In this example, here are the differences spotted:

  1. *EXTRA Dir: Extra directory exists in the destination but not the source.
  2. *EXTRA File: Extra file exists in the destination but not the source.
  3. Older: This file is older compared to the one in the destination.
  4. Newer: This file is newer compared to the one in the destination.
  5. New file: This file does not exist in the destination. It’s also called a Lonely file

Complete list of RoboCopy file classifications:

----------- ----------- ---------------- --------------- ------------- ------------
Classifi_   Exists In   Exists In        Source/Dest     Source/Dest   Source/Dest
cation      Source      Destination      File Times      File Sizes    Attributes
----------- ----------- ---------------- --------------- ------------- ------------
Newer       Yes         Yes              Src > dest      n/a           n/a
Older       Yes         Yes              Src < dest      n/a           n/a
New         Yes         No               n/a             n/a           n/a
Extra       No          Yes              n/a             n/a           n/a
Changed     Yes         Yes              Equal           Different     n/a
Same        Yes         Yes              Equal           Equal         Equal
Tweaked     Yes         Yes              Equal           Equal         Different
Mismatched  Yes (file)  Yes (directory)  n/a             n/a           n/a
----------- ----------- ---------------- --------------- ------------- ------------

Note: A Tweaked file is defined to be one that exists in both the source and destination, with identical size and timestamp, but different attribute settings.

Robocopy is an amazing tool to compare & sync files in two folders (along with sub-folders several levels deep) and list out the differences in a log file.



Synchronize folders using Robocopy

To synchronize folders (only from the left → right folder) with RoboCopy, use the same command-line syntax we used earlier to compare folders, but this time without the /L (list-only) switch.

RoboCopy does a one-way sync (from Left → right folder)

RoboCopy’s Mirror copy mode (/MIR) is similar to Microsoft SyncToy‘s “Echo” feature. Remember, RoboCopy does only left → right synchronization — NOT a two-way synchronization.

It’s important to note that the /MIR switch also removes any extra files and directories in the destination folder tree. The “Extra” files/folders are items present in the destination but not present in the source path. The /MIR (mirror) command makes sure that the destination folder structure (and files) is an exact copy of the source.

Use the following command-line syntax to sync items from left → right folder:

robocopy source_folder destination_folder /MIR /MT

The /MT switch is to do a multi-threaded copy for a super-fast copy operation. It’s an optional switch, but extremely useful.

The sync status is outputted in the console for each item processed, and the statistics are shown at the end of the output.

To log the output to a file, use this syntax:

robocopy source_folder destination_folder /MIR /MT /TS /LOG:D:\Diff.txt

compare contents of two folders - robocopy sync mir

compare contents of two folders - robocopy sync mir

Automate the commands using a Batch file

You can add all the Robocopy commands to Notepad, and save it as a Windows Batch file (with .bat extension). Simply double-click on the Batch file whenever you want to sync the mentioned folders. You may even run the batch file at the required intervals using Task Scheduler.

compare contents of two folders - robocopy sync mir


Method 8: Compare Two Folders using File Hash Checksum with the FCIV utility

Using the cryptographic hashing algorithm — e.g., MD5, SHA-1, you can verify if two files are identical or not. The checksum is a hash value used for performing data integrity checks on files. It’s a kind of signature for a file.

The Microsoft File Checksum Integrity Verifier tool (FCIV) is a command line utility that computes MD5 or SHA1 cryptographic hashes for files. You can output the list of files along with their MD5 or SHA1 hashes to an XML database, and then compare files in the target folder with the hashes stored in the XML database file.

This method is especially helpful after you’ve used one of the above methods — e.g., Robocopy or WinDiff to synchronize two folders, and want to verify the integrity of each destination file by comparing the file hash/checksums with that of the source location.

Scenario:

Let’s compare the contents of the following folders recursively using FCIV:

D:\Websites\Winhelponline\blog\wp-content\uploads
E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads

Download Microsoft File Checksum Integrity Verifier from Official Microsoft Download Center, and extract fciv.exe to a folder — e.g., d:\tools. Then follow one of these methods:

Option 1

  1. Open a Command Prompt window and use this command-line syntax:
    d:\tools\fciv.exe -r -add "D:\Websites\Winhelponline\blog\wp-content\uploads" -bp "D:\Websites\Winhelponline\blog\wp-content" -sha1 -xml d:\hashdb.xml

    compare folder contents - fciv checksum

    The above command-line computes the SHA1 hash for each file recursively and outputs the file names and the corresponding hashes to an .XML file named hashdb.xml.

    The -bp parameter is used to specify the base path to remove from full path name. It ensures that the base path is not included in the XML output so that you can compare the items in the destination folder path having the exact same folder structure.

    Here is how the XML file contents would look like:

    compare folder contents - fciv checksum

  2. Now, in the Command Prompt window, switch to the destination folder by running the following command:
    cd /d "E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content"
  3. Then, run the following command and press ENTER:
    d:\tools\fciv.exe -v -sha1 -xml D:\hashdb.xml

    The above command verifies your destination files against the hash checksums you’ve captured earlier in the XML file hashdb.xml.

    In the output, the list of modified files (based on hash checksum) and the list of missing files (in the destination) are displayed:

    compare folder contents - fciv checksum

    Here is a sample output.

    Starting checksums verification : date/timestamp
    
    List of modified files:
    -----------------------
    uploads\2019\05\w10-enhanced-search-settings.png
            Hash is         : 8dac0a4400d2b700f81bf70c189e49eb6a620a2f
            It should be    : cc2a1523ef070dc2e498dbf5f8cf5edac1b6900d
    
    Unable to Hash file uploads\2019\05\windiff-header.png
    
            Error msg  : The system cannot find the file specified.
            Error code : 2
    
    Unable to Hash file uploads\2019\05\windows-store-error-0x800706d9.png
    
            Error msg  : The system cannot find the file specified.
            Error code : 2
    
    uploads\2019\05\w10-enhanced-search-settings.png
            Hash is         : 8dac0a4400d2b700f81bf70c189e49eb6a620a2f
            It should be    : cc2a1523ef070dc2e498dbf5f8cf5edac1b6900d
    
    Unable to Hash file uploads\2019\05\windiff-header.png
    
            Error msg  : The system cannot find the file specified.
            Error code : 2
    
    Unable to Hash file uploads\2019\05\windows-store-error-0x800706d9.png
    
            Error msg  : The system cannot find the file specified.
            Error code : 2
    
    
    End Verification : timestamp

The above method is very helpful to check the integrity of matching files in the destination folder. But it doesn’t show the list of extra files in the destination. That’s where Option 2 below has an additional advantage.

Option 2

Rather than outputting the files list and checksums to an XML database, you can output the items in the source and destination to separate text files. Then the text files can be compared using WinDiff, WinMerge, DiffChecker.com or WinCompare utility.

Run these commands from a Command Prompt window.

fciv -r "D:\Websites\Winhelponline\blog\wp-content\uploads" -bp "D:\Websites\Winhelponline\blog\wp-content" >d:\source.txt
fciv -r "e:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads" -bp "e:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content" >d:\destination.txt

compare folder contents - fciv checksum

Then, compare the two files source.txt and destination.txt using a file compare tool like WinDiff, WinMerge, DiffChecker.com, DiffMerge or WinCompare.

compare folder contents - fciv checksum

You’ll see the list of missing files, extra files and modified files (based on the computed checksum) in the two folder locations.

I synchronized the two folders using RoboCopy and then compared the folders again using FCIV hash checksum. They’re now identical!

compare folder contents - fciv checksum


Method 9: Compare Two Folders Using PowerShell

Using PowerShell you can recursively compare two folders easily.

Start PowerShell. Copy the following snippet and paste them all in the PowerShell window, and press ENTER

Enter the source folder and destination folders, replacing the source_folder_path and dest_folder_path placeholders respectively.

$Folder1 = "source_folder_path"
$Folder2 = "dest_folder_path"
function Get-Directories ($path)
{
    $PathLength = $path.length
    Get-ChildItem $path -Recurse | % {
        Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.substring($PathLength+1)
        $_
    }
}

Compare-Object (Get-Directories $Folder1) (Get-Directories $Folder2) -Property RelativePath | Sort RelativePath, Name -desc

compare the contents of two folders

You’ll see an output like this:

compare the contents of two folders

The side indicator <= means the file or folder exists only in the source. i.e., missing in the destination.
The side indicator => means the file or folder exists only in the destination. i.e., missing in the source.

Editor’s note: The above code compares only the file names without matching any other details like size, date modified, etc. PowerShell has the capability to do a hash-comparison of items in two folders and output the results, which is described in the next method.

Sync folders

With some 10 additional lines of PowerShell code, you can even copy the different or missing files from source → destination, or vice versa. For more information, see Sync Files And Folders With PowerShell.




Method 10: Compare Two Folders Using PowerShell with File Hash Checksum

To compare files in two locations using file hash checksum, you can use PowerShell. PowerShell by default uses the SHA-256 algorithm, which you can change using the -Algorithm parameter. As said earlier, hash checksum is a kind of signature for a file which is unique.

In this example, we’re going to compare the following folders:

D:\Websites\Winhelponline\blog\wp-content\uploads\2019
E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019

Use the following command-line to compute the hash for files in the source folder and output the results to source.txt file.

Get-ChildItem 'D:\Websites\Winhelponline\blog\wp-content\uploads\2019' -Recurse | Get-FileHash | Select Path, Hash | export-csv d:\source.txt -Delimiter "`t" -NoTypeInformation

Then, run the following command to remove the base paths from the file.

(gc -path d:\source.txt -raw) -replace 'D:\\Websites\\Winhelponline\\blog\\wp-content\\', '' | Out-File d:\source.txt

The list of file names (with relative path) along with the SHA-256 hashes are outputted to the source.txt file.

compare folders using powershell hash

Similarly, run the commands for computing hashes for items in the destination, and then remove the base paths from destinations.txt:

Get-ChildItem 'E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019' -Recurse | Get-FileHash | Select Path, Hash | export-csv d:\destination.txt -Delimiter "`t" -NoTypeInformation
(gc -path d:\destination.txt -raw) -replace 'E:\\RameshPC-Synced Backup\\Websites\\Winhelponline\\blog\\wp-content\\', '' | Out-File d:\destination.txt

The above command computes the hash for each file in the destination folder recursively, and the output is written to d:\destination.txt

Compare using PowerShell

Next, compare the files using the following PowerShell command:

Compare-Object -ReferenceObject $(Get-Content d:\source.txt) -DifferenceObject $(Get-Content d:\destination.txt) | fl

compare folders using powershell hash

It shows the different files and the location of each item denoted by the symbols <= or => for left folder and right folder respectively. The output also shows extra files in the source as well as the destination.

Compare using WinDiff

Alternately, you can run WinDiff and compare d:\source.txt and d:\destination.txt. You can use the following WinDiff command-line to compare the files:

d:\Tools\windiff.exe d:\source.txt d:\destination.txt

compare folders using powershell hash

The WinDiff output shows differences highlighted in yellow/red. From the above screenshot, we can infer that the file 0xc1900101-error.png is different. Also, the destination folder or the right folder (denoted by the symbols !> in WinDiff) has an extra file named 1h.txt.

PowerShell supports many hash algorithms and so it has an advantage over the FCIV.exe tool. The Microsoft FCIV tool supports MD5 and SHA-1 algorithms only, which are outdated and can be ineffective due to collision issues, albeit very rarely. SHA-1 collision means that two files have identical SHA-1 hashes but different content. So, SHA-256 may be the preferred algorithm for file or folder comparison in a production environment.


Note that the command-line methods 11 & 12 below only help you compare the structure of two directories along with files. They don’t compare file sizes or modified date to know which is the newer. They only do a plain comparison based on file names and you wouldn’t know if one of the destination files becomes corrupt during a copy operation. In a production environment, you may need to use other tools or methods if you need to verify the integrity of your files in the source and destination folders, to make sure they are the exact same copies.

Method 11: List the folder structure using TREE command

To compare two directories along with sub-directories, the good old TREE command comes extremely handy. Suppose you want to compare the two folders (including file names) D:\SOURCE and D:\DEST, follow these steps:

  1. Open a Command Prompt window.
  2. Type these commands:
    TREE  D:\SOURCE /A /F >D:\SOURCE.TXT
    TREE  D:\DEST /A /F >D:\DEST.TXT

    This outputs each of the command results to separate text files in your D:\ drive’s root directory. The text files will have your directory structure in a tree format, which makes it easy for comparison work.

    Quick Tip: To compare only the directories, remove the /F flag.
  3. Next, fire up your text comparison utility like WinDiff, WinMerge or DiffMerge. Alternately, you can compare the two text files online via diffchecker.com.
  4. Go to DiffChecker.com and paste the contents of d:\source.txt in one column, and D:\DEST.txt in another column. Click Find differences.
    compare two folders and files list
    Directory comparison

    compare two folders and files list
    Directory and files comparison


Method 12: List folder structure using the DIR Command

To compare the contents of two directories, another command-line you can use is the DIR command.

Use the following command-line syntax to generate the list of files in a directory recursively. You can then compare it using a compare tool or DiffMerge.

DIR D:\SOURCE /S /AD /B >D:\SOURCE.TXT
DIR D:\DEST /S /AD /B >D:\DEST.TXT
Quick Tip: To include files along with the directory listing, remove the “/AD” flag.

There is one problem though, when using the DIR command. It lists full directory and file paths in the output, as below:

  1. D:\Source\Apr\Week1
  2. D:\Source\Apr\Week2
  3. D:\DEST\Apr\Week1
  4. D:\DEST\Apr\Week2

In that case, you’ll need to remove the comparison base paths D:\SOURCE\ and D:\DEST\ in the respective text files by editing them with Notepad. To bulk replace text, use the Replace option in Notepad and remove every occurrence of the base path.

Once done, save the files and compare the contents online via DiffChecker or using an offline text-comparison tool like WinDiff or WinMerge.

compare two folders and files list
Directory comparison – List generated using DIR instead of TREE

I bet you’ll find directory contents output generated using TREE to be much easier and far neater than DIR.


Hope the above methods helped you quickly compare the contents of two folders recursively, identify missing and modified files and reconcile differences.

Let’s know your comments!


One small request: If you liked this post, please share this?

One "tiny" share from you would seriously help a lot with the growth of this blog. Some great suggestions:
  • Pin it!
  • Share it to your favorite blog + Facebook, Reddit
  • Tweet it!
So thank you so much for your support, my reader. It won't take more than 10 seconds of your time. The share buttons are right below. :)

About the author

Ramesh Srinivasan founded Winhelponline.com back in 2005. He is passionate about Microsoft technologies and he has been a Microsoft Most Valuable Professional (MVP) for 10 consecutive years from 2003 to 2012.

9 thoughts on “How to Compare the Contents of Two Folders and Synchronize them”

    • @Bob: WinDiff? It works fine in Windows 10. Can you post the exact error message you’re seeing?

  1. Great article! Very thorough and lots of options for different skill sets. Thanks for including screens of the different interfaces as well as the examples of the outputs.

    Reply
  2. I run a phpBB board, where the {root}/files directory for attachments has grown to where it exceeds the file count limit imposed by my ISP on lftp (8000 files). I can no longer backup my site with integrity using lftp to mirror my site to my home Windows computer.

    I can run a cron job on the server, that will list the entire contents of the …/files directory. And, I can automate a DOS script that will fetch that file via ftp; and, I can produce a local directory listing. A daily comparison between the two would reveal the files I have not yet ftp’d, which I could then do, again with the DOS script. What I lack, is a way to automate the comparison of the remote and local files to render those files I am missing. Unless I missed it, the examples in this tutorial describe a way to do that via a GUI, not an automated procedure. Can you advise? Thanks in advance… -Dan

    Reply
  3. Very good and thorough article. I tried WinDiff first but somehow it did not work for me. I used WinMerge and it did the job. The explanation with screenshot and all is very well done. Thank you.

    Reply

Leave a Comment