topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 2:34 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Create a list of duplicate files in a group of folders but not remove anything  (Read 3265 times)

questorfla

  • Supporting Member
  • Joined in 2012
  • **
  • Posts: 570
  • Fighting Slime all the Time
    • View Profile
    • Donate to Member
Another day, another Wild Goose Chase. 

Today I was asked to create the above from a folder containing about 50,000 files.  Many of those files are probably duplicates but I cannot just delete them even when I find them as they are the resource pool for a DB.  I just need to find out how many of them are actually in there twice or even three or four times and create a list showing the paths to all of them.

Someone else (TG!)  not me .. is going to have to figure out which ones to keep and which not.

But I have to create this list showing the full path to all the files that exist in more than one location but are really duplicates.
In the Best of Worlds, this would be done using a hash of the document and not just the filename because it is entirely possible, even likely, that the same file was entered by two different people at different places.  Yet that file would still be a duplicate meaning that the BB links should show a single location for both entries instead of having the file in there twice.

I have been testing a few duplicate finders but most of them are a bit over zealous and want to help get rid of the problem.  In this one case, I only need to create a file showing the path listings to all the duplicates but Do Nothing.  Just make the list. 
I wondered if anyone had any suggestions of a particular tool to use for that.  Hopefully something you have used before for a similar task?

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,641
    • View Profile
    • Donate to Member
Duplicate File Finder by Rashid Hoda - a bit old now but still works fine.

2015-01-20 15_14_06.pngCreate a list of duplicate files in a group of folders but not remove anything

2015-01-20 15_15_16.pngCreate a list of duplicate files in a group of folders but not remove anything

It's also portable if you copy the files somewhere after installation, and then uninstall it.

yksyks

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 476
    • View Profile
    • Donate to Member
CloneSpy has an option to Export list of all equal files.

questorfla

  • Supporting Member
  • Joined in 2012
  • **
  • Posts: 570
  • Fighting Slime all the Time
    • View Profile
    • Donate to Member
Perfect!  And many thanks for the replies. :Thmbsup: :Thmbsup: