Welcome Guest.   Make a donation to an author on the site July 30, 2014, 10:27:28 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
The N.A.N.Y. Challenge 2013! Download dozens of custom programs!
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 3 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: NANY 2011 Release: Duplicate Photo Finder  (Read 27063 times)
Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« on: December 20, 2010, 07:28:55 PM »

NANY 2011 Entry Information

Application Name Duplicate Photo Finder
Version 1.3
Short Description Find and delete duplicate photos.
Supported OSes Windows 2000/XP/
Web Page Duplicate Photo Finder
Download Link Download Installer - Download Source Code
System Requirements
  • Windows XP Home or higher / Windows 2003 / Windows Vista / Windows 2008 / Windows 7
  • .NET 4.0
  • 200 MB of free RAM
Version History
  • v1.3 - Update - Increased speed in edge case, now impossible to delete 'originals' in edge cases
  • v1.2 - Update - Improved hashing methods
  • v1.1 - Update - Source released
  • v1.0 - Initial release
Author Renegade


Description
Duplicate Photo Finder is a photo file management tool to find photos by folder and let you delete duplicates. It aims to be minimalistic with no significant learning curve.


Features
Duplicate Photo Finder:

  • Lets you choose an "original" and "duplicate" folder to search in
  • Finds duplicates by file size, file signature (file hash), or photo signature (pixel data hash)
  • Lets you delete individual photos or many photos at once
  • Deletes to the Recycle Bin (just in case your brain falls out and you delete the wrong photos)
  • Only JPG files are scanned
  • Has no complex features to confuse you

Duplicate Photo Finder is for "photos", and not for other graphics or files. As such, it only scans JPG files because digital cameras take JPG photos and not PNG or GIF or XLS or DOC... Wink

It is also aimed at regular people and hobbyists, and not professional photographers. As such, there is no RAW image support and no professional-type features, e.g. no undelete, no database support. Again, this is aimed at being minimalistic and easy to use.


Planned Features
Planned features include speed optimizations, multi-threading for additional speed, and other image comparison methods (exact image comparisons).


Screenshots



Yes. That is me and my wife~! cheesy

And yes, that is me sticking my tongue out~! tongue 2x~! tongue


Usage
Installation
Installation is trivial. Run the installer and follow the directions. The application is installed to:

<drive>:\Program Files (x86)\Renegade Minds\Duplicate Photo Finder

Running the application the first time has no surprises and there is no configuration required.  


Using the Application
The installer puts an application shortcut under the Start Menu. i.e. Start > Duplicate Photo Finder. There is no desktop shortcut unless you create one yourself.

1) Browse to 2 folders using the folder browsers.
2) Click the "Compare" button.
3) Decide which photos you want to delete.
4) Click a delete button.
5) Repeat steps 1~4 for other folders that you wish to compare images in.

TIP 1: Use the "Simple" mode first, then do the more thorough "Image Signature" scan afterwards. It will save you time and produce the best results.

TIP 2: Take your time. The program is relatively idiot-proof, and deletes photos to the Recycle Bin, so you can still retrieve deleted photos. However, this is only proof up to "Idiot v5.0"; please do not try to upgrade yourself to "Idiot v6.0" as you will win~! cheesy


Uninstallation
Go to the Windows Control Panel applet for uninstalling programs. Click the program's entry, then click "Uninstall".


Known Issues
The application does not make use of threading/multi-threading, and as such requires that the user wait for tasks to complete before starting a new task. Multi-threading is planned.

There is no help file, documentation, or credits yet.

While the program deletes duplicates to the Recycle Bin, and they can be retrieved, there is no substitute for not doing stupid things in the first place. Wink Remember, this is a bulk deleting utility and can delete large swaths of data quickly. Take your time. There's no rush. smiley




Enjoy~! cheesy
« Last Edit: January 24, 2014, 09:24:39 PM by Renegade; Reason: Updated links. » Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
skwire
Charter Member
***
Posts: 4,021



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #1 on: December 20, 2010, 07:49:00 PM »

That's perfect for a NANY.   thumbs up
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #2 on: December 20, 2010, 08:21:06 PM »

That's perfect for a NANY.   thumbs up

Cool.

I'll get stuff sorted shortly. I need to run out the door to get that new mouse now. smiley

I've setup a page for Duplicate Photo Deleter. (Again, need to clean that, but it's up.)
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: December 20, 2010, 10:10:58 PM »

 embarassed Not NANY 2011???
« Last Edit: December 20, 2010, 10:25:37 PM by Perry Mowbray » Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #4 on: December 20, 2010, 11:28:30 PM »

embarassed Not NANY 2011???

Yeah... Can you move the thread?
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: December 21, 2010, 11:54:16 PM »

All done!
Logged

Deozaan
Charter Member
***
Posts: 6,270



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #6 on: December 22, 2010, 12:31:18 AM »

CARP! I wish I'd known about this yesterday. I just moved 3,000+ photos into the same directory on someone's computer because they were spread all over, and there were plenty of duplicates, but sometimes the filesize didn't match. So instead of going through every one comparing filesize and timestamp, I just had TeraCopy automatically rename the naming conflicts.

Is there any way this app can find duplicates if they don't have exactly the same name?  e.g. PIC-1028.jpg & PIC-1028_2.jpg.
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #7 on: December 22, 2010, 12:39:50 AM »

CARP! I wish I'd known about this yesterday. I just moved 3,000+ photos into the same directory on someone's computer because they were spread all over, and there were plenty of duplicates, but sometimes the filesize didn't match. So instead of going through every one comparing filesize and timestamp, I just had TeraCopy automatically rename the naming conflicts.

Is there any way this app can find duplicates if they don't have exactly the same name?  e.g. PIC-1028.jpg & PIC-1028_2.jpg.

At the moment I'm only comparing file sizes. File names are irrelevant and I don't look at them at all.

I plan on adding in better comparisons by loading the actual image then comparing that data.

I won't be doing anything like ratios in image comparison, e.g. these pictures are 80% similar. I just think that's a stupid comparison method that's completely unreliable. e.g. You take a picture of someone you are with against a backdrop  of some nice scenery, then they take your picture. It's a small amount of difference, but it's important. Since the amount of coding to do facial recognition and all that is really far beyond what I'm prepared to put in right now, I'm not even going to try because I don't see any point in doing a crappy job. 

The software right now is stable, but alpha (feature incomplete).
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
skwire
Charter Member
***
Posts: 4,021



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #8 on: December 22, 2010, 12:02:45 PM »

Is there any way this app can find duplicates if they don't have exactly the same name?  e.g. PIC-1028.jpg & PIC-1028_2.jpg.

You can easily do this using file checksums.
Logged

Deozaan
Charter Member
***
Posts: 6,270



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: December 22, 2010, 12:18:27 PM »

Is there any way this app can find duplicates if they don't have exactly the same name?  e.g. PIC-1028.jpg & PIC-1028_2.jpg.

You can easily do this using file checksums.

How do you easily compare 3,000+ checksums?
Logged

skwire
Charter Member
***
Posts: 4,021



Another Coding Snack request? Om nom nom...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #10 on: December 22, 2010, 12:20:46 PM »

Compute them, build a list, report duplicates.   cheesy
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #11 on: December 30, 2010, 05:48:59 AM »

<clears throat /> Ahem...

Ooops.

Got a memory issue popping up that affects large images and don't know if I'll be able to solve it in time.

Sorry... I think this is dead.
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: December 30, 2010, 08:33:50 PM »

It's never dead until it's dead... but I changed the title to suit  Wink
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #13 on: December 30, 2010, 08:46:21 PM »

It's never dead until it's dead... but I changed the title to suit  Wink

Still working on it... Managed to hack around one super-nasty issue, but with what I want to do in there, it's tough. (IO is a problem as I'm taking an entirely bulk approach and working with my own photos, which are all 10 megapixels, so it's GB at a time and even with optimized super-fast hashing it's slow.)
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: December 30, 2010, 08:52:19 PM »

Still working on it...

 Thmbsup  Just change the title and the OP to suit if it gets to Release
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #15 on: December 31, 2010, 02:56:01 AM »

Made it in time. smiley

Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: December 31, 2010, 03:05:50 AM »

Made it in time. smiley

 Thmbsup
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #17 on: December 31, 2010, 11:04:55 PM »

I've updated the software to v1.1.

Changes:

* Image previews now 25~35% faster
* Image load times displayed
* New JPG icon for images
* Selected folders now highlighted in folder browsers
* Press F1 for a nice "About" screen smiley
* Refactored some code
* Added massive amounts of comments to code, complete with stupid humor :p
* Released source code YAY~!

I've commented the source extensively for anyone that wants to have a look or perhaps learn some stuff in C#.

Please note that I've done some very bad things in there (e.g. Application.DoEvents() to maintain a responsive UI). I know that. However, in the interests of getting things done I had to sacrifice in places. e.g. No threading there.

However, there are enough comments there and the really bad things are mostly marked.

Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: January 01, 2011, 08:45:54 AM »

If you're going to continue adding features... one I'd like to see is the ability to test just the image data, not the whole file (which will include EXIF data too)... once I've geotagged my images it messes the matching up  Sad
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #19 on: January 01, 2011, 11:04:42 AM »

If you're going to continue adding features... one I'd like to see is the ability to test just the image data, not the whole file (which will include EXIF data too)... once I've geotagged my images it messes the matching up  Sad

The current feature set hashes the image, and not the file.

Here's the source:

Formatted for C# with the GeSHI Syntax Highlighter [copy or print]
  1. List<UInt32> hash1 = new List<uint>();
  2. SuperFastHashUnsafe sf1 = new SuperFastHashUnsafe();
  3. Bitmap bmp1;
  4. ImageConverter converter = new ImageConverter();
  5.  
  6. ...
  7.  
  8. bmp1 = (Bitmap)Bitmap.FromFile(fi1.FullName);
  9. byte[] bytes = new byte[1];
  10. bytes = (byte[])converter.ConvertTo(bmp1, bytes.GetType());
  11.  
  12. // Add the has value to the list.
  13. hash1.Add(sf1.Hash(bytes));

And the relevant MSDN article:

http://msdn.microsoft.com...awing.imageconverter.aspx

If I'm not mistaken, only the pixel data is considered, which results in a safe comparison.

I COULD be mistaken though.

If you could send me 2 files to test, I'd appreciate it.



Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #20 on: January 01, 2011, 11:05:58 AM »

Oh, I should mention that's for the second comparison method. The first is file size only. Check the top combobox there.
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,800



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #21 on: January 01, 2011, 11:08:57 AM »

I thought I did it right... but didn't match (can you remind me later for the files?)
Logged

Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #22 on: January 01, 2011, 12:20:28 PM »

I thought I did it right... but didn't match (can you remind me later for the files?)

It matches folders only. Not individual images... Design flaw considering the UI. I'll need to update that. But I will remind you tomorrow to send me the pics.
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
mouser
First Author
Administrator
*****
Posts: 33,184



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: January 01, 2011, 05:17:26 PM »

Written up on ghacks Thmbsup

http://www.ghacks.net/201...1/duplicate-photo-finder/
Logged
Renegade
Charter Member
***
Posts: 10,890



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #24 on: January 01, 2011, 09:29:09 PM »


Cool! That's a pleasant surprise~! cheesy
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
Pages: [1] 2 3 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.065s | Server load: 0.03 ]