Welcome Guest.   Make a donation to an author on the site October 20, 2014, 03:06:53 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Your Support Funds this Site: View the Supporter Yearbook.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 3 4 5 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: photo duplicate scanner  (Read 44071 times)
Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« on: January 14, 2008, 11:46:50 AM »

Between my wife and I, we take lots of pics using our nice little point-n-shoot Panasonic Lumix.
Living in a new city, new friends, many new sites to see, and a new baby girl (5 months). So about once a week we transfer all our photos and many .MOV files from our SD cards to our desktop.

Usually, files would retain the cameras default naming system. EX: P1010368.jpg, or P1000952.mov.
But for the past few weeks, my naming system is messed up. Now I have dozens, if not hundreds of hi-res photos that are duplicates (average size of 2-3MB per photo).

I'm not sure if this is something caused by my constant app install/uninstalls or not, but I think it can be solved.
Rather than manually scan through my photos, I recall having seen or heard of an application that does it automagically? and removes/moves/renames/copies duplicate photos?

If there is an app like this, what are its features?!
hhmm ... I just came from this thread. But it didn't offer anything I need.
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
tomos
Charter Member
***
Posts: 8,603



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #1 on: January 14, 2008, 12:08:04 PM »

I used to have one called similar images finder but I never used it undecided
so I googled -
similar images finder

one out of the first three listed is free, one has a trial, I didnt look further

if you do try any of them let us know how you get on!
Logged

Tom
jgpaiva
Global Moderator
*****
Posts: 4,710



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: January 14, 2008, 12:11:40 PM »

From what i understand, you don't need a "similar images finder", all you need is a "duplicate file finder", which is included in serveral alternative file explorers (like dopus) or, alternativelly, you could use a standalone tool, like this one.

disclaimer: i've never used dupfilefinder. It does look good, though tongue
Logged

tomos
Charter Member
***
Posts: 8,603



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #3 on: January 14, 2008, 12:14:41 PM »

From what i understand, you don't need a "similar images finder", all you need is a "duplicate file finder", which is included in serveral alternative file explorers (like dopus) or, alternativelly, you could use a standalone tool, like this one.

disclaimer: i've never used dupfilefinder. It does look good, though tongue

would that find them if they have different names - dont think dopus would do that
would have to compare all files for same size and then do a bit comparision
Logged

Tom
iphigenie
Supporting Member
**
Posts: 1,166


curiosity FTW!

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #4 on: January 14, 2008, 12:15:34 PM »

the ones that deal with images specifically have the nice feature of showing potential matches side by side, with a quick "yes/no" way to cycle and delete them.

Comparison usually starts with size, meta information (image info) then binary checksum
Logged
iphigenie
Supporting Member
**
Posts: 1,166


curiosity FTW!

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: January 14, 2008, 12:17:01 PM »

I think you could also use a rename-images-based-on-meta-information feature, i.e. use exif date/time to organise all the images, making spotting duplicates easier.
Logged
nosh
Supporting Member
**
Posts: 1,388


View Profile Give some DonationCredits to this forum member
« Reply #6 on: January 14, 2008, 01:30:25 PM »

Easy Duplicate Finder is free and it's also the fastest dupe finder I've used. From the info you provided it seems perfect for the job.
http://www.easyduplicatefinder.com

DoubleKiller is a more advanced dupe finder
http://www.bigbangenterprises.de/en/doublekiller/

If you're looking for an image dupe finder, something that finds dupes even if pics have been resized, try Unique Filer
http://www.uniquefiler.com/


Logged
Falcon_four
Supporting Member
**
Posts: 8

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #7 on: January 14, 2008, 01:51:52 PM »

One I've found, although it's not free: PhotoSort <http://www.asmdev.net/products/photosort/>

I use this myself and it is quite good - it has support for various RAW images and is quite easy to use.

>>>>F-4>>>>

Logged

>>>>F-4>>>>
katykaty
Supporting Member
**
Posts: 221


View Profile Give some DonationCredits to this forum member
« Reply #8 on: January 14, 2008, 07:31:10 PM »

I use Duplicate File Finder by Rashid Hoda which has plenty of options and will find duplicates even if they have different names and/or are in different folders.

See screengrab attached.

Development seems to have ceased though, his website is down and has been for months, and it's tricky to find on the web.

I found a download link to version 1.1.0.2 - I use 1.1.0.3

It is/was freeware but I don't think it was open source - I wonder when it starts to count as abandonware?

I could post the installer but I'd probably be in breach of the license, even if it was freeware and the developer awol.


* Screenshot - 15_01_2008 , 01_16_42.gif (71.87 KB, 973x741 - viewed 470 times.)
Logged
PhilB66
Supporting Member
**
Posts: 1,510


View Profile Give some DonationCredits to this forum member
« Reply #9 on: January 14, 2008, 07:47:00 PM »

Duplicate File Finder (Freeware, Version 1.1.0.3, Direct Download). Found on this page http://web.archive.org/we...http://dff.nazrashid.com/
Logged
Kimchee53
Participant
*
Posts: 4

View Profile Give some DonationCredits to this forum member
« Reply #10 on: January 14, 2008, 09:35:28 PM »

D'Peg - by somewareonthe.net - I think. Have used D'Peg for many years, and it has improved continually. It now also handles movies & sound files. IMHO it has about a 98% acurracy (did I spell that right?) rate. It's a bit tricky to get the settings for what you want it to do (on first try), but author is VERY helpful (as is the help built in). Price is pretty reasonable, and sometimes you'll get another program as a "gift" from the author (not always).

Hope this helps a bit.

http://www.somewareonthe.net/dpeg.htm

link to site is above.
Logged
Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #11 on: January 14, 2008, 10:36:51 PM »

Thanks for the many many suggestions everyone!
I'll give them each a try, and see where I get!
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
mwb1100
Supporting Member
**
Posts: 1,329


View Profile Give some DonationCredits to this forum member
« Reply #12 on: January 15, 2008, 01:20:18 PM »

In a few days Bits du Jour will be having an offer on some software called "Image Comparer" which claims to be able to find similar photos - not just exact file copies.  I have no idea how well it works, I only noticed the advertisement:

http://www.bitsdujour.com...index.aspx%3freview%3d161

So for anyone who is looking to cull out a photo library, this might be an interesting bit of software to test.
Logged
MikeMcLoughlin
Supporting Member
**
Posts: 31

View Profile Give some DonationCredits to this forum member
« Reply #13 on: January 16, 2008, 05:41:36 AM »

jammo

I've tried many and the best one I've come across is DupDetector, and it's free.

It was/is from a company called Prismatic Software

http://www.prismaticsoftware.com

Although it looks like they sadly no longer exist.

It's worth Googling to see if you can find it elsewhere or I'll gladly email it to you.

It's not very pretty to look at but works really well.
Logged
PhilB66
Supporting Member
**
Posts: 1,510


View Profile Give some DonationCredits to this forum member
« Reply #14 on: January 16, 2008, 07:11:26 AM »

jammo

I've tried many and the best one I've come across is DupDetector, and it's free.

It was/is from a company called Prismatic Software

http://www.prismaticsoftware.com

Although it looks like they sadly no longer exist.

It's worth Googling to see if you can find it elsewhere or I'll gladly email it to you.

It's not very pretty to look at but works really well.

Dup Detector freeware version 3.201 still available via this link:

http://web.archive.org/we....com/Utility/Utility.html

Here's a screenshot and a brief description:



"Dup Detector finds duplicate and near duplicate images by opening and reading image pixel data. Large collections and large image file sizes can slow Dup Detector but it works well running in the background. Large image collections often are filled with duplicate images under different names. Use Dup Detector to clean them. DupDetector reads 9 image file types from a folder or its subfolders or from an image list."
« Last Edit: January 16, 2008, 07:17:43 AM by PhilB66 » Logged
J-Mac
Supporting Member
**
Posts: 2,867


see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: January 17, 2008, 03:04:51 AM »

I use Duplicate Finder by Ashisoft Software.  Very nice - I've used it for a few years now.  Uses several different methods to search for duplicates and lets you view them in thumbnails to compare.

Jim
Logged

"I am getting so tired of slitting the throats of people who say that I am a violent psychopath."
lanux128
Global Moderator
*****
Posts: 6,108



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: January 19, 2008, 12:37:18 PM »

yet another tool for this task, Finddupe from the makers of Jhead.

Finddupe is a tool for quick detecting of duplicate files on a hard drive under Windows. Duplicate files can be just detected, hardlinked, or deleted.

Finddupe has several possible uses:

Deleting duplicate files
When working thru somebody else's photo collection, or MP3 collection, this tool is useful for deleting the files that are duplicate. Depending on how the media is organized, there can be a lot of duplicate files in a collection.
http://www.sentex.net/~mwandel/finddupe/
Logged

Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #17 on: January 19, 2008, 10:28:06 PM »

"Dup Detector finds duplicate and near duplicate images by opening and reading image pixel data. Large collections and large image file sizes can slow Dup Detector but it works well running in the background. Large image collections often are filled with duplicate images under different names. Use Dup Detector to clean them. DupDetector reads 9 image file types from a folder or its subfolders or from an image list."
THANKS ALL!

This is the one I chose to use for my needs.
I really liked how it displays the 'suspected duplicate' next to the original.

Now I just need to find a method to RENAME all my files by the creation date. Something that reads the EXIF data?
I hate DOpus, if anyone knows how. I use DOpus as a newbie, haha, I know it has super powerful features. I'm reading up on it right now, see if that's something it can do for me.
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #18 on: January 19, 2008, 11:06:21 PM »

*AMEND TO ABOVE*

After a ONE MINUTE SEARCH, hahaha, silly me, right?
I went to LifeHacker and found this article Namexif Batch Renames Digital Photos by Date

But as always, I read the comments and came across:  JoeJoe's freeware utilities, Rename Master.
It worked like a charm.

I didn't want JUST a renamer for EXIF data, but an all-around EVERYthing renamer, and this fits the ticket!
smiley
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
tomos
Charter Member
***
Posts: 8,603



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #19 on: January 20, 2008, 04:04:54 AM »

Now I just need to find a method to RENAME all my files by the creation date. Something that reads the EXIF data?
I hate DOpus, if anyone knows how. I use DOpus as a newbie, haha, I know it has super powerful features. I'm reading up on it right now, see if that's something it can do for me.

jammo
[edit] got my wires crossed - see edit below [/edit]
have a look at this (but read following post too - small bug here but I would still recommend it)

here are two more tools that will be useful in this task.

http://www.sentex.net/~mwandel/jhead/
http://jpgtime.learsy.com/
that Learsy one is a thing of beauty Thmbsup
 (see attachment in previous post)
# no install
# simple to use -
  • select files (all if you want) / click Set time from EXIF - it show the Exif time in the "New Time" column
  • compare
  • select those you want to change
  • click Apply changes

then go off and donate to the author (donationware thumbs up)
http://www.muralpix.com/donation.html

BIG EDIT:- embarassed this one here changes modified date back to creation/shooting date -
doesnt change the name - sorry bout that confusion smiley
« Last Edit: January 26, 2008, 03:23:16 AM by tomos » Logged

Tom
PhilB66
Supporting Member
**
Posts: 1,510


View Profile Give some DonationCredits to this forum member
« Reply #20 on: January 20, 2008, 06:12:13 AM »

Now I just need to find a method to RENAME all my files by the creation date. Something that reads the EXIF data?

AmoK Exif Sorter

Features:

+ Fast renaming of pictures based Exif- and IPTC data
+ Support of all Exif / IPTC data ( date picture taken, year, month, week, day, hour, second, size of photo, camera model, camera manufacturer, exposure, aperture, original filename, description, place, keywords, and many more)
+ Easy definition of rules for renaming via Tags (%year%, %month%, %day%, ...)
+ Photos may be renamed, moved or copied
+ Preview of new filenames in real-time
+ Assistant for renaming photos
+ Processing of single files or directories including sub directories
+ Integrated picture viewer
+ Handles thousand of pictures at once
+ Drag & Drop support
+ Automatic update check
+ Profiles for different cameras, users and renaming rules
+ 40% faster than AmoK Exif Sorter 1.x
+ Independent of operating system (runs on Windows, Linux and Mac)
+ Free English support via email
+ 100% free for private and commercial use
+ With and without setup/installer available

Screenshots
Logged
Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #21 on: January 20, 2008, 11:41:48 AM »

So far, I'm sticking to using JoeJoes Renamer for everyday usage. It works well with my methods of categorizing my photos.

But now, I've got a new dilemma: my camera outputs video files as *.mov
And as pointed out by my original post, the file names are all wonky. Is there any way to batch rename all based on creation date? There is no exif type data on these.
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
4wd
Supporting Member
**
Posts: 3,345



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #22 on: January 23, 2008, 03:17:44 AM »

But now, I've got a new dilemma: my camera outputs video files as *.mov
And as pointed out by my original post, the file names are all wonky. Is there any way to batch rename all based on creation date? There is no exif type data on these.

You mentioned DOpus above, so I'll assume you have it.

The easy way:

1) Select files/movies/etc.
2) Hit the Rename button.
3) Hit the Advanced button - assuming it's in Simple mode.
4) Tick 'Enable file information fields' - you'll now notice a selection button on the right side
    of the 'New name' input field.
5) Hit that selection button, (a down arrow), and select 'General->Date (Created)'.
6) The new filenames will be displayed in the preview area - hit the OK button.
7) Job Done.

The 'Enable file information fields' also gives you access to a lot more renaming info, including EXIF data.

Cheers,
4wd (ex Amiga DOpus beta-tester smiley
« Last Edit: August 10, 2010, 07:31:09 PM by 4wd » Logged

I do not need to control my anger ... people just need to stop pissing me off!
brahman
Supporting Member
**
Posts: 132


View Profile Give some DonationCredits to this forum member
« Reply #23 on: January 26, 2008, 12:14:31 AM »

Hello,

my problem is similar to the threadstarter, yet needs a very different approach:

I often rename video etc. files and then still have the unrenamed files on my backup drive.

I use SynchronizeIt! (great app from grigsoft.com - highly recommended) to keep my backups in synch.

However, after my renaming, it wants to copy the entire file and delete the same unrenamed file on the backup volume, because it cannot detect that I renamed the file. This can be very lengthy on gigabyte sized files.

So now I am searching for an app, which does not only find dupes and deletes them, but which I can tell: compare these two folders, the first folder is source, the second target, files have different names but same time and size (so no need to do lengthy byte by byte or checksum compare) and rename all same files from target name to source name.

Has anybody found something like this?

Regards,

Brahman
Logged

Regards, Brahman
Jammo the OrganizedFellow
Supporting Member
**
Posts: 212


see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #24 on: January 26, 2008, 01:55:14 AM »

You mentioned DOpus above, so I'll assume you have it.

The easy way:

1-7
smiley
THanks for that info.
I tried that too. That will work well for further organizing of my files.
Might even eliminate my file renamer that I downloaded from JoeJoe software!
Logged

As an aspiring web developer/designer, it is a constant struggle to cope with my ADHD + Hypomania/Bipolar Disorder.

The slow growth of my web dev projects is eclipsed by my patience, understanding and desire to learn AS MUCH AS POSSIBLE as I slowly progress.

X_____jamjammo_____
Pages: [1] 2 3 4 5 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.063s | Server load: 0.11 ]