Author Topic: photo duplicate scanner (Read 139,164 times)

J-Mac · « **Reply #25 on:** January 27, 2008, 04:38 AM »

Quote from: 4wd on January 23, 2008, 09:17 AM

...You mentioned DOpus above, so I'll assume you have it.

The easy way:

1) Select files/movies/etc.
2) Hit the Rename button.
3) Hit the Advanced button - assuming it's in Simple mode.
4) Tick 'Enable file information fields' - you'll now notice a selection button on the right side
of the 'New name' input field.
5) Hit that selection button, (a down arrow), and select 'General->Date (Created)'.
6) The new filenames will be displayed in the preview area - hit the OK button.
7) Job Done.

The 'Enable file information fields' also gives you access to a lot more renaming info, including EXIF data.

Wow! I knew there was a rename command within DOpus but never really looked into it. I already had both Renamer and Bulk Rename Utility when I purchased DOpus and so had not looked into the DOpus rename facility. I haven't had need for renaming a large number of files lately, but I have been looking at Bulk Rename Utility, even for small numbers of file renames, just to try to get used to the interface. It has quite a daunting interface - so much so that I would normally have discarded it by now, figuring if the developer hadn't seen fit to make a program easily usable, then I wouldn't bother. But the author does actually apologize for this on his site. Apparently he developed it for himself originally and never thought to make it user-friendly for others. But he has made it available to all for free, and has a good tutorial for it. So I was using it lightly to learn, since so many DC members give it high praise. Also, most of my renaming is digital photos and I use ACDSee's renamer for them, as it is pretty good itself and I am usually already in ACDSee when I decide to rename them.

But now that I read your post and just tried the DOpus renamer, I will definitely give it a workout. Looks to be a lot more comprehensive than I had realized!

Thanks!

Jim

4wd · « **Reply #26 on:** February 03, 2008, 09:15 AM »

Quote from: J-Mac on January 27, 2008, 04:38 AM

But now that I read your post and just tried the DOpus renamer, I will definitely give it a workout. Looks to be a lot more comprehensive than I had realized!

Also remember that you can save any rename pattern you use regularly. I'll assume you have now converted to using the DOpus Advanced Rename requester

Just set up the rename pattern for photos/files/whatever how you want it, then:

1) Hit the Presets: Add button (above the list on the right).
2) Give your new rename pattern a name.
3) It appears in the list.

You can now just select files, select the preset from the list on the right and Robert's your father's brother

Also, for a preset you use all the time:

1) Create a new lister toolbar button.
2) Give it the function 'Rename PRESET=<insert your preset name>'
3) Select some files, hit your new button.

4wd

ashwken · « **Reply #27 on:** February 05, 2008, 02:46 PM »

Just thought I'd throw this one into the ring:

Lupas Rename
http://rename.lupasfreeware.org/lupasrename.php

From this page:
Lupas Rename is a FREEWARE program developed to rename a big number of files.
It works on Win95, Win98, WinME, WinNT, Win2K and WinXP.
It is a simple .EXE file and doesn't need any other external libraries.
==========

I'm not running Vista, so I don't now how it'll behave here.

Granted the last release was 2005-04-01...

It's has a very comprehensive set of rename functions, supports audio and EXIF tags, good depth for renumbering, and a clear and concise interface.

PhilB66 · « **Reply #28 on:** May 22, 2008, 11:38 PM »

VisiPics is another duplicate images finder that belongs in this pack (via ghacks).

[attach]

compn · « **Reply #29 on:** October 05, 2008, 01:46 AM »

doublekiller is pretty good. double killer pro is faster
visipics is great but slow.

most of the duplicate file programs i've used get choked up when dealing with huge amounts of files (e.g. 40gb folder with millions of images).

most (all?) do not have the ability to save a database of hashes. so you can save time in the future.

many have options to select multiple dup files, but the filters can be confusing or not as specific as you need them to be.

i guess what i'm saying is... we need a 'comparison of duplicate file finders'

filters:
filter results by directory
filter results by size/type/name/lackofname (select numberd filenames only)
filter results by drive/folder

scan:
selective hashing (scan first and last 10 bytes of a file)
multithreaded
same size sorting for faster scan

interface:
side by side compare
delete to recycle bin
move dupes to selected folder
good status/progress indicators

Curt · « **Reply #30 on:** October 06, 2008, 05:40 PM »

I wonder why I didn't see this thread the first time it was around? Never mind; I didn't. I have been using Image Comparer for a year or so, and I like it a lot, and believe it fulfils ALL of the requested features. In fact it is hard for me to imagine anyone would want more features! Okay okay, we all want more features, but, you know what I mean: Image Comparer is REALLY well equipped...

$35 desktop version: http://www.bolidesoft.com/imagecomparer.html
$149 command line version: http://www.bolidesoft.com/imagecomparer/commandline.html

Quote

Get a free Image Comparer™ license. Just find any bugs or any mistakes in grammar, spelling, or style in this site or in the software's user interface.

compn · « **Reply #31 on:** October 09, 2008, 04:24 AM »

Quote from: Curt on October 06, 2008, 05:40 PM

I have been using Image Comparer for a year or so, and I like it a lot, and believe it fulfils ALL of the requested features.

thanks for the tip. the more programs to choose from, the better!

it looks like this program has trouble with an enormous amount of files too.
from the forum: http://www.bolidesoft.com/forum/about1064.html

Quote

I have a gallery with a large number of pictures in it (> 60K) and if I try to do a compare within the gallery the remaining time indicator runs out in about an hour into to compare with only 5% of the files compared. If I let the program run all day, the compare slows down (after 20 hrs) until it is taking several minutes to do one compare. I then have to stop the program as it would take a week to finish the job at the rate it was going. Sub folder comparisons (10K) of the same pictures do not have this problem.

Curt · « **Reply #32 on:** October 09, 2008, 07:32 AM »

- I have never tried this many images in one go, so I didn't know. My fav folder contains 6,700 images, which causes no problem. User should remember to save gallery, so IC can re-use the result the next time - user should not just create a new gallery each time. Edited: Also, users comparing > 60k images should have a lot of RAM, shouldn't they.

Curt · « **Reply #33 on:** December 30, 2008, 04:45 PM »

Visual Similarity Duplicate Image Finder from MindGems is $25 but should equally be listed, because it will do the job fast & good. In fact I am thinking about having it replace Image Comparer! The $40 Pro version will read all picture formats.

For exact copies, I will use either Fast Duplicate File Finder (also from MindGems) or Duplicate Cleaner. Both $0.

Mary101 · « **Reply #34 on:** December 31, 2008, 06:47 AM »

i have known a lot of informations.

Curt · « **Reply #35 on:** December 31, 2008, 09:21 AM »

- good, Mary101, learning from new informations is half this site ;-)
(- read http://www.donationcoder.com/forum/index.php?topic=16383.msg144291#msg144291 )

Welcome to DC!

sajman99 · « **Reply #36 on:** January 06, 2009, 10:43 PM »

Greetings to my new friends here at DC. As a photo enthusiast, I have enjoyed this thread and its many useful recommendations. I thought it worth noting that both the previously mentioned Visual Similarity Duplicate Image Finder and PhotoSort have demo versions which work quite well with only minor limitations. Specifically, neither allow moves/deletes but otherwise work just fine without any time limitation. Of course, if you can afford to register and get full functionality, then by all means...

Likewise, FirmTools' Duplicate Photo Finder at http://duplicatefinder.firmtools.com/ offers what they characterize as a "trial" version of their product. To me, it's a demo version like the others mentioned because it functions without time limitation but cannot perform moves/deletes. This Firmtools software IMHO needs a similarity/tolerance adjustment like Visual Similarity Duplicate Image Finder and PhotoSort have, but it does utilize cache and process quite quickly. If you just want to find the most similar pictures in your collection without a lot of fuss, Firmtools' solution might be your cup of tea.

Finally, I didn't see a mention of XnView in this thread. Among its myriad attributes, this excellent freeware image viewer at http://www.xnview.com/ works amazingly well in finding similar images via "Tools/Find similar files". With its adjustable similarity/tolerance level, it doesn't get much easier to process an image collection.

sajman99 · « **Reply #37 on:** January 06, 2009, 10:48 PM »

Oops, I meant to say "if you can afford to BUY and get full functionality, then by all means..." As noted, Visual Similarity Duplicate Image Finder and PhotoSort are both demos, not trial software.

sajman99 · « **Reply #38 on:** January 07, 2009, 11:21 PM »

Similar Images Finder at http://www.crown-s-soft.com/sifinder.htm is a $34 tool which looks promising. However, only after installing did I see in the Readme file that "Evaluation version limits the number of processed images to 200." I don't believe this important information is mentioned on their site, but maybe I overlooked it. Since these similar image utilites tend to really bog down when processing many files, this 200 picture limit seems kinda silly to me. I mean, Similar Images Finder may fly through 200 images like lightning, but how will it process 5,000 images? I guess it costs $34 to find out, huh? If anyone has experiences with this commercial software- especially regarding its processing speed with many images, I would certainly appreciate hearing about it. Thanks.

Curt · « **Reply #39 on:** January 07, 2009, 11:34 PM »

- please forgive me, sajman99, but don't you think it might be a good idea to actually RUN the program? Besides, the lack of tooltips made me skip testing it further than the initial scanning. But other than that, I must say Similar Images Finder is incredible fast!

sajman99 · « **Reply #40 on:** January 08, 2009, 12:28 AM »

Hello Curt. In hindsight, I can see how you assumed I didn't run SIF because I said it "looked promising". Sorry for my imprecise language. In fact, I ran it many times and it works really fast processing the maximum 200 images as you correctly noted. But it "looked promising" because that's all I really know about it due to Crown-s-soft's overly restrictive trial. Frankly, processing 200 images is child's play compared to processing 5,000 images. Since these similar image utilites tend to choke processing large numbers of images, I am not exactly inspired to pay $34 for something which may or may not work efficiently. With modern processors, the days of "leave it overnight" image comparison should be in the distant past.

Also Curt, thanks for posting about MindGems' Visual Similarity Duplicate Image Finder. A few years ago it ran so slowly it required overnight processing of a large image collection. Currently, it has superior speed and is definitely a viable option.

4wd · « **Reply #41 on:** January 08, 2009, 01:24 AM »

Quote from: sajman99 on January 06, 2009, 10:43 PM

Finally, I didn't see a mention of XnView in this thread. Among its myriad attributes, this excellent freeware image viewer at http://www.xnview.com/ works amazingly well in finding similar images via "Tools/Find similar files". With its adjustable similarity/tolerance level, it doesn't get much easier to process an image collection.

OFGS!!!

I've been using XnView for years and I've never known about that feature because I don't use Browser mode.

Thanks sajman99!!

Lashiec · « **Reply #42 on:** January 08, 2009, 08:21 PM »

Ooops, same here. And to think I was checking up similar pictures by hand...

sajman99 · « **Reply #43 on:** January 10, 2009, 10:06 PM »

Greetings 4wd and Lashiec, and thanks for the encouraging comments.

I suspect there are many XnView users who weren't aware of its "Tools/Find similar files" feature simply because its developer Pierre-E Gougelet is constantly improving and adding features. If I recall correctly, that feature was implemented in early 2007 somewhere around version 1.90. Also in browser mode is the "Tools/Compare" feature which displays images side-by-side or in tabbed/overlay view. Good stuff.

Yet another freeware "photo duplicate scanner" worthy of mention is SimilarImages at http://celebnamer.celebworld.ws/similarimages/ .

Curt · « **Reply #44 on:** January 11, 2009, 12:17 AM »

Quote from: sajman99 on January 10, 2009, 10:06 PM

... Yet another freeware "photo duplicate scanner" worthy of mention is SimilarImages at http://celebnamer.celebworld.ws/similarimages/ .

SimilarImages may be The Slowest scanner/processor I so far have tried for this kind of job, but it turned out to also be (one of) the most accurate! On my Vista, the $35 Image Comparer will use 45 minutes to scan&process my 8.000 photos. The free SimilarImages used 61 minutes for the same job, but also gave a better, more precise result.

sajman99 · « **Reply #45 on:** January 12, 2009, 02:11 AM »

Curt, glad you liked the accuracy of SimilarImages. Its "Dataset Cache" will provide a major speed boost for all subsequent scans, but of course that first run can feel like forever when processing a large collection. Evidently, it's no easy task to achieve both accuracy AND speed in this type of software.

I have found only one minor problem with SimilarImages. It doesn't detect captioned images as reliably as other tools such as VisiPics. To clarify, I mean pictures that have text overlays, text bubbles, thought bubbles (ie. like in a comic strip) etc. My impression is that SimilarImages' algorithm is not optimized for this type of picture. Since each software has its own unique algorithm which "sees" the images in a slightly different way, I prefer to use more than one tool to scan for similar images.

SimilarImages' display of image matches reminds me of the shareware ImageDupeless, another worthwhile contender with respectable speed and accuracy.
ImageDupeless 1.6.3 at http://www.imagedupeless.com/en/index.html ; /$29.95. Looks like it hasn't been updated since 2005 and is limited to 4,000 images in the trial version .

sajman99 · « **Reply #46 on:** March 04, 2009, 09:49 PM »

And the beat goes on...

Want a small, portable, and free image comparison tool which delivers very good accuracy?
ImgSearch http://kolmck.net/e_apps.htm ; latest version 3.08

Not likely to be your first choice when scanning a large collection, but ImgSearch is still a nice addition to one's toolbox.

sajman99 · « **Reply #47 on:** July 14, 2009, 08:12 PM »

Recently I discovered the following...

(1) The previously mentioned ImgSearch is a small and rather plain-looking freeware which works very effectively as a similar images finder if configured properly. If anybody tried ImgSearch and received poor results, I encourage you to take another look.

(2) Michael Vinther's small freeware file manager Commander at http://meesoft.logicnet.dk has been mentioned in the DC forum before, but I didn't see where anybody noted a special ability of Commander relevant to this thread.

Good day, sajman99

sajman99 · « **Reply #48 on:** August 07, 2009, 09:03 PM »

Developer Rafal Fraczek's Similar Images Finder is a free tool for finding both duplicate and similar images. The application can run in Windows and Linux, requires no installation, and renders results/matches in a .htm file.

With the current version of SIF, you can check whether you already have an image in your collection by selecting it (the reference file) and comparing others in a directory. But the developer has indicated via email (and graciously allowed me to reveal) his plans to implement a "compare all images" feature in the next version of the software. This would mean SIF would operate in the conventional manner of Dup Detector, VisiPics, etc.

Good Day, sajman99

Curt · « **Reply #49 on:** August 14, 2009, 07:17 AM »

I've "always" thought that the normal asking price for Bolide Image Comparer, $35, was a little too high, but this one-day-offer, coming up Wednesday, should make the price just right. $23 at Bits du Jour / Daily-Deals. I already bought it for $35, and I am satisfied with the program.

http://daily-deals.iconico.com/software/image-comparer/

http://www.bitsdujour.com/software/image-comparer/

Quote

Image Comparer
Get Rid Of Similar-Looking Images!
Image Comparer Media
If you're a professional photographer, amateur enthusiast, or just one of those people that takes a lot of pictures, then you're going to LOVE today's software: Image Comparer scans your entire digital image/photo/picture collection and analyzes each file's visual content to locate similar shots, allowing you to edit out the bad, blurry, and extraneous versions. Image Comparer analyzes your digital images and automatically selects the best shot out of the many duplicates on your system, letting you to move or delete duplicate images with just a couple of mouse clicks. It uses a content-based image search that "sees" whether or not images are redundant rather than comparing file details or time-stamps. You can even search for rotated and flipped images!