Photo managers with face recognition?

[JavaJones]

Have any of you played with face recognition in your photo tagging workflows? Do any of you use it on a regular basis? If not, why not?

After using Picasa with the face recognition feature since I think version 3.5, I've become pretty attached to having this capability. Once you see what you can do with it and how often it's useful, I think most people would be sold on it. The "Make Face Movie" functionality alone is gold, but I've also had lots of requests from people since I told them about this for photos of themselves. It's a really cool thing to have. Unfortunately it seems relatively rare still in the world of photo managers. So I'm on the hunt for the best photo manager/cataloger/organizer that has some face recognition functionality.

You're probably thinking if I like Picasa so much, why not just use that? Well, there are a number of problems there. First, just on the face recognition itself, while I find the actual recognition capability to be quite good, it's rather unpredictable and hard to fully control the scan process. Any faces that are detected are easy to deal with, the problem is that scanning doesn't always detect all faces and it seems there's no way to force a re-scan without losing all your existing tagged faces in that folder. There's also no way to resume a scan that may have stopped for some reason. So while I have a great catalog of about 100 tagged faces/people and 1000's of photos of them labeled, I know there are many more that aren't tagged yet and I don't want to manually tag them. There's also no way that I can see to organize people into groups, which would be nice. Perhaps worst of all Picasa doesn't use standard meta data tags for face data, so the info is not portable to other apps. More generally speaking I also find Picasa's editing functionality rather limited, especially compared to higher-end (non-free) apps like Lightroom. Lord how I wish Lightroom just had good face recognition, I would just use that!

So what apps have I found and/or tested so far? Here's a short list, with some test comments following.

• Picasa - See above, in general I love the actual face recognition and tagging capability and the UI for dealing with faces, it's mostly bugs that put me off it
• iPhoto - Not tested, I'm not on a Mac and I also hate apps that enforce their own folder organization scheme
• IDImager - Has face recognition but is not cheap and from my (admittedly brief) tests the actual face recognition not only doesn't work that well, but is also cumbersome to use
• Windows Live Photo Gallery - Tested briefly, found it cumbersome and not very accurate either, confirmed by Cnet review: http://news.cnet.com..._3-10363727-248.html
• Photoshop Elements - Not tested, but reviews say it too is cumbersome and not as good as Picasa: http://graphicssoft....otoshop-elements.htm
• digiKam - This is an open source photo suite which is a part of KDE I guess, and I hadn't heard of it before, but for an open source tool it's surprisingly nice; unfortunately only the 2.0 beta has face recognition and I couldn't find a Windows binary version to test, only source so far
• Fotobounce - A desktop app that claims to center its photo organizing around people "because that's the way we think users like to work with their photos"; requires Adobe AIR even though it's fully a desktop app (come on, seriously?); this app has an interesting, clean UI, definitely focused on face recognition, decent recognition quality and speed but the UI is overall slow (probably because of AIR) and the actual workflow is definitely clunkier than Picasa; so this one is promising but its lack of other features makes it pretty much a 1 trick pony for me so its integration with other apps (like a simple right-click "edit in this app" feature) will be key if I'm to use it any further

That's all I've found so far besides a few spammy hits with mention of one or two other apps where I couldn't confirm face recognition was even a feature, and I wasn't about to go installing more apps willy nilly to find out.

One annoying thing about all the apps I've tried, including Picasa, is they also don't seem to use the full power of my system in the scanning process. I have an i7 920 and 6GB of RAM, but CPU usage barely gets above 7% (less than 1 core fully utilized) when doing face recognition in Picasa. Fotobounce actually did pretty good int his regard, bouncing between 10 and 40% usage initially, later rising to 70-90% which is near ideal, but it was the best of the lot as far as fully utilizing resources, and the UI is rather "heavy" and clunky anyway. Maybe the bottleneck is elsewhere (disk I/O), but given how fast these apps can build thumbnails I have my doubts that's the problem; probably just need multithreading/better multithreading.

So that's it. Any and all additional suggestions greatly appreciated! Free or pay, doesn't matter to me.

Edit: More testing of Fotobounce and I have to say it's fairly promising. I'm still not a huge fan of the overall "feel" of the UI, but it's reasonably functional, albeit not as smooth as Picasa. I *do* think it does a better job finding faces than Picasa, but it also has more false positives. Then again it's hard to compare directly as well because Picasa, unlike Fotobounce, has options for how accurate you want the detection to be. So I could easily lower the accuracy threshold and probably find more faces but also get more false positives. At least Picasa gives you that option. Additionally I think part of the reason some photos weren't identified in Picasa is due to bugs in the scanning engine actually *not* scanning particular photos, rather than an inability for the algorithm to find a given face. In fact Picasa displays a fairly remarkable ability to find faces, better than any other algorithm I've seen, it's just that the scanning engine is not reliable or very controllable.

- Oshyan

[JavaJones]

Ok, I'm not terribly impressed with Fotobounce in the end. It found some photos Picasa didn't, but also missed a lot that Picasa found. It seemed particularly thrown off by sunglasses and faces turned downward or sideways, as well as faces that were small/in the background (although this may not be a bad thing). It did do a decent job of recognizing faces tilted at an angle but still facing the camera. Picasa does this alright as well.

Ultimately it's just not good enough to recommend over Picasa, especially given Picasa's superior UI and much more sophisticated capabilities. A good example is what you can do with face-tagged photos once you tag them. Here's a visual aid: http://www.youtube.c.../watch?v=fLQtssJDMMc It's actually *really* cool when you do this with photos of yourself or friends.

- Oshyan

[lanux128]

tbh, i'm not a big believer of these type of software, as i have some apprehension over exposing my pictures/music/etc to software that modify data. however having said that, i am keen to know if there any good programs out there that could change my mind.

as for face recognition, Picasa has had a headstart since versions that supported it came out back in 2008. also having the advantage of being backed by Google means that future updates are more likely to be guaranteed.

[JavaJones]

Yeah, you definitely have to be careful with apps modifying your file's meta data. Many apps are surprisingly, frustratingly careless in this regard. So always test on a small group of photos that are *copies* of originals, just in case. If I were to do a review, that's definitely something I would test and report on.

Believe me I'd love to stick with Picasa. But there are enough long-standing issues, with multiple Google support threads and no response from Google, that it's a bit discouraging. Some of these issues date back to 2009 or even earlier, a lot of the same things I'm talking about, with scans never completing, or very obvious faces not being detected. The baseline functionality and UI are great and I applaud them for that, but it still needs a lot of work.

Not to mention that even if the face stuff was perfect, it would still be lacking some important basic editing functionality. So unless it can finally write out data to standard meta data tag or sidecar format, it still won't likely keep me as a user long-term.

In the end probably Lightroom will get this capability once it's beyond the clunky implementation in Photoshop Elements. Then I'll have everything I need. It's just a matter of when, and it so happens that I am on a photo organizing kick right now and would really like to get this all out of the way. I'm still hoping there's a better tool out there. It seems surprising that a *free* tool from Google of all companies - not exactly focused on imaging software - would have the best capability in this area, in a field where there are multiple high-end, expensive options like Lightroom, Bibble, Capture 1, DxO Optics, IDImager, etc.

- Oshyan

[Renegade]

I don't know of any non-mobile end-user software. But you might want to see if OpenCV or OpenCVDotnet for sample projects -- that's pretty common:

http://opencvlibrary...ary.sourceforge.net/
http://code.google.com/p/opencvdotnet/

I have yet to explore the SDK, but it looks promising from what I've seen.

[JavaJones]

Oh I'm not a coder, so I need an existing end-user app. There are already a number of free/open source face recognition libraries anyway I believe. Certainly one that is being implemented in digiKam. And actually I have high hopes for that as the app itself seems quite nice otherwise too. So if the face recognition is good enough it could be an awesome overall option. Surprising as I've never heard of it before now. And being open source it's likely to have a pretty tweakable system as far as face recognition goes (at least I hope so). So I'm definitely keeping my eye on that. In fact it's available as source now I believe, so if any enterprising person wants to compile a Windows version, that would be awesome.

- Oshyan

[kfitting]

IDImager is weak in this area (as of version 5.1.1) because the third party tools used are not yet reliable enough in the author's opinion (Hert is the author of IDImager, Hert's comment on Face Recognition) to open up all the features to for use by IDImager.  There are a few threads on the IDImager forum about this if you want more information, and a script written to import Picassa face data to IDImager (I have not used this so cant comment on it).

These programs (I bought and use Bibble and IDImager) are expensive because of their power and flexibility in other areas.  I use IDImager for overall image management/cataloging, and Bibble as my RAW converter.  Those are the things that matter most to me.  I am greatly impressed by the faces feature in Picassa, but am not willing to give up the other things.  If faces is what you want, then Picassa might be the best option for now (or iPhoto if you're a Mac user). 

To each his own.

[JavaJones]

I'm with you generally speaking kfitting, Picasa is not nearly powerful enough for me. But I *do* really like the face recognition functions which is why I'm looking for a more high-end app that can duplicate this capability. I know that the high-end apps are expensive because of other powerful features they have, but then again apps like Sagelight Image Editor are cheaper (for now) and, while not as comprehensive, do have some pretty innovative and unique (and powerful) RAW-related processing functions (see Sagelight v4). Face recognition can be based on existing libraries which solves a good part of the difficulty of the problem, it's the UI implementation in IDImager that I'm disappointed with and I do think that's fully under their control and generally simple enough to "do right" that there's little excuse for the functionality to be so poorly implemented in present versions. As I said, Picasa gets it right, if nothing else they can use that as a model. In that thread you linked to, the IDI developer does say the auto-recognition functionality in their current library just wasn't good enough, which I can understand. But again I have to say, if others can do it, why can't they?

- Oshyan

[CWuestefeld]

I've been thinking about this same thing for some time now. I've been meaning to embark on a giant project of properly organizing 20K-30K images. I've had ACDSee Pro for quite some time, but I've never actually gotten started. Part of the reason for this is that it seems that face recognition would be such a big jump start to the project, and ACDSee doesn't have it -- nor does it look like they *will* have it in their upcoming v4.

But reading this thread prompted me to look around a bit more, and I found a couple of things that might allow the overall workflow to be doable.

I think the conclusion for now is that Picasa is the best for the actual face recognition itself. It looks like with some extra tools, one could create a workflow where the images, once dumped to disc and maybe after RAW developing, get run into Picasa for face recognition. Then use a 3rd party tool to dump the Picasa face data into the image metadata, and then proceed as normal through your later editing and organizing (assuming that your editing will preserve those tags).

I found two such 3rd party tools that can dump Picasa's data into the image metadata. Quoting from their respective web pages:

• AvPicFaceXmpTagger - This program was written because of the new Google Picasa 3.5 face recognition feature. Due to the fact that Picasa 3.5 doesn't store the face recognition data inside JPEG pictures, this program fetches that data and stores it as XMP-MP meta tags inside the JPEG pictures. http://www.anvo-it.d...icfacexmptagger:main
• Picasa Face Embedder - Picasa Face Embedder takes the face data generated by Picasa and embeds it into your images. This enables Windows and software such as Adobe Photoshop and ACD Systems ACDSee to use the data. The software uses the database files that Picasa generates. These files hold the names of people that Picasa recognises in your photos and the rectangles defining where each person is. Picasa Face Embedder takes this information and embeds it into the files themselves thereby binding the information to your photos so that when you copy or rename your photos the information is still present. The software uses a simple drag and drop interface. To start using the software simply drag a folder containing your photos onto the main window and the software will begin processing your photos. http://cris.lovell-s...picasa-face-embedder

[JavaJones]

Nice work, I had come across AvPicFaceXmpTagger before but not Picasa Face Embedder. I'll look into that one. With either solution the problem remains that it is an extra step and requires a bit of annoying jiggery pokery for every update. So if I find an error after doing all my face tagging, I need to resync for example. And every time I add new photos I need to have this extra step.

Realistically all that would not be a deal breaker were Picasa's face recognition even consistent and 100% working. The sad thing is it works really well part of the time, not at all another part of the time, and only ok some of the time. The actual UI implementation is for the most part very good. The identification of similar faces is also fairly good. The problem is the scan process just dies sometimes never finishing detection, and there's no way that I can see to actually *resume* it and ensure it will finish. Either that or there are a lot of faces the system for some reason can't identify even though they seem a lot more obvious than tons of other faces it *does* recognize. Ok no, that latter possibility is very unlikely...

Anyway, thanks for chiming in. At least I have one more potential tool in the toolbox.

What I'm really hoping for is a coder willing to try compiling the latest digiKam 2 beta for Windows.

- Oshyan

[IainB]

I just thought I'd mention in this thread that I have been using Picasa pretty heavily and relying on it as my main IMT (Image Management Tool) ever since it was released. Formerly, my favourite IMT was ACD-See, but I have stopped using that now.
One of the things that Picasa introduced was face recognition. It seemed to be a bit quirky at first, but by now Picasa has been steadily improved to the stage where any of its shortcomings seem to have been pretty much been addressed.

The OP (Original Post) by @JavaJones listed several deficiencies of Picasa. I have listed them below, and added comments as to current status.
Face recognition:

• Unpredictable and hard to fully control the scan process. Any faces that are detected are easy to deal with, the problem is that scanning doesn't always detect all faces and it seems there's no way to force a rescan without losing all your existing tagged faces in that folder. [Currently: Picasa periodically rescans folders for unscanned/new images, and you can force Picasa to rescan images in one or more folders, and a nested set of folders]
• There's also no way to resume a scan that may have stopped for some reason. So while I have a great catalog of about 100 tagged faces/people and 1000's of photos of them labeled, I know there are many more that aren't tagged yet and I don't want to manually tag them. [Currently: As above, face rescans are not an issue. As for tagging in general, photos can be selected to the working tray and tagged en masse as a group. Picasa enquires whether you really want to tag that large a number of images, and then tags them if you tell it to go ahead. Tagged groups can be treated as an album, so that you can show them all together, even if the members of that group also belong to other tagged groups or albums.]
• There's also no way that I can see to organize people into groups, which would be nice. Currently: You can select all the photos pf a person, tag them en masse (as above) to a tag - e.g., "Friends". I have not seen a way to cause subsequent scan/recognition of a new photo to associate the face in that photo with the same tag group(s) as an already recognised photo of the same face.[i/]
• Perhaps worst of all Picasa doesn't use standard meta data tags for face data, so the info is not portable to other apps. [What international data exchange standards for this metadata should Picasa be using?]
• More generally speaking I also find Picasa's editing functionality rather limited, especially compared to higher-end (non-free) apps like Lightroom. [This may still be the case. Perhaps if Picasa emulated Photoshop, it could be even better, but it may never happen as Picasa was probably created for a specific market as a general-purpose tool to build and maintain family albums and post them to Picasa albums on the Internet for sharing.]

[JavaJones]

Thanks for the reply Iain. I agree that Picasa has progressed since my original post, but honestly not as much as I would hope in the face recognition dept. Worse yet, other packages haven't really stepped up to the plate either, so the options are not much different now than they were, unfortunately. In response to your specific points:

While Picasa does rescans on its own (you can set whether you want it to do so on a per-folder basis), it still seems to miss a lot of faces for some reason. This happens even if you force a rescan. It is mysterious. This answers point 2 as well. Maybe my experience is different than yours. My environment is probably atypical for a Picasa user: I work mostly with RAW files and I have over 50,000. So admittedly it's a tall task to ask Picasa to deal with all that with face recognition. But that's what I must ask, or ask for a similar app at a professional level that *can* handle that. There should be *some* solution for this.

To point 3, the idea of tagging multi-selected people to create "groups" is interesting, but it's really a relatively limited workaround that doesn't actually accomplish the goal of persistent grouping. As you said, you can't have it auto-tag new photos with a recognized face, so it's not persistent and requires continuous, laborious maintenance. Not only that but one possible benefit of having categories would be to e.g. collapse categories to reduce visual clutter and allow me to concentrate on the groups of people I am more interested in maintaining records of. A solution using the existing tagging system doesn't allow that.

Regarding meta data, there are a couple standards for it and have been for some time. Picasa supports several of them, but each to varying degrees, and long-standing bugs have caused corruption and unreadable data for other apps (e.g. Lightroom). A quick search will turn up lots of threads spanning the last several years describing these issues. The real problem I guess is Picasa doesn't necessarily play nice with other apps. I wouldn't really care to use Picasa at all and would just focus on other apps *if* other apps offered good face recognition.

As to Picasa's limited editing capability, I probably shouldn't have even made the Lightroom comparison. A better one might be Photoshop Elements, or even Paint.net. Being more specific, there are really just a *few* tools that should be added and/or tweaked to make Picasa much more capable, in my opinion. But the flip side of my point - and the real desire for me - was for a higher-end app like Lightroom to support some of the cool capabilities of Picasa, particularly good face recognition. That's the real disappointment for me. As I said above I basically only use Picasa for the face recognition stuff. All my editing is done in other apps, mostly Lightroom and Photoshop. I would be *happy* if I didn't have to jump over to Picasa for face stuff. So really it's the other software publishers lagging that I'm most frustrated with. Lack of competition not driving progress and all that. Hopefully Lightroom 4 will come out soon with native face recognition and HDR tone mapping.

- Oshyan

[Curt]

I just saw this online face recognition: http://www.pictriev.com/facedb/fs2.php
-from the makers of PhotoScape.

[IainB]

@JavaJones: Just had a thought - which is the tool that you currently regard as your image file/database manager? Is that Lightroom?

[JavaJones]

Yes, basically Lightroom now. I'm *reasonably* happy with it, but still not thrilled. I may go into another period of testing alternatives soon too...

- Oshyan

[J-Mac]

Didn't see it mentioned above, though I may have missed it in some of those long posts! Picasa insists on creating a contact for every recognized face, which IMO completely ruins the already sucky Google Contacts. Back when they first introduced facial recognition/labeling this wasn’t made abundantly clear and after going through a lot of face labeling I found that my contacts list was overflowing with tons of new junk. All my "real" contacts had been doubled, tripled or more; my dogs all had multiple contact listings; I really hate the feature! I then immediately turned off the facial stuff and deleted ALL my Google Contacts and re-imported them fresh.

I don’t know if they have changed this so-called "feature" cause I never tried it again.

Thanks!

Jim