Music Identification via Sound Card

[Nzyme]

Hi,

I am looking for a software that can do the following:

When playing any music (from media player or web browser) and when the program is activated, it should monitor audio passing through the sound card and should identify the music via audio finger printing.

It can take a sample of audio clip, analyze it (audio fingerprinting) and then get the information from online databases. It should then list the following:

Track Name, Artist Name, Album Name, Album art & other tags if possible.

Please note that there are other programs currently that listen through the microphone (like Shazam) but this program should only monitor audio from the sound card of a computer.

I am really surprised that there are hardly any such programs. There are programs that can record audio from the sound card and there are programs that analyze (through audio finger printing) and retrieve information from online databases. They I am sure that these 2 functionalities can be combined and the above can be achieved.

Thanks!

[mouser]

There are programs that can record audio from the sound card

It is my understanding this this can be extremely difficult, is dependent on vagaries of the sound card/driver, and may be close to impossible for many/most modern sound drivers.
It may have been easier with older versions of windows.
This may be your main obstacle.

[Renegade]

There are programs that can record audio from the sound card

It is my understanding this this can be extremely difficult, is dependent on vagaries of the sound card/driver, and may be close to impossible for many/most modern sound drivers.
It may have been easier with older versions of windows.
This may be your main obstacle.

Actually, it's not hard at all. You can simply grab the waveout (or similar) on most cards to get the audio.

What's difficult is the audio fingerprinting. It's not technically difficult as you only need to hook into the API for some service like Grace Note, but the business side of it is daunting as you're pretty much guaranteed that you'll lose money on it.

Now, if you can use something like Echoprint.me:

http://echoprint.me

Or MusicBrainz:

https://musicbrainz.org/doc/Fingerprinting

(A bit more out there as well.)

Then you can avoid the royalty costs of Grace Note.

However, you are then basing your software on something that you don't know if it will be around tomorrow, which often ends up badly. e.g. All the people who tried to use MS DRM for WMA, etc., and those that used the Google translate API, etc. etc. etc.

If you can run your own data server, then you're good, but you then run into server costs. Again... you're going to lose money on an application like that unless it has significant value elsewhere that people will pay for.

The best place to look is in audio software that does something else and also does audio fingerprinting an an ancillary function.

But as for writing up a small utility that did that, it's possible to do with some of the above open source, but the payoff is zero and it's a week or more of work, and a lot more if the entire solution runs locally instead of over the Internet. And, depending on how its released, the end user may need to sign up for developer accounts with service providers then go through the entire hassle of figuring out API keys.

[Edvard]

There is an open-source music recognition project called (appropriately) "My Music Recognition" that is currently looking for testers:

'Testers wanted' announcement:
http://newsletter.sourceforge.net/ct.html?ufl=1&rtr=on&s=x8pb08,22glk,10s8,2ocb,7opb,1zwy,l1rr
Home site:
http://sourceforge.net/projects/mymusicrecognition/
GHacks review:
http://www.ghacks.net/2014/06/30/music-recognition-identifies-songs-playing-computer/

(The sourceforge site is offline at the moment; give it a few...)
EDIT: (sourceforge is back up)

As an alternative, I've heard bits here and there about Tunatic:
http://www.wildbits.com/tunatic/

[Innuendo]

I think there's a lack of such software for the desktop PC because it's easier just to take your phone out of your pocket and have it identify the song.

[Renegade]

There is an open-source music recognition project called (appropriately) "My Music Recognition" that is currently looking for testers:

'Testers wanted' announcement:
http://newsletter.sourceforge.net/ct.html?ufl=1&rtr=on&s=x8pb08,22glk,10s8,2ocb,7opb,1zwy,l1rr
Home site:
http://sourceforge.net/projects/mymusicrecognition/
GHacks review:
http://www.ghacks.net/2014/06/30/music-recognition-identifies-songs-playing-computer/

(The sourceforge site is offline at the moment; give it a few...)
EDIT: (sourceforge is back up)

As an alternative, I've heard bits here and there about Tunatic:
http://www.wildbits.com/tunatic/

I gotta admit, I'm a bit surprised to see someone create a single application with this lone functionality. It makes sense for some larger applications, but a dedicated one? I'm surprised.

[Renegade]

There is an open-source music recognition project called (appropriately) "My Music Recognition" that is currently looking for testers:

'Testers wanted' announcement:
http://newsletter.sourceforge.net/ct.html?ufl=1&rtr=on&s=x8pb08,22glk,10s8,2ocb,7opb,1zwy,l1rr
Home site:
http://sourceforge.net/projects/mymusicrecognition/
GHacks review:
http://www.ghacks.net/2014/06/30/music-recognition-identifies-songs-playing-computer/

--- stream of consciousness thread warning ---

Well, I've been trying it out, and so far... jack all.

Some of the songs I've been playing/testing:

Led Zeppelin:

Rock and Roll
The Ocean
Good Times Bad Times
Whole Lotta Love <--- Didn't get no love!

Judas Priest:

Defenders of the Faith
Breaking the Law <---- The most recognisable metal riff in history!

Pink Floyd:

Goodbye Blue Skies
Wish You Were Here


The installation was smooth enough, but it took forever to enter an API key as the software froze up with every action. I managed to struggle through the pain, but it was horrid. Once the API key was entered, it seemed to run ok, but I got zero results from it.

So... I guess I'm back to my original position. This isn't stand alone software that you can simply write and give out. This is a major friggin' effort to get this done sanely.

Oh crap...

It just recognized "Wish You Were Here" now! And "Money"!

WTF?!?

Ok... giving it a bit more of a chance...

And now it's recognized "The Ripper"!

Trying "Breaking the Law" again... And it got it!

"Rock and Roll" again...

Nailed it again.

Seems that it needs to "warm up".

Checking on "partials" now... (Which I did before above...)

7 minutes into "Stairway to Heaven" and no recognition... not good... Finished and no recognition.

Back to "The Ripper"... 1:20 into it and no recognition. At 2:25 it got it, but the song is only 2:51 long.

Sorry. But I'm giving this a "fail".

The entire point to audio fingerprinting is to take a snapshot and then return a result. This doesn't do it reliably.

Cripes! Now it got "Drugged and Driving" by the Dayglo Abortions at 1:00!

Bizarre...

I don't know what to make of this.

One last test from Youtube here:

https://www.youtube.com/watch?v=eimUHl8rbts (Fair warning - the content of the video will upset some special snowflakes.)

There's some background music there that I don't know what it is, and I'd really like to know... which got me started on this post...

The music starts at around 3:05 or so.

No luck.

Fail.

It seems to be able to identify some songs on disk (which are probably already known), but not reliably, and not quickly, and also can't identify music set against speech in a video.

I guess I'm back to the online solution not really working all that well. I guess Gracenote charges for a reason.

[Vurbal]

Missing on The Ocean and Good Times Bad Times is a pretty fundamental fail. If you hadn't mentioned your results with Money, I'd be inclined to suspect it was ignoring bass frequencies. Of course, The Ocean is just as recognizable from just the guitar.

Oops,  guess you didn't mention Money.

[Nzyme]

There is an open-source music recognition project called (appropriately) "My Music Recognition" that is currently looking for testers:

'Testers wanted' announcement:
http://newsletter.sourceforge.net/ct.html?ufl=1&rtr=on&s=x8pb08,22glk,10s8,2ocb,7opb,1zwy,l1rr
Home site:
http://sourceforge.net/projects/mymusicrecognition/
GHacks review:
http://www.ghacks.net/2014/06/30/music-recognition-identifies-songs-playing-computer/

(The sourceforge site is offline at the moment; give it a few...)
EDIT: (sourceforge is back up)

As an alternative, I've heard bits here and there about Tunatic:
http://www.wildbits.com/tunatic/

Thanks Edward for the links. Unfortunately, the program did nothing but froze my PC. I had to forcefully logoff my PC after trying for a couple of times.

[Edvard]

Woops... sorry about that. 

[Renegade]

Thanks Edward for the links. Unfortunately, the program did nothing but froze my PC. I had to forcefully logoff my PC after trying for a couple of times.

If you saw what I did, that wasn't really a freeze, but a seriously annoying hang. Might have been different, but it was miserable to get up and running with it on my end.

[toastymah]

So, you want software that can record from your soundcard then do music identification, right?  Good.  I did too... in 2011.  So I wrote that software and made it integrate into Microsoft Lync's API so that it would scrobble what I was listening to in my status bar.  Worked really well.

So here's the semi bad news.  My software used Echoprint to do identification.  If you don't know what that is then definitely look it up.  The tough part to take in, however, is that they discontinued their API for identify in January of this year.

Fear not, though.  There are other options; Two specifically.  MooMa.sh has an API that is supposed to map directly like Echoprint would, which would be fantastic.  Their API is free and this is my top choice.  Only, I'm having trouble getting an API key from them.  C'est la vie...  Option 2 is a service which I already have an API key for: doreso.com.  Now, this service is well and fine, much like Echoprint and MooMa, but there is a considerable difference.  With EchoPrint and MooMa, the software you use must first encode an audio sample into some base64'd array before making a GET request with it tacked onto the end of a parameter.  Simple HTTP request that returns a JSON object.  Doreso returns JSON too, but they don't want a base64 array of EchoPrint encoding.  Instead, they just want the whole sample as a wav file uploaded with each request.  While this is completely feasible by making some non-trivial changes to the source, I'm not super thrilled about having to send that much data per request.  There's that whole pesky distribution of recorded material to third parties legal gray area too.  So, while the later is technically possible, I'm much more inclined to wait for MooMa to send a key my way.

I'm not sure if anyone is still interested in this topic any more so if you are, please reply back here and I'll know it's not dead.  If so, feel free to ask or comment.  It would be good to see if there are people that may also benefit from some of this code.

[Curt]

I used to record a lot of music via the sound card, and have the recording program auto-name the tracks. But somehow the various programs never managed to name any Danish track, so over time I lost interest.

Will any of the solutions you are thinking about define anything but "American" music?

[mouser]

although this isn't something i'm interested in, just wanted to thank toastymah for a cool post -- it's always nice to hear from coders who have figured something out when it's outside my area of expertise. 

[toastymah]

I used to record a lot of music via the sound card, and have the recording program auto-name the tracks. But somehow the various programs never managed to name any Danish track, so over time I lost interest.

Will any of the solutions you are thinking about define anything but "American" music?

So, there are several different ways to accomplish that and I have some suspicions as to what you had working / didn't have working there.  First, there exists several programs that can rip from sound cards as standalone applications or as plugins to media players.  'Ripper' for WinAmp comes to mind.  The majority of these rippers will simply look for meta data contained in the file/stream and build you a file with the song name, artist, etc., and append other found meta data to the file.  In these cases, there is no real audio fingerprinting being done.  The data is either there for the ripping application to grab or it is not.

The other option is the type of application that I made; which rips a sample from sound card and queries some centralized database for song information.  Echoprint was amazing at this because EchoNest created a tool to work in bulk to build their samples, which ultimately were stored in the smallest necessary array.  Further, the tool did not need to listen to the music at the speed you do, it could do what it needed to do very fast.  With that encoding, one would need to query EchoNest's servers for the closest found code and they gave back a JSON object with all kinds of great information (artist, song name, album, 'hotness').  This could be used to rename files and/or add meta data.

The obvious question is, "How do they know what the meta data for this song is?".  As far as we know, they had relied on the community to tell them with their bulk encoding/decoding and connecting to EchoNest to tell them what those songs were.  That's pretty much what their bulk encoder did, it grabbed everything about the files you had and allowed you to send that to them.  If your files had meta data then they got song encodings with accompanying meta data.  Also, it's likely that the EchoNest partners (one of which is Spotify) is providing additional information to them.  This is speculative, however, so I digress.

For your Danish track issue, the problem has different possible reasons.  First, you may want to check how your application is actually attaching meta data to your music, whether offline or online.  If the program works with files and can do it's job offline, then there's a pretty good chance that it's simply targeting meta data.  In this case, you will want to find what is different about your Danish tracks as opposed to your others.  Something as silly as the Danish tracks using UTF-8 in their meta data while your other tracks using Unicode could be all the reason for your issue.  If, however, you find that your ripper only works to identify tracks online, then it is likely that it is fingerprinting the audio and connecting to a database for meta data information.  In this case, you're somewhat stuck because you are at the mercy of the robustness of identifying database.  For example, MooMa claims to have 7 million songs in their database.  Spotify holds more than 20 million songs, so just that service alone would cause MooMa to potentially have less than 50% accuracy.  This actually ends up not being the case because popular songs are identified more frequently and the more obscure ones are least likely to be identified.  So in your case, if your software is connecting to a centralized database, then you would just need to be hopeful that there are enough Danish music lovers to have previously identified your song and uploaded that meta data to the server at least once.

I hope this helps and makes things a little bit clearer.

although this isn't something i'm interested in, just wanted to thank toastymah for a cool post -- it's always nice to hear from coders who have figured something out when it's outside my area of expertise.

Thanks for the kudos.  Happy to help when I can.