Welcome Guest.   Make a donation to an author on the site October 23, 2014, 10:09:05 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the full one-year retrospective report on DonationCoder.com.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Netflix contest: $1M to developer of best movie recommendation engine  (Read 14325 times)
KenR
Super
Blogger
***
Posts: 825


View Profile Give some DonationCredits to this forum member
« on: October 02, 2006, 01:13:48 PM »

Netflix is holding a 1 million dollar contest awarded to the coder who can betst improve their movie recommendation engine.



from http://oreilly.com/
« Last Edit: October 02, 2006, 01:21:17 PM by KenR » Logged

Kenneth P. Reeder, Ph.D.
Clinical Psychologist
Jacksonville, North Carolina  28546
Gerome
Charter Honorary Member
***
Posts: 154


View Profile WWW Give some DonationCredits to this forum member
« Reply #1 on: October 02, 2006, 01:43:13 PM »

Hi,

Bah... I'd prefer 1 million Euros smiley
Logged

Yours,
(¯`·._.FBSL Help file]
(¯`·._.
KenR
Super
Blogger
***
Posts: 825


View Profile Give some DonationCredits to this forum member
« Reply #2 on: October 02, 2006, 01:47:08 PM »

Ok Gerome,
You want to try to convince us that is NOT complaining?
Ken
Logged

Kenneth P. Reeder, Ph.D.
Clinical Psychologist
Jacksonville, North Carolina  28546
brotherS
Master of Good Ideas
Honorary Member
**
Posts: 2,105


To make a difference, be different.

View Profile Give some DonationCredits to this forum member
« Reply #3 on: October 02, 2006, 02:06:53 PM »

Netflix is holding a 1 million dollar contest awarded to the coder who can betst improve their movie recommendation engine.
What a great idea! I read articles in the past that Netflix seems to be innovative (I'm no customer nor related to them) and this proves it. By running the contest until "at least October 2, 2011" they show they really plan to spend the million to gain a 10% performance win, and I also like the idea of their "Progress Prizes: $50,000 (USD) Cash each award", which is handed out once a year for those who are improving the performance the most, but not reaching the 10% hurdle.

@Gerome: we'd still accept you if you'd show less negativity, really!
Logged

Thank you.
Gerome
Charter Honorary Member
***
Posts: 154


View Profile WWW Give some DonationCredits to this forum member
« Reply #4 on: October 02, 2006, 02:10:58 PM »

Hi,

@Gerome: we'd still accept you if you'd show less negativity, really!

What about negativity ?
Because 1 million of Euro is > to 1 million of USD ?
ROFL
Logged

Yours,
(¯`·._.FBSL Help file]
(¯`·._.
allen
Charter Member
***
Posts: 1,173



Powered by Beard

see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: October 02, 2006, 11:01:06 PM »

Hi,

@Gerome: we'd still accept you if you'd show less negativity, really!

What about negativity ?
Because 1 million of Euro is > to 1 million of USD ?
ROFL

But perhaps the point, Gerome, is that 1 millsion USD > Nothing
 -- and as an American company, USD would make sense as the prize, no?
Logged
Gerome
Charter Honorary Member
***
Posts: 154


View Profile WWW Give some DonationCredits to this forum member
« Reply #6 on: October 03, 2006, 02:31:07 AM »

Hello,

But perhaps the point, Gerome, is that 1 millsion USD > Nothing
 -- and as an American company, USD would make sense as the prize, no?

Ok Allen et al, I've just seen my jokes are just badly interpreted... I'm sorry for you... smiley
BTW, this offer seems nice.
Logged

Yours,
(¯`·._.FBSL Help file]
(¯`·._.
brotherS
Master of Good Ideas
Honorary Member
**
Posts: 2,105


To make a difference, be different.

View Profile Give some DonationCredits to this forum member
« Reply #7 on: October 03, 2006, 03:02:16 AM »

I'm sorry for you...
And here you go again...
Logged

Thank you.
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #8 on: October 03, 2006, 04:13:21 PM »

interesting blog critisizing the approach:
http://www.feedblog.org/2...6/10/netflix_ranking.html

Quote
If you assume they can get 4x compression out of this dataset you're still looking at 50G which you'll probably have to query at runtime. This probably means a memory based cluster and to do that you'll need at least $15-20k at a minimum to get started.

What Netflix should do is allow teams to pitch them their proposals and fund their research with a winning prize of $1M.

Logged
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: October 06, 2006, 08:10:50 PM »

I'm not a coder but it seems to me you wouldn't necessarily need to use the whole data set to prototype and see if your results net reasonable gains. I presume they are supporting this to a reasonable degree by providing a good amount of info about their current system (if not source code). If that's the case it should be reasonable to extrapolate results for your test data set size/range and work against that. If you can show significant potential in a new approach I would think they will work with you to scale it up and see if it's really practical. It only makes sense from their end to do so.

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: April 19, 2007, 04:42:48 PM »

Alex (www.3form.org) has convinced me to join with him in working on this a little since both of us have backgrounds in machine learning.

I have my doubts about the possibilities of winning this but it seems an interesting challenge and worth a little bit of time playing with it.  I'm happy to talk with anyone in the donationcoder irc channel (#donationcoder on efnet, or hit the chat button aboce) who thinks they might also want to try entering.
Logged
urlwolf
Charter Member
***
Posts: 1,784



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #11 on: April 20, 2007, 06:34:51 AM »

all,

Do you know of any webservice where users log which movies they have actually seen? Something like last.fm for music, but using movies instead...

Actually, same thing for books read would be great too..
@mouser: I sent you an email, cannot reach the #IRC channel for some reason...
Logged
CWuestefeld
Supporting Member
**
Posts: 939



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #12 on: April 20, 2007, 11:34:44 AM »

I made a stab at this late last year with a friend. The amount of data isn't a big deal for the storage capacity of a modern PC. More difficult is being able to process it in a reasonable time.

We started with a bit of success. Our very first attempt was enough to beat Netflix's baseline and yield a score that qualified for the leaderboard (at the time). Doing this was surprisingly easy. Nothing but a good working knowledge of statistics was needed to beat their baseline. Simply normalizing the scores of all the users according to their standard deviations was enough to do this.

Our project fizzled out after just a couple of weeks. The problem was that neither of us has an understanding of the collaborative filtering techniques that appear to be necessary to do really well. And experimentation was difficult. The size of the data set is such that we could only really make one run a day (we were using MS SQL Server, if you're wondering).

Also, I was a little frustrated because I think that their criteria measure the wrong thing. At first it was an intellectual challenge anyway, but that wore out. The challenge is to predict the score that a customer would assign for each of a set of movies. The thing is, this isn't really an interesting question to solve. I don't care very much if Netflix thinks that I'd give this movie a 3.1 or 3.5. What's really interesting is only selecting a list of movies at the top end of the scale, so Netflix can give me a list of recommendations when I ask "what good movies have you got for me today?".
Logged



mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: April 20, 2007, 01:06:42 PM »

great to hear you worked on it!!

my main source of skepticism is a suspicion/fear that there is some inherent noise in the data, and that winning the contest may in fact require being better than the noise would allow without pure luck.  in other words, a person's rating for any given movie may vary a little bit depending on their mood at the time, or based on some truly unpredictable factors.  for example, imagine i ask you to predict the votes of 100 people and i happen to know that 95 of them always vote one way, but the other 5 flip a coin to decide which way to vote.  now you can't hope to get a perfect score because of the element of true randomness (coin flip).  At some point, trying to get better than a certain score on the netflix challenge data is going to look like that -- though whether that happens well before the million dollar prize accuracy i dont know, but that's my fear.
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: April 20, 2007, 01:07:39 PM »

the forum makes for great reading: http://www.netflixprize.com/community/
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: September 12, 2007, 07:58:28 PM »

about 20 days until first round closes..
i'm playing around with some code and hope to have it done in time to submit.
Logged
tinjaw
Supporting Member
**
Posts: 1,926



I'm so glad breakbeat techno isn't an illegal drug

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: September 12, 2007, 08:01:37 PM »

Go mouser. Give us an update on your scores. I remember you talking about an ongoing scoring mechanism during the contest.
Logged

 
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: June 26, 2009, 10:16:13 PM »

After 2 years it looks like there is going to be a grand winner.. amazing.

http://tech.slashdot.org/...ze-May-Have-Been-Achieved

The nice thing will be reading about the final algorithms.. we've actually known most of the details of it since the authors are academics and published papers on their algorithms.. But it will be interesting to hear how they squeezed out the last drops of performance.
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: July 27, 2009, 01:15:23 AM »

The contest is now closed and the winner chosen, and WOW it seems like a surprising ending to a surprising contest..

By A Nose: Netflix Prize Leaders One-Upped With One Day Remaining:


Homepage of the winning group:

wow. just wow.
« Last Edit: July 27, 2009, 01:27:40 AM by mouser » Logged
TheQwerty
Supporting Member
**
Posts: 84

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #19 on: July 27, 2009, 05:58:59 AM »

Note that no winner has been chosen yet.
Netflix is going to evaluate the entries and also take performance into consideration before they select one or re-open the contest.
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: July 27, 2009, 06:01:27 AM »

my understanding is that they were only supposed to verify that the code works, not decide based on efficiency, etc. and that they could not "re-open" the contest unless there was some violation of the rules.

do you have any more info about this and links?
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #21 on: July 27, 2009, 06:02:15 AM »

I've read some of the previous papers published by both teams that broke the 10% barrier and it will be fascinating to read the final papers.
Logged
TheQwerty
Supporting Member
**
Posts: 84

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #22 on: July 27, 2009, 07:03:22 AM »

my understanding is that they were only supposed to verify that the code works, not decide based on efficiency, etc. and that they could not "re-open" the contest unless there was some violation of the rules.

do you have any more info about this and links?
Well for links, just look at what you posted, none of them have an official statement that Ensemble won. tongue  And the official Netflix blog says that they'll begin verifying: http://blog.netflix.com/2...tition-closes-but-no.html

Qualified entries will be evaluated as described in the Rules. We look forward to awarding the Grand Prize, which we expect to announce in a few weeks. However if a Grand Prize cannot be awarded because no submission can be verified by the judges, the Contest will reopen. We will make an announcement on the Forum after the Contest judges reach a decision.
So maybe not taking performance into consideration, but there's also ...

Thanks. In fact, this is a very happy day for us - our team is top contender for winning the Grand Prize, as we have a better Test score than The Ensemble. (Probably this is the first post revealing this in the forum smile)
Which led me to believe that the leaderboards are not calculated on the full/exact data set that the winning algorithm will be tested on.

Don't really know.. but I wouldn't get ahead of yourself congratulating The Ensemble until Netflix has made an announcement. Certainly not with results as close as these.
Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: July 27, 2009, 04:56:11 PM »

Quote
Which led me to believe that the leaderboards are not calculated on the full/exact data set that the winning algorithm will be tested on.

ah that's an interesting point.. i'm trying to remember now what the rules said about that.
it does seem like with scores this close, it's a bit silly to say that one of these 2 teams one, and the other "lost."

« Last Edit: July 27, 2009, 04:58:18 PM by mouser » Logged
mouser
First Author
Administrator
*****
Posts: 33,579



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #24 on: August 12, 2009, 09:36:34 AM »

updates:
http://www.google.com/hos...ZJLIyUQzLsWXaNLAD9A0TFMO0
http://www.netflixprize.c...ity/viewtopic.php?id=1520
Logged
Pages: [1] 2 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.049s | Server load: 0.21 ]