Author Topic: FIFA World Cup Predictor 2010 - A Contest For Best Predictions? (Read 81331 times)

mouser · « **on:** May 28, 2010, 06:34 PM »

The FIFA 2010 World Cup Predictor page has been online and available for use for a few months now.
The url is: https://www.donation...r.com/wcp/index.html

It's an example of Markov Chain Monte Carlo simulation, and is focused on determining the best bets to make on the competition, by looking for bets where the teams have better predicted chances of winning than the current odds-makers are assessing.

The heart of the prediction system is taking simple pairwise team match-up predictions, and using these to simulate millions of tournaments and then calculating expected payoffs.

However, for the system to be useful, it relies on having knowledgeable people making these large numbers of "pairwise team match-up predictions."

Rather than try to come up with a single best set of these pairwise predictions, the page let's anyone create and edit and simulate the tournament using their own set of pairwise predictions.

I'd like to have a contest to see who can come up with the best pairwise prediction estimations. Note that this isn't the same as predicting the overall winner, and it's definitely not the same as advising who to bet on. It's very likely in fact that the predicted winning team will NOT be one that the script advises should be bet on.

So.. what i thought would be nice is to have a contest to see who does the best job of estimating these pair-wise match-up predictions. These predictions basically require you to estimate the final score that is likely to result for every combination of 2 teams in the tournament play each other (yes there are a lot of potential match-ups!).

You create and edit your predictions from this page: https://www.donationcoder.com/wcp/go/
And then you can simulate the tournament and get betting advise as you go.

As far as a contest goes, what I'd like to do is offer a $250 prize to the person who comes up with the best set up match-up predictions PRIOR to the tournament start. On the starting day of the tournament I will take a copy of all of the saved ratings files and at the end of the tournament compare who was most accurate in there pairwise matchup score predictions when the two teams met.

Additionally, anyone who wants to enter needs to reply to this thread and ask that their ratings be considered.

This is one of those things where not many people know about this software, and it's all in good fun -- so please if you know someone who is a fan of the world cup and considers themselves somewhat experienced, please pass them the link to this thread. I'd love to get some feedback and see people try to take it seriously. Please help spread the word!

rgdot · « **Reply #1 on:** May 28, 2010, 06:41 PM »

Spreading the word and participating too

And by the way Forza Italia, not that I think they have the team to win but still

mouser · « **Reply #2 on:** May 28, 2010, 06:48 PM »

I think the prize money is probably the least interesting thing about entering.. Most of the fun is in seeing how one's pairwise match-up predictions effect overall tournament predictions and betting advice.

Although i'm not a gambling man by nature, in the interest of scientific inquiry, if there are a number of good ratings entries into the contest, i will authorize the betting of a few hundred dollars based on the ratings submitted, and we'll hand the money back out to the entrants in some fashion.

More than anything i just want to believe that the work that went into building this tool is actually put to use

Maybe we can give out a few mugs to some entrants as well.

mouser · « **Reply #3 on:** May 28, 2010, 06:50 PM »

ps. If you do plan on creating a ratings file and entering it -- you will want to have a forum account and log into this forum so that your ratings file is protected from someone else modifying it. if you are recognized only as a "guest" then your ratings file will be publicly editable.

rgdot · « **Reply #4 on:** May 28, 2010, 06:52 PM »

Doing it for fun, not any prize, because my bias always and almost entirely unconsciously gets the best of me any way

Shades · « **Reply #5 on:** May 28, 2010, 11:24 PM »

Yep, same here...'Mechanico naranga' (the Dutch team, as they are called here in the South Americas) never fulfills their promise somehow. But no matter how much disappointments I have endured over the years, there is always that glimmer of hope.

Soccer really brings out the nationalist in people, I guess

ljbirns · « **Reply #6 on:** May 29, 2010, 08:51 PM »

I'm in. If I win I will donate the prize to Donation Coder. I am looking forward to the World Cup, even though I live in the US and baseball is my sport of choice..
I have run a number of simulations over the past 6 weeks. I pick SPAIN to win for what it is worth.
My entry under ljbirns of 5-29

mouser · « **Reply #7 on:** June 03, 2010, 07:06 AM »

I thought it would be worth while for me to post a screenshot of a simulation and visually demonstrate the difference between the simulator picking a winner and advising a bet.

Let's look at one person's ratings data used to simulate the world cup on June 3rd, 2010, with current betting odds. After running for 30 minutes or so here are the results:

Screenshot - 6_3_2010 , 6_55_46 AM.png

We see that using this person's pair-wise team match up ratings, the simulator predicts that Spain has the best chance of winning the tournament (27% chance) followed by England (17%), Brazil (14%), Germany (9%), Italy (7%), Netherlands (6%), Argentina (6%); this is shown by the blue bars.

However, looking at the far right column, we see the expected profit from betting on the different teams; red numbers mean a positive profit is expected (negative numbers mean predicted loss of money).

Note that none of the teams with a high likelyhood of winning are good bets(!) Why? Because too many people are already betting on them and the payoffs for betting on them are so low.

The simulator advises betting on Honduras and South Korea, because even though these teams have a relatively small predicted chance of winning (1% and 4% respectively), their high payoffs make the bet a smart one.

REMEMBER: THIS IS THE RESULT OF ONE PERSON'S RATING FILE ESTIMATING HOW THE TEAMS MATCH UP PAIRWISE -- IF YOU USED A DIFFERENT PERSON'S RATINGS YOU WOULD GET DIFFERENT RESULTS!!!

ljbirns · « **Reply #8 on:** June 03, 2010, 07:11 AM »

world cup on June 3rd, 2010, with current bedding odds

And I thought the world cup was soccer...........

mouser · « **Reply #9 on:** June 03, 2010, 07:12 AM »

fixed

ljbirns · « **Reply #10 on:** June 03, 2010, 07:19 AM »

And just as I was going to bet !

mouser · « **Reply #11 on:** June 03, 2010, 07:26 AM »

Here is another simulation run using a different user's ratings file:

Screenshot - 6_3_2010 , 7_22_01 AM.png

In this case, the simulator predicts Spain has highest chance of winning (25%), with Germany (23%) and England (18%) close behind.

However in this case, it recommends the best bet is on Germany, with an expected profit of twice what you bet.

mouser · « **Reply #12 on:** June 04, 2010, 03:07 PM »

So has anyone made a complete set of ratings they feel good about and think are reasonable? I ask because i'd like to place some bets using them.

ljbirns · « **Reply #13 on:** June 04, 2010, 03:19 PM »

My entry under ljbirns of 5-29

I'm in. I did not include second round games for team that I believe will not make it past the first round.

Lew

rgdot · « **Reply #14 on:** June 04, 2010, 03:28 PM »

I believe mine is reasonable
05-28 rgdot

Lashiec · « **Reply #15 on:** June 04, 2010, 03:37 PM »

EA's own simulation agrees with your initial findings, mouser. My rarely used national pride is betting hard on that outcome as well

ljbirns · « **Reply #16 on:** June 04, 2010, 06:51 PM »

mouser · « **Reply #17 on:** June 04, 2010, 06:54 PM »

i think i ma have used an earlier version of yours in my first example, ljbirns.
btw it would be nice to crop your screenshot.

ljbirns · « **Reply #18 on:** June 04, 2010, 07:04 PM »

Thanks Mouser
Cropping is not my strong suit

mouser · « **Reply #19 on:** June 04, 2010, 07:29 PM »

unfortunately lj, your predictions don't make for a very appealing bet. although there is a 9x expected payoff from a couple of bets, which is fantastic, the actual chance of those teams winning are so low that it's very unlikely you'd win. though the honduras bet is attractive at 1000 to 1, if they really did have a 1% chance of winning. plus it would sure be a nice story to tell if they won

i think the other example run i showed is from rgdot. his predictions have a more attractive clear bet, on germany -- in the sense that if his predictions are right you have a pretty good chance of making a reasonable profit.

mouser · « **Reply #20 on:** June 04, 2010, 07:33 PM »

a smart thing to do, for you people creating ratings -- is to maybe do it in iterations.

that is, now that you see what bets the WCP simulator is suggesting, go back to your ratings and see if there are any games involving those teams that the simulator suggests betting on, that upon further reflection you would change your prediicted score for.

and do that iterative tweaking of your ratings until you are satisfied and the recommendations are stable.

mouser · « **Reply #21 on:** June 04, 2010, 07:33 PM »

i'm going to bet a few hundred dollars based on your guys' predictions -- and anyone else who publicly posts on this thread, so fame and fortune await you.

anodyne · « **Reply #22 on:** June 04, 2010, 10:29 PM »

So has anyone made a complete set of ratings they feel good about and think are reasonable? I ask because i'd like to place some bets using them.
-mouser (June 04, 2010, 03:07 PM)

My predictions are I think not bad; I revised them after my team (England) ended up doing rather better than I think they actually will, because of my bias when forecasting individual matchups. The revisions IIRC involved forecasting rather better results for Spain & Argentina, and rather worse for England.

I would not wish to place too much credence on the forecasts for games between the weaker teams, especially when they have not met recently. FIFA of course does this when they publish their rankings ... but IMO once you get below about the top 20, you're making it up ...

mouser · « **Reply #23 on:** June 04, 2010, 10:33 PM »

It would be fascinating to do some sensitivity analysis on the data -- that is, to see how much differences in ratings affect the final predicted outcomes.
I suspect that changing around the scores of weak teams moderately wouldn't make much of a difference.
I could add a feature that increased the noise of weak teams.

anodyne · « **Reply #24 on:** June 04, 2010, 11:52 PM »

It would be fascinating to do some sensitivity analysis on the data -- that is, to see how much differences in ratings affect the final predicted outcomes.
I suspect that changing around the scores of weak teams moderately wouldn't make much of a difference.
I could add a feature that increased the noise of weak teams.
-mouser (June 04, 2010, 10:33 PM)

Changing the forecast of results between weaker teams is not likely to change the predicted percentages for the top teams. However, if the "best team to bet on" is e.g. 100-1 or 500-1, then changing the forecast for weaker teams could (obviously) change that statistic.

I think the predicted outcomes are in fact likely to be relatively sensitive to individual forecasts. When I made my changes, I didn't have to change too many of the forecast results to end up with a noticeably different set of final percentages.

It would be interesting to use the so-called "Wisdom of Crowds" and average the individual forecasts for each pairing, and then use those averages to generate a set of percentages. You might have to allow fractional goals to do this of course ...