Artificial intelligence bests humans at classic arcade games - DonationCoder.com
HOME | Blog | Software | Reviews and Features | Forum | Help | Donate
Click here to
donate and join now!
Welcome Guest.   Make a donation to an author on the site April 18, 2015, 06:37:58 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
The N.A.N.Y. Challenge 2014! Download dozens of custom programs!
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1]   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Artificial intelligence bests humans at classic arcade games  (Read 674 times)
mouser
First Author
Administrator
*****
Posts: 34,401



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: February 26, 2015, 12:09:23 PM »

There has been some buzz recently around a few articles that demonstrate machine learning in the video game domain.

Here's one writeup:

Artificial intelligence bests humans at classic arcade games
http://news.sciencemag.or...mans-classic-arcade-games

For the academically inclines, I would recommend:
Playing Atari with Deep Reinforcement Learning
http://www.cs.toronto.edu/~vmnih/docs/dqn.pdf

Which talks in detail about the methods used.

The use of the term "deep" seems to me to be as much about coming up with a catchy term that has gone viral and is being hyped like mad -- and has little innovation behind it -- but the new wave of practitioners using neural networks for large scale problems are getting undeniably impressive results.  

Again, getting back to the video game results:

There is nothing particularly novel in the approach -- the domain is wonderful, and the basic focus on using the same architecture and parameters to tackle a large collection of learning problems -- and using large dimensional raw input, is great.  And the results are impressive.  Again -- in my mind this is more a story of the new wave of practioners who are getting very good at leveraging fairly standard neural network techniques on larger and larger problems.

Having said that, this line of work offers little qualitative improvement on the hard problems in AI -- on serious multiscale hierarchical planning, scene recognition, etc.  For that we are still waiting for some paradigm shifts.
Logged
TaoPhoenix
Supporting Member
**
Posts: 4,062



0 - 60 ... then back to 0 again!

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #1 on: February 26, 2015, 01:03:16 PM »

My enthusiasm for AI often exceeds my editing, but here's a couple of thoughts.

Certainly "Q or Reinforcement Learning" feels like a bit of partly complicating the obvious. So it seems that from what little I know of chess programming, they can't guarantee the best move, so they  "monte-carlo simulate it" - aka run scads of iterated tests and then the program "tends to notice that such a certain X move tends to lose or win". Sometimes the other side escapes, but it's that "tends" that matters.

So in Space Invaders, if you get stuck on the side, you "tend to get trapped" because you're missing half your movement range. In certain conceptual ways, that feels like "sorta easy" programming to me.

What I don't see is any interaction with "precursor tutorials" such as if your friend comes over and hangs out with pizza for an hour to show you stuff. You still have to play the game, but it sounds like the games tested were "easy to play with clever middle level tricks". So unlike hardcoded strategies, you make your friend's suggestions "a hypothesis" - that's how he always played, so the computer looks there first with at least a baseline. Then some of the friend's suggestions turn out to be sub-optimal. (I think that was called H:0 and H:A in statistics. Yes?) PacMan sounds like it would be a good test here.

Take a game where the human says "I don't know what I am doing" regarding gameplay and I bet the computer will get stuck.

« Last Edit: February 26, 2015, 01:10:15 PM by TaoPhoenix » Logged
mouser
First Author
Administrator
*****
Posts: 34,401



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: February 26, 2015, 01:18:52 PM »

I don't mean to sound harsh, and please don't take offense, but I don't think it's helpful saying things like "Q or Reinforcement Learning feels like a bit of partly complicating the obvious" without understanding the math and foundation for these algorithms.  Q-learning and other reinforcement learning techniques are elegant, efficient, and based on very sound principles.  They aren't the holy grail of human-level intelligence but they are very elegant algorithms. There are great books on this stuff for those who want to learn about it.  The now classic book on reinforcement learning is by Sutton and Barto (here), which I recommend.  

ps. Your idea to use an expert to initialize training and start as a baseline is an area of active research in current AI -- and in fact was part of the early days of AI.
« Last Edit: February 26, 2015, 02:39:25 PM by mouser » Logged
Pages: [1]   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.033s | Server load: 0.05 ]


Share on Facebook
submit to reddit