topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • December 16, 2018, 10:21 AM
  • Proudly celebrating 13 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: When is a raven like a writing desk?  (Read 1543 times)

Edvard

  • Coding Snacks Author
  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 2,965
    • View Profile
    • Donate to Member
When is a raven like a writing desk?
« on: November 05, 2017, 04:02 AM »
Or more succinclty, when is a turtle like a rifle?

Researcher: ‘We Should Be Worried’ This Computer Thought a Turtle Was a Gun
...
By manipulating a few pixels in an image, you can trick a neural network—even one that's great at recognizing cats, and is trained on hundreds or thousands of images of felines—into thinking that it's looking at something completely different.



from CodeProject News

Stoic Joker

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 6,483
    • View Profile
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #1 on: November 07, 2017, 03:24 PM »
Or more succinclty, when is a turtle like a rifle?

Well the do both store all their energy in a shell.


Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 8,595
    • View Profile
    • The Blog of Deozaan
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #2 on: November 07, 2017, 04:05 PM »
Maybe I missed it, but it seemed to me that neither the article nor the video explained what about the change caused the AI to see the turtle as a rifle. :huh:

Stoic Joker

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 6,483
    • View Profile
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #3 on: November 07, 2017, 05:19 PM »
Maybe I missed it, but it seemed to me that neither the article nor the video explained what about the change caused the AI to see the turtle as a rifle. :huh:
Since we really don't know how extensively these systems are actually being used...there's probably a - security - reason for that being )intestinally) omitted. Kinda like when the Myth Busters demonstrated how high security biometrics could be fooled...without actually showing you how they could be fooled.

Either that or the hack is waiting on a patent application - You just can't tell these days...

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 8,595
    • View Profile
    • The Blog of Deozaan
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #4 on: November 07, 2017, 05:38 PM »
Could be.

I wasn't looking for exact steps to reproduce the outcome. I would have been satisfied with a generic answer, like "we made the entire thing have a woodgrain texture like the butt of a rifle instead of a natural shell texture" or something.

But maybe even that would have had too serious of security/patent risk.

Edvard

  • Coding Snacks Author
  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 2,965
    • View Profile
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #5 on: November 07, 2017, 09:12 PM »
Maybe I missed it, but it seemed to me that neither the article nor the video explained what about the change caused the AI to see the turtle as a rifle. :huh:

They didn't detail what the change was, but if you watch the video, notice the changes in the model.  The size is slightly larger, and there are weird color swirls on the shell sections, and some spiral markings on the underside, as well as two holes.  I suspect those color swirls and markings were precisely engineered to produce an "adversarial example".  Kinda like how changing just a few pixels in a QR code can represent a completely different batch of information.

TaoPhoenix

  • Supporting Member
  • Joined in 2011
  • **
  • Posts: 4,605
    • View Profile
    • Donate to Member
Re: When is a raven like a writing desk?
« Reply #6 on: November 19, 2017, 10:05 PM »

One interesting thing here is I am reminded of a gripe I used to have about the Loebner AI tests - when I last looked at that stuff about 5 years ago, the testers often set about "hard abusing" the "black box replier" with semi-bogus questions "knowing they were in a Loebner test".

I got grumpy because it seemed few / none of the entrants had put in "anti-troll" code to deal with stuff like that. To me, anti-troll code should be fairly easy to write, because the bogus questions are often bogus, so "truncate low" with a defensive sweep like "scan nouns and compare class - why is a cake and the Queen in the same sentence?"

Same idea here - unlike those animorph pics as joke memes, to the human eye this is "clearly a turtle" so maybe use 3 scan algs and they should all "converge on the answer and if not, kick it to a decider module".