Author Topic: try my new experiment with HotOrNot pictures! (Read 11546 times)

urlwolf · « **on:** January 29, 2007, 07:49 AM »

Hi,

As some of you know, I do research on cognitive science. It is usual to run experiments over the web. I have great plans for psychology experiments, mainly if they are fun. This one uses HotOrNot pictures, so it should be.

It takes about 20 min. You will see some pairs of pictures repeating, that is normal. By doing this experiment you help me in my research, but no other benefits are implied.

The experiment is not quite ready (I need to iron some bugs out), but you can have a look anyway and help me finding problems before it goes live. Here:
http://hon.mashupslive.com/test/
username jose
passwd jose
It's protected to avoid google indexing it.

Update: just checked the instructions I sent.
There are two problems with the instructions, here:
"DESCRIPTION: You are invited to participate in a research study that
investigates" (ENDS): it should continue "how people judge attributes
such as attractiveness".
It says "I am the man/woman" where it should say "I am a man/woman" *shrugs*.

Also, it currently loads only the female pictures, no matter what the
participants chose. The coder knows about these problems, and maybe by the time you read this, he'd have fixed it.

PS: I asked Mouser if it's ok to post experiment links here, and he's ok with it. I'm saying this because I don't want to post something that could be considered advertisement (I need to recruit people anyway) in a nice community like ours.

mouser · « **Reply #1 on:** January 29, 2007, 08:04 AM »

Happy to help a long time member test some stuff.. Im not sure if we are actually participating or just helping to test..
ps. it says to click start to begin but there is no start button on my page (Firefox) so i just put in my (fake) email and click ok and im stuck on that page. maybe the experiment is to see how angry people get when they aren't shown promised hot pictures of the opposite sex?

app103 · « **Reply #2 on:** January 29, 2007, 08:16 AM »

maybe the experiment is to see how angry people get when they aren't shown promised hot pictures of the opposite sex?
-mouser (January 29, 2007, 08:04 AM)

tomos · « **Reply #3 on:** January 29, 2007, 08:23 AM »

there is no start button on my page

ditto / IE

signed: angry young man

app103 · « **Reply #4 on:** January 29, 2007, 08:25 AM »

There is no start button when viewing the page in IE, but there is a red progress bar at the top, that the page with the pics automatically loads after it fills up.

Is this some sort of timer to allow the visitor time to read the page before it automatically begins?

Darwin · « **Reply #5 on:** January 29, 2007, 09:38 AM »

Hey! My post didn't "make it"... Just to report, I'm seeing the same thing here running Maxthon (uses the IE engine). I don't even get the red progress bar, though, just a long beige rectangle...

nudone · « **Reply #6 on:** January 29, 2007, 10:57 AM »

Hey! My post didn't "make it"... Just to report, I'm seeing the same thing here running Maxthon (uses the IE engine). I don't even get the red progress bar, though, just a long beige rectangle...
-Darwin (January 29, 2007, 09:38 AM)

me too.

urlwolf · « **Reply #7 on:** January 29, 2007, 10:59 AM »

aha! this is why I wanted to test this.
All this AJAX stuff is fragile, one has to test all kind of situations.

The 960 pics are loaded at the beginning, that's why there is a red progress bar.

Email is optional.

When the progress bar is complete (pics are loaded), then the experiment would show a start button. If this is not working (I just tested, It isn't; I tested it in Opera, FF and IE before sending this) the only explanation I have is that Andrey, the coder, is changing the code from local pictures to HoN pictures. Sorry about that, I'll contact him.

Will post again as soon as it's working.

Mouser: there will be pics of hot people there, don't worry

Nighted · « **Reply #8 on:** January 29, 2007, 07:35 PM »

Who says the pictures will be of the opposite sex?

urlwolf · « **Reply #9 on:** January 30, 2007, 01:25 PM »

Ok, here we go again.
Now it should work. Both Male and female pictures load directly from HoN.

No start button needed, so don't worry if you don't get one.

Note: please send the link to whoever you want, but do not tell them what it's about (I need people). You will be debriefed (i.e., I'll tell you hypotheses and results) after I got enough people for the statistical tests to work.

I have tested this under FF and Opera.

Note: you can select pictures from same or opposite sex, it's up to you

Thanks!

urlwolf · « **Reply #10 on:** January 30, 2007, 01:26 PM »

http://hon.mashupslive.com/test/
Forgot. No username/passwd needed anymore.

mouser · « **Reply #11 on:** January 30, 2007, 02:17 PM »

seems to be working now.

it advanced to the picture page before i had a chance to finish entering my email.

you should definitely add more information and reminders about what is going on - like a reminder about which picture i was supposed to be choosing (the one more attractive to me presumably).

i assume that i am supposed to be seeing certain pictures over and over again. if not, bug.

urlwolf · « **Reply #12 on:** January 30, 2007, 02:34 PM »

Yes. There are repeted pairs.
In fact you'll see each picture 8 times. Sometimes even the pairs are repeated. You think a warning should be present at all times about this?

Wow, so HoN loads really fast (no time to enter email!). That's good.

mouser · « **Reply #13 on:** January 30, 2007, 02:36 PM »

i saw many pictures many more than 8 times (im guessing like 30 times+) and pairs repeated a lot.

urlwolf · « **Reply #14 on:** January 30, 2007, 02:57 PM »

Actually, you are right, pictures can have 16 repetitions as well now that I think of it. There are 100 different pics, and a total of 960 repetitions, so the average is about 9.6 repetitions per pic. It's normal, and it sticks out a lot in a situation like this, where one really likes to see variety

So the instructions (summary: "just click on the one you find more attractive") are not evident and should be on screen at all times?

mouser · « **Reply #15 on:** January 30, 2007, 05:12 PM »

should be on screen at all times?

i think so.
also you probably know about such effects but you better really be prepared for people getting bored and starting to click relatively-randomly as the experiment goes on. it takes a lot of time to go through 400+ pairs and when you start seeing the same pairs over and over again you start to get really restless.

Darwin · « **Reply #16 on:** January 30, 2007, 05:14 PM »

Hmm... site never finished loading for me (Maxthon). I'll give it another go later on tonight. Busy getting ready to lecture on the vastness of geologic time and macroevolution...

nudone · « **Reply #17 on:** January 30, 2007, 05:16 PM »

Hmm... site never finished loading for me (Maxthon). I'll give it another go later on tonight. Busy getting ready to lecture on the vastness of geologic time and macroevolution...
-Darwin (January 30, 2007, 05:14 PM)

yup, same problem here.

tinjaw · « **Reply #18 on:** January 30, 2007, 06:01 PM »

Well, I made it from start to finish in Firefox 2 with no problems.

I am assuming this is still in alpha mode, so I also assume you won't put all participants though it in the manner it is in now. If it wasn't for a DC buddy, I would have stopped about 1/4 of the way through.

jgpaiva · « **Reply #19 on:** January 30, 2007, 06:03 PM »

If it wasn't for a DC buddy, I would have stopped about 1/4 of the way through.
-tinjaw (January 30, 2007, 06:01 PM)

Agreed. I also found the process a bit too tedious. And i ended up doing a few mistakes because i became distracted. (i think that's the idea, though. am i right?)

urlwolf · « **Reply #20 on:** January 30, 2007, 06:35 PM »

If it wasn't for a DC buddy, I would have stopped about 1/4 of the way through.
-tinjaw (January 30, 2007, 06:01 PM)
Agreed. I also found the process a bit too tedious. And i ended up doing a few mistakes because i became distracted. (i think that's the idea, though. am i right?)
-jgpaiva (January 30, 2007, 06:03 PM)

Hmm, interesting. I've been in many experiments that are intolerably boring (like deciding if a string is a word or not, thousands of times). The fact is that, for experimental hygiene, we need to use simple materials and have lots of repetitions. It is not the purpose of the experiments to bore people, believe me, although it's not for entertainment either.

Doing this experiment on campus, where students are used to boring experiments, would have made it feel like a very good one

believe me. That's how boring experiments normally are. People get paid to get bored. Doing experiments on line has many advantages (more diverse population, faster data collection, usually you don't have to pay people) but it is definitely true that the attention span of a person surfing is very short and you cannot bore them, or they leave. I'm all for designing game-like experiments. But sometimes (often), like in this case, this is not possible. Designing experiments is hard enough as it is, so if you add the 'playability' constraint, you may never get a design that works.

I also think that the current way of running experiments may produce data that are inaccurate due to low motivation. In this case, it doesnt matter since the effect I'm trying to get should work even if you don't pay attention, but motivation is a problem for most other effects. Damn.

I do appreciate that you went through it all just because I posted it here. The feedback I got is really useful.

mouser · « **Reply #21 on:** January 30, 2007, 06:53 PM »

Ah yes let me clarify:
I've seen some of the incredibly painful experiments that undergraduates do for credit and sometimes money.
This is about 1000000000x more fun than those.

If you were paying someone to sit down in a lab and take it I don't see any problem at all. My comment about distraction and boredom was more aimed at the case where you would be asking strangers on the web to participate for no reason.

Perry Mowbray · « **Reply #22 on:** January 30, 2007, 09:01 PM »

I do appreciate that you went through it all just because I posted it here. The feedback I got is really useful.
-urlwolf (January 30, 2007, 06:35 PM)

No dramas: nearly fainted when I saw 479 though!!

Do we get to get an update on your results??

- Perry