topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Wednesday December 11, 2024, 11:32 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: Digg, the wisdom of crowds, the hive mind, netscape, and competitors  (Read 31813 times)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
there's been a lot of talk recently about the new sites that are planning on challenging the popularity of digg.com.

for example see:
http://www.readwrite.../digg_ceo_jay_ad.php

Digg CEO Jay Adelson took time out of his busy schedule to email me some thoughts about Netscape's new digg-inspired community news site. Jay's thoughts below, but first some context. I've written two posts about the new Netscape site. In the Read/WriteWeb post I had two main points:

1) I think introducing paid editors into a community site may end up being as problematic as the 'hive mind' that it aims to prevent - because it introduces potential bias and favoritism.

2) The prominence of internal links and editors influencing discussions with "commentaries", IMO deflects attention away from the actual articles - which leads me to think Netscape wants to keep people onsite, in order to expose them to more advertising (which there is a lot of on the new Netscape site). This of course is an old-style portal strategy.

I followed that up with a ZDNet post which suggested that the paid editors now hold the balance of power - and how appropriate is that for a community site? I also pointed out that because Netscape has released a working version of non-tech categories before Digg, that this could spell trouble for Digg as it attempts to expand beyond tech.


Personally -
This probably won't win me too many fans here, but i generally consider myself an anti-hive mind type.

I should start of saying that I find digg.com to consistently have great links.  and i think the ideas about "the wisdom of crowds" are *generally* insightfull. letting lots of people indicate whats interesting or good can usually lead to great results.

however i am genuinely disturbed by the fadish nature, the susceptibility to gaming the system, the influence of organized groups biasing results, and the general randomness that results.

these problems aren't unique to community run sites - they are prevelant in most media - where silly items can get huge amounts of attention if they happen to get some momentum on a slow news day, while genuinely important issues can be completely ignored because it somehow doesn't excite the masses.

SO,
i find myself much more in favor of finding a way to combine the community approach of these sites with some sort of higher level "editors" or filters, humans with background knowledge and some ability to objectively make a decision about the relative newsworthiness of subjects.

just my 2 cents.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
i hope other people will chime in with their views, because we are going to be adding a kind of system like this to dc soon..

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 633
    • View Profile
    • Donate to Member
Never under-estimate the stupidity of large crowds.  I'm totally serious about this. 

It's amazing what "normally rational" people will do when they're in a crowd.  We've all see the same thing in email and forums.   People who normally wouldn't say "boo" start flame wars all the time.  Digg and similar sites can suffer from the same type crowd/email mentality.

Mouser, if you can find a control mechanism to stop the stupid and find the forgotten stuff, you'll do well.  :Thmbsup:
Insert Brilliant Sig line here

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
It's amazing what "normally rational" people will do when they're in a crowd.


this is a really good point - don't know why i haven't thought to cite these facts when arguing against "wisdom of crowds" concepts.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
I heartily agree that the "meta"/hive/aggregate approach is prone to all kinds of abuse, inaccuracy and other unpleasantries. Unfortunately I don't see the prevailing alternative (general mass media) doing a necessarily (or consistently) better job. There are a few good tech blogs that point out cool stuff like Digg and others do, but amusingly enough many of those writers are now citing Digg, etc. as sources. And how is this terribly different from Slashdot (where the crap-to-quality ratio is also questionable)? There are "editors" at Slashdot, but it's not like they're looking for "quality". They too are looking for what will be of greatest interest to their audience. So why not go right to the source and let the audience themselves decide directly?

Another thing to ponder in this whole consideration is "what is quality/worthwhile"? A reviewer of anything is generally only worthwhile to an individual if they know the reviewer's style and preferences. No one can truly "review" something without personal bias. So any given individual's assessment of what's interesting is only as reliable and useful as each person deems. One person's "must read daily blog!" is another person's snooze fest. Some people like the New York Times, other people think it's a trashy rag.

So how does this relate to the aggregator/"meta" sites? Well, if a reviewer is inherently biased, "the public" is inherently generalized and, in some sense at least, less "biased" as a result. With a news story's top billing being the result of a 1000 people's votes it immediately becomes interesting for that fact alone. Why are these 1000 people so interested in this thing? Does that imply everyone *should* be interested in it? No, but at the very least it's something to pay attention to for anyone interested in sociology. :D And chances are if you're interested in the same topic, you'll be interested in the story too.

Anyway, do we really imagine any other news outlet really does things differently? How is it determined what will be front page or even be run at all? Groups, editors, etc. The size of the group is the main differentiator here? Or that they're "experts"? Are these people qualified to better inform us as to what is interesting? The whole idea of an expert being better at *that* than a larger group of people voting directly seems kind of illogical to me. And ultimately such editors and editorial committees answer to the readers anyway, so directly or indirectly the results are similar.

Besides,  I don't think anyone is making any claims that Digg, etc. is giving you anything *but* what is popular, and in that they are succeeding admirably. If someone is interested in popular, then they know where to find it, and that's perfectly valid. Most of these sites also pare it down a bit, narrow the target from just "popular" to something popular in a given area - popular tech news for example. So if I'm interested in technology we are looking at the aggregate results of voting from other people who are also interested in technology (presumably). In theory many hands make light work, so 1000 people putting their most interesting stories of the day into a big hopper and seeing the best float to the top has genuine appeal. It seems to me it means we waste *less* time experiencing content that isn't useful or interesting. And isn't that a potentially positive thing?

So then if the results of these sites *are* of legitimate interest and use, if there really is nothing wrong with the aggregate approach to content rating (as long as you see it for what it is), then what is really at issue? It seems like it's the potential for corruption. But virtually any "system" is vulnerable to bribery, fraud, etc. A single reviewer can easily be bought out, an editorial committee can be bribed or blackmailed, a voting site can be hacked and votes can be spoofed.

Ultimately it's up to everyone to see a given "media outlet" for what it is - know its methods for collecting, categorizing, and prioritizing content - and then use it according to how useful it seems to them. This is true of any source from which you get your news. So, given that, what I'm mainly interested in with all of this is why there is a big backlash against it. Frankly I feel there are far more insidious things going on in terms of information dissemination that are worthy of our discussion and efforts to improve.

All that being said I'm not surprised to see people so bothered by this "most meta" thing. I expect the fervor will die down in a few years, both for visitors (and voters) of these sites and for critics. They'll either become a legitimate part of our culture and be accepted, warts and all (or hopefully be improved, at least to be the best at what they are intended for), or they'll fall by the wayside. Either way people will probably forget what all the hubbub was about. And perhaps that's the real danger - that people, in general, are not critical and conscious enough of their media outlets and the methods they employ; that when things like this happen they are generally forgotten about sooner or later and the potentially legitimate issues that existed are swept under the rug. The real question is whether the greatest threat is on our table now in the form of meta content sites, or if it is already well integrated into our culture...

[edit] Ooo! P.S. Let's not forget the fact that a lot of the tendency for otherwise nice people to be a-holes online actually comes from the unfettered anonymity, not the "crowd mentality". Crowd dynamics do create similar effects sometimes, but in general those dynamics are most in effect in real life, real crowds. And again in that case perhaps it is partly the anonymity afforded by a large group of people, the difficulty of picking one person out of the crowd. So perhaps the root is the same. But I think it's important to note that it is not simple "crowd mentality" that allows for these disturbing personality switches. It's also important to note that anonymity affords many people the ability to express positive or insightful things they might not otherwise say or do. [/edit]

- Oshyan
« Last Edit: June 16, 2006, 11:27 PM by JavaJones »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
fantastic points as usual oshyan.

and the point about slashdot is excellent, because i have found it much much worse in all respects than digg (except maybe for skill level of commenters).

i do think that, in theory, the idea of having some people with a technical background in the area and good background domain knowledge acting as a kind of filter would be useful to add to the mix of a social news site like digg.com.

those people could add some context to links, or say: no this link that you are all highlighting is really an insignificant thing - but if you're interested in this then maybe this other one is worth highlighting.  or: everyone is linking to this site, but if we list that site then surely we should list these others: x,y,z.

in other words, i'm suggesting leveraging the wisdom of the crowd, but adding a layer on top of that, or a filter, which calls on people with some domain knowledge and expertise to act as a balancing force to eliminate some of the faddish effects, provide additional relevant info, and prevent abuse.

side note:
there have been some (credible?) accusations that digg.com really has a kind of unspoken system where certain people (friends of the site) can artificially squash or promote stories.  and one could view the idea of certain people having more points to vote with as being a step in the direction i am talking about, so guess the value of this idea is very much dependent on having reasonable people as your expert filters..

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
Yes, as with anything no system can ever really compensate for stupidity, selfishness, or corruption of those in power, as long as there are human beings in control at some level anyway. That is part of why totally automated sites are interesting, because in theory they remove at least the majority of those issues, but of course they have issues of their own, and are still open to being hacked, corrupted, etc. through other means.

Ultimately the best thing one can do, I think, is just build up a good body of trustworthy, intelligent editorial staff. Perhaps *they* can use sources like Digg, etc. to save them the time of having to search everything themselves for worthwhile stories (I think a lot of news outlets probably already do this). But ultimately they are the arbiters of the content and as you said one of the valuable things they can do above and beyond what an automated system can do is provide expert commentary, as well as find, evaluate and highlight competitors and similar sites/articles/etc. In the end if people are rewarded with good, unique content (or content they enjoy - "good" is so subjective) they will continue to visit and read your site, whether it's automated or run by an editorial body.

- Oshyan

alex3f

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 12
    • View Profile
    • Free Knowledge Exchange
    • Read more about this member.
    • Donate to Member
IMHO, the arguments about wisdom of crowds mainly result from a wrong word choice.

Crowds are large and very loosely organized groups of people. I think, it is not anonymity per se, but rather perceived individual ineffectiveness and irresponsibility makes a crowd. In a large and disorganized body of people, it is easy to think that individual decisions has no impact on the group behavior. As a result, people don't feel any responsibility and actually stop thinking as individuals. In this situation the crowd is like a superfluid liquid free to move in any direction without any reason and resistance.

This is not what we see in many social websites. I think it is quite wrong to use the same term for participants of a social website. In every case, where a group of people is intelligent, we find structural organization in their activity. People at those websites are organized by social software that structures their interactions. Of course, it is not a traditional organization motivated by managers through employment obligations and compensation. Instead, this is a participatory organization, where social software coordinates people's activities in a more subtle way. This can be done just by altering the effort associated with different kinds of activity.

IMHO, social software has some advantages over human executives. It is more scalable, more responsive, fair, and transparent.  It can be protected from corruption at a much lower cost than human management hierarchy.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
very good points alex.

i think a balanced view is warranted.
it seems to me there are clear advantages (breadth of coverage, resilliance to outliers, fast reaction speed) but also very real disadvantages (the potential for abuse, random trend/flocking effects, and lack of expertise in a domain).

for me the interesting thing is how to balance these things and protect against the disadvantages.

my concern is that the big social sites we see now (digg for example) seem extremely open to abuse and manipulation, and i suspect there is a lot of that going on in digg and similar sites.  whenever there is such a monetary gain to be had from easy manipulations it's going to happen.

it will be interesting to see how in the future people manage to find ways to immunize social networks from these problems..

Rover

  • Master of Smilies
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 633
    • View Profile
    • Donate to Member
alex3f - How does a social web site differ from the crowd mentality?  I think you nailed the "reasoning" behind the crowd actions (or thinking) but I fail to see how a site like digg really differs.

They still get "caught up" in the moment and stop thinking.... a lot like most 14-18 year olds :)

Insert Brilliant Sig line here

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
well, not to speak for alex, but he was clearly differentiating the kind of out of control actions by crowds,
with the kind of spontaneous self-organization of structured roles that emerge in more general social organizations.

alex3f

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 12
    • View Profile
    • Free Knowledge Exchange
    • Read more about this member.
    • Donate to Member
I think, success of collective intelligence and social websites in particular is best described in terms of evolutionary computation. There are some minimum conditions necessary for evolution to take place. These are diversity, change, and selection. Internet websites satisfy diversity requirement quite easily due to the global nature of the Internet. In addition, these websites are built in a way to encourage humans to perform change and selection operators. In digg's example, the activity is structured around suggesting a new message (change) and promoting an existing message (selection). This provides for the simplest evolutionary process that will slowly drive the population of messages toward increasing fitness. Most social websites I had seen so far, implement at least a minimal evolutionary model.

If we look at a crowd from the same perspective, we notice that it lacks every necessary condition. Members of crowds normally lack diversity, they often come from the same background and have quite similar opinions. It is often homogeneity that brought them together in the first place. It is quite unsafe to try to challenge the opinion of a crowd so crowd doesn't provide some change mechanism. There is not much opportunity for selection either due to lack of diversity. Crowds are subject to positive feedback and information cascades, see James Surowiecki "The wisdom of crowds" (his analysis why crowds fail to be intelligent).
« Last Edit: June 27, 2006, 12:25 AM by alex3f »

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
that's actually a really interesting way to think of it.
viewed in terms of selectionist/evolutionary computation, you need:

1) a way of introducing novel changes
2) a way of favoring the population of good changes (or new submissions)
3) a reasonable cost function that gives you something useful

you and i have talked before about implicit and conflicting cost functions - one way to look at the potential problems of digg is from the standpoint of it having degenerate and conflicting cost functions.

groups of people are highly motivated to increase the survival of memes (links) for their own purposes, for their own gains.  the implicit cost function therefore becomes not the selection and survival of links most interesting to the readers, but rather the links belonging to the few people who are gaming the system.

this is actually coming up recently in the discussion of the very small % of readers of digg who ever vote on anything.  if all the people voted, you might have a more representative cost function.

but instead we have a very tiny # of people (colluding to different extents) to try to promote their memes. so a natural question might be: how could you improve the cost function to better reflect the aims of the site, and how could you improve the novelty function to get a better distribution of potentially interesting content.

the ideas of adding experts is to try to improve the cost function.  as for adding more diverse novelty.. not sure..

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
so a natural question might be: how could you improve the cost function to better reflect the aims of the site

Yes, yes! This is why reward for input interests me so much.

Great discussion guys. A lot of new insight here I think.

- Oshyan

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
a nice blog post directly on these subjects:

http://www.shmula.co...the-wisdom-of-crowds

Digg isn’t De.licio.us

Digg, I believe results in less intelligence. The crowd is imitative; there are many followers in this space. De.licio.us, on the other hand, is the result of truly independent choices: it’s simply personal bookmarking, made public. The assumption is that a human will bookmark what is useful to them; making that bookmark public allows others to benefit from your discovery. In contrast, Digg is very supportive of crowd psychology, which can be very dumb, mimicky, and in the end not all that helpful.

alex3f

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 12
    • View Profile
    • Free Knowledge Exchange
    • Read more about this member.
    • Donate to Member
I agree that digg is too imitative. In addition, Digg allows only to promote a website, but not to demote it. Once some website of little value is promoted by a small group of people of critical size, it is bound to go up just because it has that critical mass. This is similar to stock market where playing long is encouraged, but playing short is discouraged. Together with high level of imitation in both cases, this design is known to produce bubbles. The other design for web aggregator would be to allow users to both promote and demote. I used it to make yellow pages that adapt to user preferences, reddit.com uses it for website ranking. In addition, the total counts of promotions and demotions in this design are not shown to evaluators, and this makes evaluation less imitative and less subject to information cascades than in digg.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
From downloadsquad.com:

...
It's not entirely inaccurate to call the new Netscape (which replaced the old Netscape.com late last night) a "Digg clone," but it's surely not the first. But if I asked you how many other Digg clones are out there, what would you guess? Twenty-five? Fifty? Yeah, more like 200+. Blog 3spots has created a constantly-updated list of the Digg-alikes on dozens of topics and in dozens of different langauges.
...



from downloadsquad.com

app103

  • That scary taskbar girl
  • Global Moderator
  • Joined in 2006
  • *****
  • Posts: 5,885
    • View Profile
    • Donate to Member
Speaking of the stupidity of crowds, here is one that will make you laugh:

This website popped up more than 6 months ago and looked rather mysterious:

http://www.eon8.org/

You probably won't be able to reach it and if you can you will not see what visitors saw before today.

Google's cache shows what it looked like before.

Many speculated as to the purpose of that site, thinking all kinds of insane things:


We were amazed to discover that the site was instantly linked with terrorism, simply for the fact that it seems mysterious. Evil was the number one first impression people had of the site, in spite of the fact that there are no threats on the site. The only thing Eon 8 says is "We don't want you here". Nothing else.
Other less disappointing opinions were social experimentation (which was correct), James Bond movie viral marketing, and promotions for video games.
For many people, being faced with a countdown timer was an instant reason to try to shut down or hack the site. This is a worrying reaction, that if someone doesn't understand something they must destroy it. As a result, the servers have been hit quite hard these last few days, but luckily 99% of the 'hackers' could easily be described as 'l4me n00bs'.
Another worrying example of paranoia was how quickly people would jump to conclusions, such as telephoning the registered owner of a dog seen in a photograph on a server that hosts a page that links to eon8.


Last night the countdown ran out and the truth was revealed.

http://24-7gaming.net/$/eon8/deployed21b.html  (mirror of original)

and the post from unfiction.com that is referred to in the creator's message:

http://forums.unfict...iewtopic.php?t=15629

When it was found to be a hoax or experiment, the wikipedia page related to it was promptly deleted by vandals and has since been restored & locked :

http://en.wikipedia.org/wiki/Eon8

Even jokes about it have popped up, speculating what it all was supposed to mean:

http://uncyclopedia.org/wiki/Eon8
(others links cane be found on the wikipedia page)

Shortly before the countdown ran out last night, there was an irc channel on the GameSurge network (#eon-8) that had over 1000 people in it gathering together to wait & see what would happen. When it was all revealed they went nuts spamming the room with obscenities, totally upset that the world didn't come to an end.

and here is a rather interesting article on the whole thing:

http://www.vitalsecu...07/eon8-summary.html

So what is it that 'normal rational people' do when they don't understand something? They fear it, assume all kinds of untruths, and seek to destroy it.

So much for 'crowd wisdom'.

The creator never revealed why he did this experiment, only implying that he was curious what would happen and just felt like doing it.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
Very interesting experiment.

- Oshyan

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
great post app, i love this quote from the summary:

Eon8 (a mysterious site that developed a creeping sense of inevitable doom, thanks to some nifty tricks employed by the site creator) was simply an experiment - the guy who made the site just wanted to see what people would do faced with a lack of information. The answer was - go absolutely freaking insane.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
Just read the "Digg isn't de.licio.us" blog post. Am I missing something? I didn't find it particularly insightful at all. His examples and conclusions don't really jive all that well for me. Particularly the examples of "crowd thinking" from James Surowiecki. His examples seem to be much more representative of when artificial external forces try to control the crowd wisdom. They are mostly heirarchical systems that are only a "crowd" in the sense that a lot of people are involved. The intelligence failures on 9/11 for example were largely the result of beaurocracy, not crowd thinking. He then points out the failures of Digg-like sites vs. more individualized approaches like De.licio.us, which I do think is an interesting comparison, but he fails to show what may have been advantageous about any pre-Digg system, or the simple lack of Digg. Would the world be better without Digg? That seems to be one of his arguments, at least indirectly, but he never supports it with the counter-examples. This guy's conclusions may have merit, but not based on the train of evidence and ideas he presents IMO.

- Oshyan

GHammer

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 34
    • View Profile
    • Donate to Member
Personally I find individuals who consistently provide good information.
I pay attention to their recommendations.
Sites with enough of those peolpe get visited frequently.
Sites that allow voting/rating usually fall to fanbois.

I use I.E., Office, and I really like WavPack.

I do not care for the group mentality of those who push firefox and open office and flac on everyone with little except a religious arguement.

Crowds/large groups are useless as a source of information. Fun for advertisers though and that is why these types of sites get so much media notice, they drive revenue. Not because they are so useful.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
Perhaps they're not useful to you, but I've personally found a lot of useful and interesting stuff through these sites. I kind of feel like the "this or that but not both" attitude is prevailing here. People are treating Digg, etc. as if they were someone's *only* source of info. Sure, if anyone uses it like that, it's not healthy and won't give you a balanced view and a broad spectrum of useful info. But Digg, etc. *are* useful. Just because they are not the "best" source of info doesn't mean they're not a good one. One should always strive for a balanced approach to finding information. Don't trust just a few bloggers to have all the right info and answers, and don't trust the "group mentality" to know all either. See what they have to say about each other and what differences there are between their two perspectives on the same things - *that* should be really instructive.

- Oshyan

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
hahahaa come on digg, this is getting annoying and stupid - top story on digg technology today:

mycaps Screenshot - 002 , 06_32_PM , Jul 06 2006.png

this is getting stupid. i'm rapidly losing patience with digging through all the incestuous self hype and crap on digg.

JavaJones

  • Review 2.0 Designer
  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 2,739
    • View Profile
    • Donate to Member
lol, yeah that's pretty bad.   ;D

- Oshyan