Survivorship Bias...an Insidious Enemy

[CodeTRUCKER]

Hi,

I recently was asked to read an article on "Survivorship Bias."  It is fairly long and it may take some people a bit to wrap their brains around the concept, but it is provocative and sound.  If one considers how pervasive this paradigm really is at every echelon of human existence, one has to wonder why thinking people have allowed Survivorship Bias to go so long unchecked and unchallenged?  While a search of the Internet would lead one to think Survivorship Bias is confined to the financial realm of investing, the author of the article allows us to see it painted with a broader swash.  This is good, IMO.

WARNING: Not to be too melodramatic, it is not so much the concept which will give you brain strain, but the inevitable and involuntary response to apply it will create quite a mental tussle (at least it has for me).  Assimilating the concept will require nothing less than a paradigm-shift in your thinking.  Personally, it has been almost eight weeks since I was challenged to read the article.  I have been trying to digest the ramifications from the first day and I still get a headache if I try to think it through to its logical end (haven't made it yet).  Further, in imagining what a world would look like if Survivorship Bias had been identified and precluded from society, I am led down a corridor which becomes quite complex.

At the end of the day... my two-month long mental sparring with Survivorship Bias leaves me with some questions which continually provoke me...

• Has there ever been a human society which was able to rise above the trap of Survivorship Bias?
• What would that culture have looked like?
• How would have technology advanced if it had been born and nurtured in an environment free of Survivorship Bias?
• How did Survivorship Bias become so globally accepted given the effects are so harmful?
• Is it even possible to completely un-learn Survivorship Bias?

Again, the read itself is long (you may want to print it out).  Anyway, here is what I suggest...
<> Find yourself a good hour(s) of uninterrupted time to read the article and absorb its contents. 
<> Grab yourself a friendly cup of hot tea or other familiar (favorite) beverage. 
<> Take your laptop, iPad, etc. to your comfy spot and get ready for your mind to be bent in a way never before conceived. 
<> Please report back when you've had a chance to let it sift through your considerations and have mentally applied it to your everyday life.  I am very curious to learn if my experience of the effects are common or unique.

One final thought.  After one reads the article please come back here and consider the following... 
DC is a community of coders, et.al., who have had to struggle through a variety of challenges of unruly design and syntax (read: failures).  In consideration of Survivorship Bias, one has to wonder how many unrealized "successes" never saw the light of day because the programmers were led away from the opportunity by the Pied Piper's notes?

Fair winds.

[Disclaimer - It is obvious I am zealous on the subject, so it behooves me to advise the reader I have no relationship with the author and I receive no compensation.]

[mouser]

Thanks for posting the link to that article -- I think Survivorship bias (there may be other names for it as well) is a really important thing for people to read about and understand.

Survivorship bias is the tendency to look to the "WINNERS" of some competition for clues to what they did right that enabled them to "win" -- rather than looking at the entire population (especially the "losers").

In many scenarios, it may be that the winners just turned out to be lucky -- and worse yet, they have convinced themselves that their success is explained by a host of skills and actions that turn out to be irrelevant -- and where the valuable clues to success can be found only in looking at what brought down the losers.

It's one of those things that makes sense logically, but is extremely difficult to avoid succumbing to even if you know you shouldn't.

[CodeTRUCKER]

...
It's one of those things that makes sense logically, but is extremely difficult to avoid succumbing to even if you know you shouldn't.

-mouser (March 01, 2014, 09:53 AM)

You're welcome and I am agreed.  Hence my headaches.  
Seriously, that is why I am asking the question why it has been allowed to be such an influential makeup in our persons.  Frankly, my enlightened awareness of Survivorship Bias has changed my life since I have allowed my investigation access to my most cherished of understandings.  I have even been provoked to look at the world differently.  Like I said, understanding the concept does take some doing, but applying it to your world will present some significant challenges.  Your mind will have to put on a good pair of work boots, coveralls and gloves because it will take work.

[Shades]

Interesting article. Life's lessons are learned by failure. Something similar was said by 40hz a few days ago.

And much smarter minds than my own have added that you can learn from failures of others as well. It's the finding out about the failure(s) of others that takes more effort. Something most people can't or won't do. After all, success stories spread more easily, which I attribute to other standard human behavior best caught with the saying 'Success has many fathers, failure is an orphan'.

[mouser]

I'm making my way through the article CodeTrucker linked to -- it has some really nice gems -- like a discussion of people who consider themselves lucky or unlucky:

Wiseman speculated that what we call luck is actually a pattern of behaviors that coincide with a style of understanding and interacting with the events and people you encounter throughout life. Unlucky people are narrowly focused, he observed. They crave security and tend to be more anxious, and instead of wading into the sea of random chance open to what may come, they remain fixated on controlling the situation, on seeking a specific goal. As a result, they miss out on the thousands of opportunities that may float by. Lucky people tend to constantly change routines and seek out new experiences. Wiseman saw that the people who considered themselves lucky, and who then did actually demonstrate luck was on their side over the course of a decade, tended to place themselves into situations where anything could happen more often and thus exposed themselves to more random chance than did unlucky people. The lucky try more things, and fail more often, but when they fail they shrug it off and try something else. Occasionally, things work out.

[TaoPhoenix]

Hmm. To me this topic doesn't seem to be so hard in a certain aspect. I don't read any of the "stories of great people". I read more of "reports of the times" books. I tend to read more "historical survey" articles on the web. That has the benefit of an analytical writer attempting to figure out why someone won and someone else lost.

[40hz]

You often see this notion enshrined in the misapplication of the "modelling success" strategy. And also in a trend that is now becoming popular with politicians where an argument gets made that "We need (as a people) to get beyond fixing blame and revisiting what we did wrong, and instead focus our attention and build on what we did right."

After a catastrophic state-wide power outage following tropical storm Irene in 2011, the governor of my state - with much reluctance - finally consented to convene a commission to look into why it took anywhere from 4 to 7 days to restore electricity following the storm. This happened only when there was sufficient public outcry over the fact that the two local power companies had drastically reduced their road crew staffing by a factor of something like 50% in the years leading up to  the storm. These companies had been allowed (with the blessing of the PUC) to go over to a "cooperative"or "shared" multi-state model that would theoretically allow them to provide the same level of response with less manpower on a "share the road crews" arrangement. It was an idea that might have worked had the storm not hit (as hurricanes and tropical storms generally do) a multi-state area.

The public outcry eventually became loud enough (and the political downside obvious enough) that the governor was unable to return any favors to the utility lobby. So he, with much fanfare, ordered a "full investigation."

When the governor finally (after several delays) stepped forward to discuss what the investigation had found and recommended, he prefixed his speech with a comment that the purpose of this meeting was "not to fix blame" but rather to focus on "what was done right."



A few years later, when we got hit with Hurricane Sandy, it was almost as bad.

Is anybody really surprised?

[superboyac]

no it is not surprising.  And the effects of these kinds of decisions/attitudes are causing a lot of suffering to a lot of people that may not even think are being affected.

[Stoic Joker]

When hurricane Frances/Fay (I forget) hit here we were out of power for 7 days. Many people were complaining about the power being out that long. I however having wandered around a bit to get an idea of the true scope of the damage...was actually impressed that they got the power on as fast as they did. But then again I helped out the situation in my own (it's probably a misdemeanor) way...and consider myself to be lucky