soon     Thu, Mar 11, 2010  Permanent link
This common misconception among data analyses reflects the very essence of human reductionism. Instead of allowing patterns to become intrinsically clear, perpetually dissatisfied with our understanding we tend to impose illusory structure on the stimuli we receive out of fear and laziness.
Wildcat     Thu, Mar 11, 2010  Permanent link
important post, thank you Chris.
I had in one of my earlier posts on the subject defined a kind of futurism which I have called :Non-immediately distinguishable futures (NIDF): I do believe that what you refer to here as studying outliers and observing the flow of events into the next black swan is exactly the job description of a NIDF style futurist.

as an aside I am not sure that I agree with the statement :"Linear projections help us continue to get things done based on fairly reliable expectations". that it may appear so on very short sequences of time stamps I agree,what I am concerned about is the recurring pattern that these kind of 'fairly reliable' predictions instill in us. the very pattern that has led to the economic crisis is such an example. it is highly probable that in the very near future some of these predictions will need be relegated to smart (and maybe intelligent) machines, whilst as the sequences enlarge in scope and time we will need a new kind of futurism, one that does not use past predictions but uses a kind of cyberintuition, a non linear and highly chaotic, hybrid of perception and machine co-existing in a far from equilibrium situation. (we may of course need a better mind machine interface first)
chris arkenberg     Thu, Mar 11, 2010  Permanent link
Thanks for your comments. To both I suggest that the tendency to simplify our expectations of systems behavior, while a deep part of the human operating system, has been a valuable mechanism. Like Descartes description of the Pineal Gland as the "reducing valve of the Soul" it's important for us humans to be able to filter out the maelstrom of data coming our way and focus on the stuff that's immediately important. Facing massively non-determinist systems such as global climate makes it difficult to act in any meaningful way, yet we get that it's not good to dump oil on our lawn or garden.

Having said all that, we're steadily evolving past the helpful reductionism to embrace complexity and living systems in order to better understand the workings of our world, so prone are we to being trundled about by their sudden turbulence.