Subscribe to DSC Newsletter

Analytics and Belief: The Struggle for Truth

For full post:

Increasingly sophisticated analytics tools and methods are available to derive business insight from data.  However, as a discipline which drives insight from data, the crucial ‘last step’ in the analytics process is about organizational decision making.  A sophisticated, intensive analysis may all be for naught if the crucial last step, framing and committing to a decision, misses the mark.

No matter how fancy or sophisticated the tools and methods, there needs to be a commitment to action in the end.  This is complicated if there is doubt concerning the validity of analytics-derived insight.  The organization has to embrace analytics culture: the notion that scientific, evidence-based techniques are the best method to discover and validate hypotheses.  While intuition can be powerful, it should be validated via data-driven experimental approaches, particularly in dynamic, complex environments.proof

Adopting analytics culture brings an organization a step further to connecting analytics insight to high-quality decisions.  For the union to be feasible, analytics culture must understand how ‘belief’ concerning the insight gained from analytics is secured in the organization.

It is helpful to consider that organizational belief, the belief in the results of insights derived from analytics, can be positioned on a continuum:

  1. Truth-in-of-itself:  establishing belief via incontrovertible evidence, such as via structured and replicable experimental proof in a laboratory.  Also, the validation of a mathematical proof which applies the rules of logic and which clarifies the boundary conditions for the proof (in the case of purely rational or epistemological assertions).
  2. Useful to be true:  establishing belief based on the notion that it is useful for an assertion to be true.  Often we adopt this framework of belief when it is difficult to establish incontrovertible proof, but there is observable practical concordance.  Indeed many practical theories in the ‘useful to be true’ category lead to engineering advances without necessarily being fundamentally ‘proven’ scientifically.  This category often encompasses the highest state of ‘belief’ achievable in business, as it is impractical, and often impossible, to scientifically validate many complex business assertions.  Usefulness implies pursuing ‘pragmatic’ truths – to adopt an engineering mindset which diffuses politics and makes decision making an objective, professional process.  This state reflects the observation of statistician George Box that “all models are wrong, but some are useful”.
  3. Possibly true:  establishing belief based on the notion that it ‘might be true’.  This method appeals to the instincts and sentiments.  As such, it risks being driven by a willful notion of ‘wanting something to be true’. This results in a susceptibility to being misled by cognitive bias traps - for instance, being waylaid by Kahneman’s ‘System 1′, our ‘lizard brain’ which leads us to make quick decisions.
  4. It will continue to be true:  this system of belief is based on the notion that things will be true based on a continuity of the status quo.  This is known as thestatus quo cognitive bias trap.
  5. It should be true:  This is belief based on implicit and willful trust in one’s own viewpoints and bounded information.  Intuition-driven conclusions may fall into this category.  Although intuition may turn out to be correct in the case of expert intuition, it can also mislead us (even when we are ‘experts’).  Kahneman and Klein’s article ‘Conditions for Intuitive Expertise‘ is an excellent guide to distinguishing where intuition is useful and where it can mislead.  When believing in ourselves implicitly, we are susceptible to the overconfidence bias, whereby we attribute greater confidence in our own judgment than is objectively warranted.
  6. It is the current trend:  slipping down a notch on the scale of ‘truth belief’, this is belief based on the notion of a loose tendency or trend.  This is susceptible to many biases, including ambiguityanchoringattentionavailability, and selective perception.
  7. It is an aspect of the truth:  this form of belief is based on believing something to be true because the people in the surrounding community believe in something.  This is susceptible to groupthink and the bandwagon effect.

For full post:

Views: 780


You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

Comment by Chris Ngo on September 30, 2013 at 2:17pm

Your article is an adjective to the verb design! Thank you!

The simplest way to describe the design process is to divide it into two phases: analysis and synthesis. But those descriptions miss a crucial element—the connection between the two, the active move from one state to another, the transition or transformation that is at the heart of designing. How do designers move from analysis to synthesis? From problem to solution? For you, truth is the missing puzzle, for business designers, communication is key. Both are necessary as water to life!

Comment by Thomas Speidel on September 16, 2013 at 10:36am
Good post, Scott. "if you torture your data enough, it will confess to any sins" (Frank Harrell). I think we should be cautious to talk about truth in this context. To me I see truth as a synonym of causation, something remarkably hard to establish, let alone proove without sound experimental design. And in the context of business analytics, strong experimental design is often in practical. Thus, at best one can talk about strong vs weak evidence, more so than truth.
Comment by Scott Mongeau on September 16, 2013 at 8:01am

Nice one :-)  

On that note, perhaps also 'strange, but true'...

Comment by Illingworth Juan José on September 16, 2013 at 7:33am
Great Post. There is also "too good to be true"

On Data Science Central

© 2019 is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service