To Be an Agilist You Need to be Informed

By Brian Lucas

This one is for Mari a truly wonderful person who I am privileged to have met.

“The only sure bulwark of continuing liberty is a government strong enough to protect the interests of the people, and a people strong enough and well enough informed to maintain its sovereign control over the government.” – Franklin D. Roosevelt

Being agile requires information.  Today both business and government are collecting massive amounts of data on many of our activities.  Predicting human behavior by tracking data, sometimes vast amounts of data has been the subject of much science fiction, but today the science fiction is a reality.  This is such a hot topic there are already a plethora of terms and sub categories each with their own set of multi-billion dollar industries attached.  I am going to spare you the heady exploration of quantum computers and indeterminate state analysis.  Instead let’s look at technology that is already in wide use.

Take Predictive Analytics as an example.  Predictive analytics comprises the following disciplines of artificial intelligence, data mining, machine learning, modeling and statistics to analyze patterns in historical facts and current activity in order to make predictions about future events[1].  Predictive analytics is heavily used in actuarial science, financial services, healthcare, insurance, marketing, pharmaceuticals, the retail industry, telecommunications, and travel amongst others.

Deep data mining has risen to prominence lately[2].  It has been touted as the future by many from Dean Abbott, President of Abbott Analytics, Inc. to Eric Siegel, PhD, founder of Predictive Analytics World.  Deep data mining takes predictive analytics one step further into the realm of influence.  It refers to the strategy of data collection involving the sophisticated analysis of patterns across a broad spectrum of data domains.  It is used to not only predict, but to sway behavior.  This technology is being used by businesses and governments alike to predict the psychological profiles of individuals.  Look at the way Pandora analyzes an internet user’s personality by tracking their favorite music and movies.  If this sounds like Big Brother know that it has definitely raised concerns from privacy advocates.

Big data is the term for a collection of contrived data sets whose resulting size and complexity often exceed the capacity of traditional data processing applications.  The trend to gather greater meaning from ever larger data sets is an irresistible and irreversible one.  The amount of inference derivable from the analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allows incredibly complex correlations to be made.  These perform various functions from the innocuous spotting of business trends to prevent diseases and combating crime.

Two of the technological revolutions that made this possible were the explosion in hardware data storage and software data manipulation.  As of 2012, it was feasible to process in a reasonable amount of time exabytes of data.  An exabyte is one quintillion bytes or 1,000,000,000,000,000,000B or 1018 bytes.  The world’s technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; every day 2.5 exabytes of data are created[3].

People often confuse Business Intelligence and technologies like Big Data.  Business Intelligence uses intrinsically descriptive statistics on data that has high information density in order to measure things and detect trends.  Big data on the other hand uses inductive statistics and nonlinear system identification concepts to infer regressions, nonlinear relationships, and causal effects from massive data sets of relatively low information density in order to reveal dependencies, relationships, and predict outcomes.

The bottom line is that everyone in business and government is looking for patterns.  Looking for patterns is nothing new; it is as old as reasoning itself.  It is at the heart of most of our technological advances throughout history.  The difference is that now when have massively complex, large and sophisticated machines to aid us in this effort.  This is indeed a two edged sword.

Since most of our activities are done online, there is so much data collected about our activities that personal privacy becomes doubtful.  There are both wild and credible stories of scientists and mathematicians creating predictive algorithms of which the intelligence community immediately takes possession[4].  How will we ever know when some entity crosses the line?  In fact there probably is no single defining line.  It is more of a grey band stretching from moral into immoral use of personal data.  This represents a challenge for everyone, but we in the Information Technology industry bear a greater responsibility.

Banning the practice of collecting data and making predictions from it is not the answer.  There is far too much good that can come from this activity.  And after all we need this data in order to be truly agile and make intelligent decisions.  What can we do?  The answer is to get involved and learn more.  The more you learn the more light you can shed on this vital subject with others.  The more informed your votes and your actions will be.  The totalitarianism of a Big Brother state can only arrive when the populace is ignorant and uncaring.  So get involved and get informed and remember till next time – keep agile!


[1] See Siegel, Eric (2013). Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die

[3] See “IBM What is big data? — Bringing big data to the enterprise”

About Brian Lucas

In his life, Brian Lucas has been a coach, farm worker, forester, health care advocate, life guard, general contractor, mechanic, mixologist, musician/singer (in a rock group), salesman and teacher. Brian has worked as a project manager, technical marketer, methodologist, manager, software architect, systems designer, data modeler, business analyst, systems programmer, software developer and creative writer. These efforts include over a hundred hi-tech initiatives in almost every business and industrial sector as well as government and military projects. Among them, he designed and developed a quality assurance system for the first transatlantic fiber optic communications network, a manufacturing system for a large computer manufacture’s seven manufacturing centers, a data mining system for steel production, an instrumentation system for cable systems, defined requirements for government’s information systems and designed and developed human performance management systems. Brian has educated and mentored many over the years, designing programs to discover and develop talent. He has also lectured extensively to a variety of audiences. Brian is currently devoting as much time as possible to the innovation of business agility and human capital management along with the next generation of agile software development. As an amateur theoretical physicist he is working on joining general relativity and quantum mechanics through a multidimensional time corollary on string theory and negating the uncertainty principle with Louis de Broglie’s wave/particle hypothesis. He is also an avid blue-water sailor and wilderness backpacker. He enjoys billiards, boxing, chess, cooking, famous battle reenactments and war gaming, fencing, flying, gardening, horseback riding, martial arts (particularly Ninjutsu), philosophy and psychology, playing musical instruments (7 so far), poker, rapid-fire target shooting, reading (he tries to read a new book every night), painting with oils, scuba diving, skiing and recently writing novels.
This entry was posted in Agile and Strategic Planning, Agile Arguments, Agile for Beginners, Agile in the Enterprise, Agile Thinking and tagged , , , , , , , , , . Bookmark the permalink.

9 Responses to To Be an Agilist You Need to be Informed

  1. Sally says:

    Thanks for clearing this up Brian. I have found the terms confusing. Do you have any data you can share on the use of these technologies to guide reorganization efforts? Particularly the more fluid agile structures you wrote about previously.

  2. lynettewills says:

    Brian,
    Again, thank you for an informative and thought-provoking post. These issues are important as we see what’s happening with the digital footprints we leave behind; and how they affect our future.

  3. Jack says:

    Hi Brian: Thanks for putting this in perspective! There seems to be so much overlapping technology here that it gets confusing thanks for making it clear.

  4. Lou says:

    Brian, I am ashamed to admit I have been lurking here since I commented on your agile organization post. I am glad you posted this since we have been barraged by vendors with conflicting claims that I believe are overselling and over promising. Are you up for an email on this subject with a few questions?.

  5. Cathy says:

    Nice post Brian! I struggle with understanding all the new technology we keep getting hit with and trying to understand its impact. Any tips you care to share on how you do it?

  6. Parry says:

    Wish I was an informed as you are! Keep Posting Brian!

  7. Devon McMasters says:

    Knowledge is the key to any success today! Peter Drucker wrote about knowledge workers, but it has gone beyond that. Today all you have to do is look at a preschooler navigating a tablet and see them using skills that we in the previous generation learned on-the-job. Knowledge if technology and computers is an absolute must to working in any industry now. I realize this is a bit off topic. I do agree with this post and think it could have gone even farther.

  8. GideonM says:

    I am so on board with your thinking here Brian! Would love to have the opportunity of working with you some day!

  9. Corinne Walton says:

    Brian, how do you keep so informed? I read your blog and others all the time. I am overwhelmed… How do you keep up with all the latest without suffering from information overload? I am not in the computer industry as you are. I love how you think though which is why I follow your blog. You seem to be able to apply intelligent thinking to any subject. Do you have any tips for me? Would you mind if I emailed you? I’d like some career advice on an opportunity I have.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s