‘Big Data’ and the right to human understanding.

One of the fascinating scientific subjects I research is how human understanding comes from narrative. Without getting too technical, narratives about relationships, environments and culture change issues come from people “observing the flows” of the natural processes, the flows by which those changes in our world take place.  The basic starting point, then, is having some way to observe those flows.  No awareness of the flows, *no story*!

This is such an important thing for combating our alienation from the breakdown of traditional cultures, really all around the whole.   It’s quite an unfortunate side effect of the great eruption of wealth in modern times, and the ever more intense global competition fostered by the world economy doing it.   A small part of how it disturbs our ability to tell stories about what’s happening to us in yesterdays post What is a “rights” agenda, with ever increasing inequity?


Mining live stories from big data is way to build human understanding

Use maps of natural silos of conversation to find who you need to talk to and how!

I ran across five wonderful examples this week alone, of ways to bridge the enormous cultural and intellectual divides the keep us from arriving at a common understanding of what to do with the earth.     My topic yesterday saw how an economy structured to produce both ever increasing complexity, inequity results in the breakdown of traditional cultures and ways of knowing, a loss of stories for giving our lives meaning.   Learning to see the problems can also be used to find solutions too, of course, the main one here perhaps just learning to see what we’er doing to ourselves.   The thought process leads to seeing what strategies are failing us is not so different from that used for discovering promising new ones.

One identifies where the cultures that guide us lose track of what’s happening to them.   The other discovers exposes the flows of events in a way allowing us to create the new stories that will matter in our lives.   It’s how all human rights are achieved, by recognizing them as the clear story that beings order to a disruptively changing world, recognizing how nature connects the dots, letting us frame not just “good stories” but also “true stories” about finding a sound new path.

The practicalities of recognizing “what’s really happening” so we can use our values to fashion the stories telling us what to do will mostly not need a lot of big words and shiny promises.   You can do it with “big data”, even if today its main use seems to be for controlling personal data to make growing amounts of money from deny people their individuality.  You can also us it to mine the data world to pick up clear signs of whole new cultures emerging you’d otherwise never be aware of, for example.   Having ways of visualizing the eventfulness of change globally, on many dimensions, would be a very *different* kind of “news feed”, a true globally holistic “news feed”.

Every community could study the eventful flows of changing relationships, personal, cultural, economic, ecological, that matter to it, rather than just listen to media largely composed of chattering entertainers and politicians after money and power.   If a way of mining data for signs of events could show people what’s really happening to their world, and that became the the talk of the community, everyone could participate in shaping the news and the new stories about our human rights tell us to do.   It would give the media a real story to cover too.   The practical job to make that possible, though, is more like science than philosophy.     It’s to learn to recognize that eventful change comes from the emergence of new forms of organization, that generally begin with a viral burst of development, that energize whole systems, altering the balance and roles withing their environments, like organisms that growth from a seed to build new natural capital or flame out.

Shown as the general stages of growth for a new form of organization, from novelty to maturity


1. –  Changes in Word Use – I am not an expert in semantic analysis, fundamental changes in word use, particularly if following a clear developmental pattern generally do indicate a change in the world of people and their way of speaking about it.    Developmental changes in word usage expose important cultural experiences of the people writing the text.   I’ve used comparisons of the Google histories of word frequencies obtained from scanned libraries of books, their “Ngram” tool. I’ve also used the histories of word use in magazines, newspapers and even Google Scholar, such as to identify

Along with the various other “story mining” methods discussed in the introduction to my scientific method for mining the stories of natural change processes, and method of interpreting them:

Learning to read the eventfulness of our world  –  People who have some personal experience with the environments in which these explosive changes took place, as eruptions of new organization for those worlds,  these documented records of the shapes of their stages of growth provide rich reminders and new challenges to imaging what was really going on to produce the new environments the created.

Papers on “General Systems”(yellow), Papers and Citations for “General Systems Theory” (black & pink)

“Pop Corn” flurries of articles on sustainability as the subject emerged in the NY Times, and Accumulative trend


The 1990 beginning of the big eruption of mentions of Hip-Hop in the NY Times also coincided with the historic sharp decline in NYC crime rates, culture change as kids changed who they looked up to.

Use of “complex” followed  growing economic complexity up to ~1960, when we appear to have either lost interest or the ability to keep up, with the fast increasing complexity of our world.

Today one might also use Twitter and other social media, and also collect data on product and book sales and lots of other sources.   Of course, the sources would vary considerably from country to country, but the method would be the same.   What’s important is for the text or numeric data being scanned for “natural coupling” be “neutral” and not influenced by the subject being explored.

What might be possible, putting it all together, is to identify natural cells of social relationships and their interests, cultural “silos” of relationships identified by their ways of using language, in real time. There are security questions whenever new kinds of information are made available, so such maps should be abstract. The most valuable feature of such a “map” of connections, though, is the ability to then see who’s NOT connecting, the isolated constituencies.

You’d see what conversations are intense in one group and missing from another, say between Twitter and the local newspaper as one possible divide., defining two communities with differing values and interests. That would be a great tool for understanding a society, and a great tool for social activist groups, letting them see how to stop “preaching to the choir”, for one example. It wold also give them insight into the words and interests of the groups they need to connect with, but hadn’t known how. Seen that way it’s a “partnership tool”, allowing people to see through the silo walls just enough to make some connections.

More examples, links, applications & … stories

2.Ngrams,  3.Doctors Need Stories,
4.Broken traditions  5. Scientists need to find the meaning of their equations


2. Ngrams – One powerful example from Google’s Ngram tool for word use frequencies is a comparison of the usages of “complex” and “information overload” with the history of US GDP. The growth of GDP is also a measure of the complexity of the lives and tasks of the people living with it. The point in time, as the US economy continued to get bigger and change faster, when use of the word “complex” abruptly began to not keep up, was coincident with the explosion of use of “information overload”. I think that is profoundly revealing for the current US experience, of people dwelling on entertainment and losing interest in the world as it gets more and more complex.

To reverse that effect and make the complex world interesting again, points to another of the basic aims of “big data” for public education, creating forms of information to reconnect people with what’s happening around them.

1) https://books.google.com/ngrams/graph?content=complex&year_start=1800&year_end=2000
2) https://books.google.com/ngrams/graph?content=information+overload&year_start=1800&year_end=2000
3) http://synapse9.com/issues/US-GDPfullhist2013.jpg

This is also discussed in the draft introduction to the scientific method behind it

4) http://www.synapse9.com/drafts/InterpretingBigData-draft.pdf


3. Doctors Need Stories – There was a great commentary in the Sunday Times Week in Review this past Sunday: By PETER D. KRAMER In an era of systematic clinical research, medicine still requires the vignette.   http://opinionator.blogs.nytimes.com/2014/10/18/why-doctors-need-stories/?ref=opinion

Why Doctors Need Stories

…Here is where I want to venture a radical statement about the worth of anecdote.  Beyond its roles as illustration, affirmation, hypothesis-builder and low-level guidance for practice, story telling can act as a modest counterbalance to a sratened understanding of the evidence. [JLH emphasis]   

Not only doctors need stories, it’s the one way people understand their places in time, today especially as a time of great need to find new places for ourselves in this whirling world.  For that people first need the material to tell the stories, the traces of how their cultural ecology is changing as accelerating growth and change alter our lives ever faster, for now at least.


4. Broken traditions: A remarkable segment of the Krista Tippett radio program On Being,  about what happens when people lose their stories…    The work of Scott Atran is on understanding the spiritual foundations of Terrorism.   The program segment is wonderful: http://www.onbeing.org/blog/twitterscript-terrorism-expert-scott-atran/2694

My take: Scott found that the way terrorists are created is none of the ways popular in the press, but by “good friends” searching for a story, living in communities with broken traditions. This is as profound as it gets!

The insight that globalization had a globally disruptive effect on the regional cultures, that effectively lost the world economic competition, is truly profound. That we need to deeply care about how kids from those cultures respond to that is *such* an important conclusion to draw.

Americans are very proud of the great advance of “technology” we’ve focused much of our life energy on… but it has gone along with sweeping away the foundations of “how to live” for a *very* large part of humanity too. When we respond to the disarray caused with handouts, it really has little or no meaning is the problem.
What can we do? We do need to honor the ancient ways of being we are erasing. Our national purpose is now effectively to do so at the fastest growth rate we are capable of sustaining. That hidden objective of the world’s dominant economic culture needs attention, our way of taking over the resources of other cultures isn’t helping them at all.

The combined creative energy of our great global productivity machine, led by the money centers of NYC, London, Paris, Hong Kong, Sidney, and elsewhere, are “oddly driven”, following a mechanistic computerized business model. It’s a business model for maximizing “the bottom line”, used as means of concentrating wealth ever faster in the hands of the wealthy, working putting money into things to take more out, to multiply forever. That system and its end products are “impractical”, just as much as the lives of people left behind without their native regional cultures have become impractical. For either interest we also clearly don’t have much time, time we need to breathe a little breath of fresh air into the dominant culture’s core purposes.

We need to follow Scott’s lead, of paying close attention to HOW things emerge from the grassroots. Nature is chock full of great examples, like the businesses that join into ecosystems of community cultures or the separate cultures that all contribute to a complex society, with higher purposes than just “taking over”. But people tend not to realize, that economic growth really *must* have a true purpose or the society will die. That’s the deeper question, why we don’t notice.

I found a wonderful new opening into that seemingly impossible quandary, in the physics of continuity, why nature works by flows of events rather than by leaps, displaying a true need to have a process for achieving any change. Unlike the rules for numbers we are mistakenly trying to rule our world by, nature’s ways of building things needs to be responsive, and finds its purposes that way. You might look around my journal for other possible starting points for a discussion.

5.  Scientists also need stories to find the meaning of their equations – One of the dilemmas of modern science is the ability to generate the equations that theory implies, but not be able to understand their meanings.  The equations of complexity are actually too complicated and inherently unpredictable, so you can’t even tell if a computer simulation even applies to a real environment….   This was discussed, in wonderful story telling form, on NPR’s Radio Lab last week on their Podcast on “LIMITS – of the Mind, the Body, of Science“.     http://www.radiolab.org/story/91709-limits/

This limit of science, of understanding what the equations mean, has been a focus of my work for the last 35 years, since  I recognized it as a new problem of modern science, and as a need to find a way to understand the systems of our world directly from their behaviors.   Our theories for complex naturally occurring systems are all both a) clearly not adequately describing what we actually observe, and b) too complicated  and unpredictable to understand and c) even a computer test of a complex system equation lacks the important ability to behave as if in the environments we’d like to emulate, and so simply lacks their actual meanings for our natural contexts.

So I’ve been studying that inverse problem for quite some time, that of “meaning recognition” for complex systems, apparently without much of anyone realizing what I was doing.

The breakthrough was realizing that the only way humans have had for understanding the complex systems of the natural world has always been directly from their behavior.    So I just needed to discover and test options, to find out what features of natural phenomena we used to give us our stories.    If people are aware of the forms of equations that natural systems might be said to follow, it’s clearly very deep in our subconscious and never really known to us, except in very special cases. People with no understanding of physics theory at all can be highly adept at recognizing the meaning of complex natural systems and their changing relationships and environments of all kinds, if they’ve had an emersion experience of them.

So.. that presented an interesting “big challenge”, to find scientific ways to “tell their stories”.   The solution was to

employ our own very normal ways of recognizing meaning

directly from the behaviors of the natural systems we work with all the time.   So I developed a pattern language for natural behavior, based on both physics and the elemental pattern of change that any alert observer notices as “eventfulness” for the behavior of systems and relationships.  That is their destabilization and stabilization, development and decline, and based the framework on change over time, combined with the associations between familiar terms of common language for recognizable phenomena, like “burst” referring to something physiologically bursting (sounding like the phenomenon too).



Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.