The following is written for circulation in the “data science” research communities, on some advances in scientific methods of system recognition I’d like to share. It starts with mention of the very nice 9 year old work published by Google on “Detecting Influenza Epidemics using search engine query data” taken from a letter to that paper’s authors. Take the reference to be to your own work, though, as it involves system recognition either in life or exposed by streams of incoming data.
I expect a lot of new work has followed your seminal paper on detecting epidemics as natural systems.
But are there people starting to focus on more general “system recognition”,
studying “shapes of data” that expose “design patterns” for the systems producing it?
Any individual “epidemic” is a bit like a fire running it’s course, and sometimes innovating the way it spreads. That change in focus directs attention to how epidemics operate as emergent growth systems, with sometimes shifting designs that may be important and discoverable, if you ask the right questions. You sometimes hear doctors talking about them that way. In most fields there may be no one thinking like doctors, even though in a changing world it really would apply to any kind of naturally changing system.
Turning the focus to the systems helps one discover transformations taking place, exposed in data of all sorts. One technique allows data curves to be made differentiable, without distortion. That lets you display evidence of underlying systems perhaps entering periods of convergence, divergence or oscillation, for example, prompting questions about what evidence would confirm it or hint at how and why.
Focusing on “the system” uses “data” as a “proxy” for the systems producing it, like using a differentiable “data equation” to closely examine a system’s natural behavior. In the past we would have substituted a statistic or an equation instead. By prompting better questions that way it makes data more meaningful, whether you find answers right away or not. I think over the years I’ve made quite a lot of progress, with new methods and recognized data signatures for recurrent patterns, and would like to find how to share it with IT, and collaborate on some research.
Where it came from is very briefly summarized with a few links below. Another quick overview is in 16 recent Tweets that got a lot of attention this past weekend, collected as an overview of concepts for reading living systems with bigdata.
I hope to find research groups I can contribute to. If you’re interested you might look at my consulting resume too. If you have questions and want to talk by phone or Skype please just email a suggested time.
Thanks for listening! – Jessie Henshaw
___________________________________________
fyi – 350 words
To start from the beginning, in the 70’s I made some fundamental discoveries about the physics of individual natural systems (1), recognizing a design pattern “hidden in the tails” of data that are normally discarded… In this case I’m referring to the *actual tails* of the data, not statistical ones, all those curious periods of “regular non-linear change” you find almost anywhere things are beginning or ending. I found it often offered verifiable suggestions regarding patterns of individually developing natural system designs.
Once you identify that pattern of regular non-linear tails as a signature of beginning or ending… when seeing it you then look for what system is either “picking up” or “trailing off”, and often find it. There is useful technique one can use, but just asking the question often results in finding great leads by itself, and gives hints to what other viewpoints one could learn more from. In that way it’s just a new variation on familiar forensic techniques, a “pattern search” method, like: see a footprint => look for the person.
When this work was first done it seemed so odd to other scientists it was completely dismissed. That also continues to be the case in most fields, that scientists often can only discuss theory and statistics. What’s new today is that business people really need their mountains of data to become more meaningful, which makes a lot of technically very capable people available, familiar with some of Alexander’s pattern language, and likely to be less stuck on abstract theories, …as a possibly better audience.
In the 90’s I developed a physics theorem for it, confirming the shape of data tails I’d identified intuitively before, a sign of progressive organizational change indicating the beginning or ending of individual natural system development(2). To demonstrate refined data analysis methods I did various system recognition studies using historical data sets (3) using LISP algorithms I developed (4). A paper on it was published and included in a pattern recognition book(5) but didn’t generate any interest. The software platform used also changed faster than I could update my routines, so all that work stopped. In the 00’s I shifted to more empirical studies, focusing on sustainability accounting problems. patterns of energy use and cultural organization and transformational change. The methods led to my other important research paper, on energy accounting for businesses as whole systems(6), all of which led to some years of related work at the UN.
For the UN I made proposals on the design of world business accounting systems, using bigdata for system recognition and a model for inclusive sustainability accounting to create a true guidance system for the UN’s SDG’s(7). To make a better record of the work I turned to presenting it as a general pattern language for working with natural systems, producing two good papers each seemingly quite well received at the 2015 meetings of PURPLSOC and PLoP (8).
For a different kind of quick overview of concepts my collection of recent Tweets on BigData, that got a lot of response recently, as 16 Tweets on Reading #BigData for Life(9),
- http://debategraph.org/Details.aspx?nid=360233
- http://www.synapse9.com/drtheo.pdf
- http://www.synapse9.com/drwork.htm
- http://www.synapse9.com/drstats.htm
- http://www.synapse9.com/pub/1999_FeaturesOfDerivativeCont.pdf
- http://www.mdpi.com/2071-1050/3/10/1908/
- http://www.synapse9.com/jlhpub.htm#un
- http://www.synapse9.com/jlhpub.htm#pl
- https://synapse9.com/signals/2016/03/20/16-tweets-on-reading-bigdata-for-life/
_________
jlh