Maybe Let’s Call it “Big Small Data”

They talk about how much we are creating new data.  Mans Collective knowledge is double every 18 months or faster or so the rumor goes and now with Big Data filling up exponentially increasing the size of data centers needed to support it one of the details that I think is overlooked is that this is not earth-shattering data.  Its tiny little bits of say 93.245 degrees and a time stamp.

Think of it this way…

Ten percent of our combined knowledge is actual knowledge (maybe less but never mind that) and the other 90% is tiny pieces of measurement data.  93 degrees here, 37 rpms here and that sort of thing.  Billions and trillions of tiny little bits of data that is mostly measurements of sensors.

What is Big Data so much little bits of data?

Well, the trick is when you have all that data now you can use fancy algorithms that learn or look for anomalies.  In the sea of data we are creating, it’s the bits that don’t fit that is our opportunities.  Our passion for this as a civilization even created subsections or trends that we don’t’ always connect with ‘Big Data’ like IoT, AI, Machine Learning and the like.  IoT comes out of the desired to generate the data and AI and Machine Learning is the only way for our small minds to wrap our heads around it.  In a matter of a few years, we will have so much data so far beyond the ability for mankind to understand as to be beyond our ability to understand how much we don’t’ understand.

We will drown in that data?  Well, I supposed we could but more than likely we will swim and swim quickly and all the tools our civilization has created will help build a new civilization built on data being the foundation of it all.  Society built on data but my fear is that data could be like sand if something goes south.

In either case for good or for bad in a decade everything will depend on trillions of tiny bits small big data…