As the amount of machine-generated data scales, indexing it
via metadata will become critical.
Think about the most successful, widely scaled networks that
let us function in today's world. No I’m not talking about internet service
providers, I mean the Really, Really Big Networks. The ones that without which
modern civilization would be very different. The telephone system. Intermodal
containerized shipping. Air traffic control. And they all have one vitally
important enabling element that made them all scalable: a Control Layer that is
not intrinsic to the electronic or physical streams that make up the network
traffic. For phones, it’s Signalling System 7, which has managed to run the
world of voice calls for decades. For intermodal shipping, it’s container
manifests. For aviation, it’s ATC. And they truly do, run the globe.
What is a Control
Layer?
In the analog era, telephone calls were “switched” using
tones that were sent along the same wires that handled the calls. This was
known as in-band signalling. While usable with the technology of the day, they
were fraught with issues that included easy faking of the signals to get free
international calls (known as “phreaking”), inability to change the path that a
call could take once it was in progress, no direct association with billing or
tariffing, and other problems. When it came time to deregulate the
traditionally government-run phone companies, and thus lay the framework for
the plethora of corporations that provide the bulk of phone service today, the
business and technological elements of call-handling had to evolve. That led to
SS7. Among its key tenets are the moving of the control over calls, billing and
routing, into a control layer that is separate from the calls themselves. This
allows calls to be switched “on the fly” between carriers, inter-company
billing and many other features became enabled by this shift to out-of-band (as
its known) signaling.
Shipping containers
arrived with out-of-band signalling as well
In the early 1960’s with a growing world population, an
ever-increasing appetite for food and goods from international sources (largely
fostered during World War II as troops went abroad), and the rise of
computerized inventory and supply chain handling, the idea of having vessels,
trucks and train cars built for specific types of cargo wasn’t going to scale
with demand. So some enterprising folks came up with the standardized shipping
container that we know today. That moved the command and control over how and
where goods are routed, into a control layer based on paper and eventually
fully electronic manifests – instead of the ship captains. This led to
tremendous flexibility and cost-efficiencies. Without these, it would be hard
to envision Walmart and then Amazon rising at the rates they have.
So went the airlines
Another advent seeded during WWII was the idea of global jet
travel. The public had witnessed the start of globalization, and at the same
time, pressurized aircraft (initially, high-altitude bombers) powered by jet
engines become possible. With this radical increase in speed, range and
passenger carrying capacity, coupled with the shift in passenger air travel
from water-based to land-based terminals, demanded that the control of where
and when the aircraft moved, needed to get out of the cockpit and into a
‘network’ which became ATC. It’s not as efficient as it could be, but ATC has
been fundamentally safe and reliable, and scaled continually, for many decades
(noting that it does have to evolve once again, to accommodate the upcoming
plethora of unmanned aircraft that are indeed coming).
Now it’s time for the
IoT to have its
Control Layer
The quantity of information already being produced by the IoT, or less
abstractly, machine-generated data (MGD), is truly staggering and showing no
signs of lessening. In fact, if you take imagery, especially consumer-generated
video, out of the equation, MGD already exceeds what humans have produced since
the beginning of time. We are fast approaching the “needs to scale” tipping
point that telephony, global logistics and aviation experienced. There are
characteristics of MGD that make it a lot more akin to these “physical world”
networks than what we’ve been experiencing with the rise of say, social media.
For starters, this is the data that could get someone killed. Road sensor data
might cause all of the lights in a city to turn red. Or open the floodgates of
a dam. Or switch a train. Plus there can be potentially diverse uses for the
same data. In addition to changing the traffic signals, the same stream of MGD
might be utilized in actuarial calculations to price insurance premiums, or provide
the edge to a quantitative analyst in a hedge fund.
With IoT data,
context matters
Given the seriousness of MGD/IoT data use-cases like those
above, it’s critical that the user (be it a person, company, government agency
or a fleet of robots) be provided with as much contextual information as
possible. This can include provenance ala identifying the source in specifics
such as make and type of sensors, literal context i.e. where the sensors are
situated (e.g. inside a moving vehicle, attached to power poles, etc.) and
whether the source-devices are maintained and calibrated. All of this bounding
information is found, and ideally conveyed, outside of the data streams
themselves. Tracking provenance and keeping the control layer separate also
have key security advantages. Yet most IoT implementations today don’t provide
such a control layer.
Metadata is the
control layer for the IoT
Standards efforts are underway to establish the equivalent
of SS7/Container Manifests/ATC for the IoT. Unlike those successful schema,
achieving this for MGD encounters many more variables than almost anything
dealt with before. Early examples of out-of-band signalling for MGD are within
vertical industry sectors. These need to give way to a universal format for
describing sensors, data provenance, regulatory and compliance issues, and of
course, monetization, in a control layer. In software terms this is defined and
carried in metadata. It’s time to start
contemplating how all of this will impact you and your organization. Rest
assured, it’s going to touch every industry, within and between. Watch This
Space.
Source: Networkworld
No comments:
Post a Comment