Domino's Rule in Edge Analytics
Data Volumes vs. Domino’s Rule
May 8, 2017
Scales weighing price against value in acquisition process
A Stochastic Multi-Objective Decision Analysis (SMODA) Model for Evaluating Defense Acquisition Procurement Criteria
May 15, 2017
IoT Data quandry

How much IoT data do I need to ingest?

Choking the Whale

IoT Data quandry
How much IoT Data Do I Need to Ingest?

Choking the Whale

Whales have the largest mouths of any animal.

The record holder is the Bowhead Whale. Their tongue alone weighs a ton.

A Bowhead is large enough to avoid some choking risks faced by smaller mammals. But, it is still surprisingly easy to choke. Sadly, all whales have at least two risks; both apply to IIoT.

The first risk is choking while feeding.

The Bowhead ingests vast gulps of water, filtering out everything except the tiny creatures it feeds on. The whale needs its filters and plumbing. But a foreign object like a net, or a fish can become lodged, stopping food processing. The whale starves to death.

This choking hazard is like an IIoT risk.  If a system ingests large data streams, to look for interesting tidbits, it’s easy for the system to bog down. That will starve the analytics. If data backhaul is effected by weather, choking risks go up at a bad time. This kind of choking happens at the worst moment.

There is another choking hazard.

It happens when a whale clears its blowhole. Sadly, fish and other objects can lodge in the hole. It can’t exhale the air it has trapped in its lungs while it was swimming to eat. And the whale suffocates.

This is like a second IIoT risk. Data streams can take a long time to process, and important results trigger additional delays. When this happens, the opportunity for improvement has passed long before the results are reported for action. The analytics system can’t “exhale” processed results.

IIoT can avoid both choking hazards with greater ease than whales.

Most IIoT applications don’t need all the data they ingest, and they need to report out even less than they take in.

It is important for IIoT to understand why there is no need to choke the system with masses of intake data, or output results. For Lone Star’s AnalyticsOS™ operating at the edge, this is a natural outcome of our preferred architecture.

IIoT is different than the Internet of People (IoP).

People are messy and randomly change behaviors and whims. Industrial Things operate with considerable uncertainty too, but they don’t engage in purely whimsical behaviors. Machine Learning for IoP applications must learn what are “normal” behaviors of messy mankind. IIoT systems operate in systems where “normal” is known.

The Big Data mantra, “I need all the data” may be true for IoP, but it rarely applies to IoT. And, we have yet to find a single case where it applies to IIoT.

A great example is the need to go back in time and understand an anomaly. Lone Star utilizes the “Swan Trap™” in our AnalyticsOS™ applications. The Swan Trap™ captures events which are outliers, and all the data surrounding the event. Thousands of hours of normal operation are not needed in complete detail.

In real applications with real industrial hardware we often see data stream reductions on the order of a factor of 1000. It’s not rare to see even greater reductions, while providing faster and richer analytics.

So, for IIoT analytics success we suggest three things:

  1. Don’t think like a whale, your choking risk is greater than the noble mammal, but you don’t have to live by taking in great gulps.
  2. Don’t apply the ideas from the Internet of People to the Industrial Internet of Things; your capital assets are not as random, or silly, as humans.
  3. Work with solutions proven to reduce data flows by a factor of 100, 1000, or more; AnalyticsOS.

About Lone Star

Lone Star provides powerful solutions that improve operations. We serve industrial markets, aerospace & defense, oil & gas, transportation & logistics, and the public sector. Our analytics products and technology-enabled services are proven; we deliver the right answer for your operational needs. We are committed to generating improved operational and financial performance through accurate and actionable answers to our client’s most critical business challenges. Our reputation is built on creating lasting value for our clients. Headquartered in Dallas, Texas, Lone Star is found on the web at

Recent Blog Posts