Empowering Real-Time Big Data for Industry 4.0
Big Data is becoming real-time. While Big-Data can be compared to huge Barrels containing Water, real-time Big Data comes along as a waterfall. But not a drop may be lost. Lost real-time data is lost forever.
The technology and architectures for Big Data and real-time Big Data therefore differ significantly from each other. While Big Data is first gathered and then analyzed the value of real-time Big Data is lost if not distributed thus directed to real-time analytics and event processing engines or to services handling Dashboards etc. in digestible portions immediately.
The Messaging part, is one of the biggest challenges for Industry 4.0 I believe. And from my point of view heavily underestimated.
Not only the Internet is not prepared to connect billions of new devices, but also the demand for managing billions of data-streams in real-time cannot be handled using messaging middleware running on “all-purpose” computers. Like the IP-routing logic, formerly running on legacy hardware using standard operating systems, was burned into HW and now is part of any router chip, connecting and managing billions of e.g. MQTT streams must be burned into hardware and taken from there on subscription base to a cloud of concurring micro services.
And because the required technology bits and pieces already exist (e.g. Solace Systems HW connects 100000 MQTT streams to a box; Docker can spin-up thousands of micro services using a single command without spinning up additional operating systems), it’s our passion to take on new architectures using Solace Systems HW tied to purpose built micro services to deliver cost effective easy to maintain solutions with a small foot print. System architectures and future solutions will be based on our 14 year experience building real-time financial market data solutions together with innovative customers and early adopters.
We have now started to explore the possibilities and challenges of Industry 4.0, here specifically the Messaging layer. The Messaging layer for Industry 4.0 will have to be defined clearly in every project, similar to projects including mass real-time data from exchanges, with the significant difference that there are not hundreds of exchanges that produce data but instead many million devices attached to thousands of micro services.
If you would like to get in touch regarding this discussion pls. contact me over LinkedIn or [email protected].
This article was first published on LinkedIn Pulse. Re-published with kind permission of the author.
About the author:
Mauricio González Evans is the owner of BCC Group International, a Frankfurt based Software House specialized in the development and delivery of high-quality software solutions for the financial industry. For nearly two decades we have been supporting customers in the area of market data technology, market data processes and Spreadsheet Integration technology
Please Note:
You may use one of these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>