These guides demonstrate the operational flexibility and speed of the Hazelcast In-Memory Computing Platform. Set-up in seconds, data in microseconds. Operations and developer friendly.
Find out for yourself how to get a Hazelcast IMDG cluster up and running. In this Getting Started guide you’ll learn how to:
Learn how to run a distributed data stream processing pipeline in Java. In this Getting Started guide you’ll learn how to:
As a next step, you will be able explore the code samples and extend your setup more features and connectors.
When recency and speed drive the value of your data, in-memory stream processing solutions from Hazelcast can elevate your business to new levels of performance.
Data is coming at you fast from every direction. To meet customer expectations, prevent fraud, and ensure smooth operations, batch processing simply won’t cut it.
Traditional batch processing requires data sets to be completely available and stored in a database or file before processing can begin. With in-memory stream processing platforms, you can respond to data on-the-fly, prior to its storage, enabling ultra-fast applications that process new data at the speed with which it is generated.
Your business is a series of continually occurring events. You launch products, run campaigns, send emails, roll out new apps, interact with customers via your website, mobile applications, and payment processing systems, and close deals, for example – and the work goes on and on. Events happen in real time, and your environment is always changing. To compete, you need to be able to quickly adjust to those changes.
Your applications require the real-time capabilities and insights that only stream processing enables. You can’t rely on knowing what happened with the business yesterday or last month. You need to know, and respond to, what is happening now. Event-driven businesses depend on modern in-memory streaming applications for:
Stream processing must be both fast and scalable to handle billions of records every second. Event streams are potentially unbounded and infinite sequences of records that represent events or changes in real-time.
Typical stream processing tasks include:
The Hazelcast Jet stream processing platform–built on in-memory computing technology to leverage the speed of random access memory compared with disk–sits between event sources such applications and sensors, and destinations such as an alerting system, database or data warehouse, whether in the cloud or on-premises.
Developers build stream processing capabilities into applications with Hazelcast Jet to capture and process data within microseconds to identify anomalies, respond to events, or publish the events to a data repository for longer-term storage and historical analyses.
With the Hazelcast Jet stream processing platform, your applications can handle low latency, high throughput transactional processing at scale, while supporting streaming analytics at scale.
You can analyze streaming events in real-time, augment events with additional data before loading the data into a system of record, or power real-time monitoring and alerts.
Stream processing does not always eliminate the need for batch processing. Traditional batch processing may be necessary to provide a comprehensive view of historical data – think of BI reports, which may access data from a system of record that is much older than the data that lives in your stream processing platform. And batch processing enables organizations to leverage existing investments for use cases where the urgency of reacting to data is less important. In some architectures, the stream processing platform and batch processing system may sit side-by-side, or stream processing may occur prior to batch processing.
Hazelcast Jet provides the tooling necessary to build streaming data applications. It gives you a powerful processing framework to query the data stream and elastic in-memory storage to store the results of the computation.
Hazelcast Jet processing tasks, called jobs, are distributed across the Jet cluster to parallelize the computation. Jet is able to scale out to process large data volumes.
Hazelcast Jet has very high-speed integration with Hazelcast IMDG, which can store large amounts of data to join it to the Jet stream in microseconds. Latency can also be reduced by using IMDG for stream ingestion or publishing results.
Hazelcast Jet works with streaming data in terms of “windows,” where a window represents a slice of the data stream, usually constrained for a period of time.
Jet supports Tumbling, Sliding and Sessions Windows.
Hazelcast Jet supports the notion of “event time,” where events can have their own timestamp and arrive out of order. This is achieved by inserting watermarks into the stream of events that drive the passage of time forward.
Hazelcast Jet provides simple fault-tolerant streaming computation with snapshots saved in distributed in-memory storage. Jobs restart automatically using the snapshots, and processing resumes where it left off.
The need to trade-off performance and correctness in event processing systems may not allow firm guarantees. Hazelcast Jet allows you to choose a processing guarantee at start time, choosing between no guarantee, at-least-once, or exactly-once.
10s of 1000s
Transactions per second
High-Speed streaming data from multiple sources, devices, and networks
1.2 microseconds
Maximum application latency
Leverage high-speed stream processing with in-memory performance
A high-speed solution for a high-speed world
In-memory streaming is designed for today’s digital ecosystem, with billions of entry points streaming data continuously, with no noticeable delays in service. Ever.
The speed of in-memory, optimized for streaming data
The speed delivered by in-memory systems can accelerate data performance by a factor of 1000X. Adding stream processing accelerates this further, through pre-processing of data prior to ingestion.
Applicable to any process that would benefit from higher performance
Is there a single application in your business that would work better at a slower rate? Stream processing is not just faster, it’s significantly faster, which opens up new opportunities for innovation.
This white paper walks through the business level variables that are driving how organizations can adapt and thrive in a world dominated by streaming data, covering not only the IT implications but operational use cases as well.
This paper is intended for software architects and developers who are planning or building system utilizing stream processing, fast batch processing, data processing microservices or distributed java.util.stream.While quite simple and robust, the batching approach clearly introduces a large latency between gathering the data and being ready to act upon it. The goal of stream processing is to overcome this latency. It processes the live, raw data immediately as it arrives and meets the challenges of incremental processing, scalability and fault tolerance.
Hazelcast Jet is the leading in-memory computing solution for managing streaming data across your organization. It is an application-embeddable, distributed computing solution for building high-speed streaming applications, such as IoT and real-time analytics. Hazelcast Jet is built on the foundation of Hazelcast IMDG, the leading in-memory data grid and one of the top data stores for microservices deployments.
Whether you're interested in learning the basics of in-memory systems, or you're looking for advanced, real-world production examples and best practices, we've got you covered.
