What is Stream Processing

Definition

Stream processing is the continuous handling of data records as they arrive, computing results event by event or over short rolling windows, so that insights, alerts, and actions are produced within milliseconds to seconds rather than after a scheduled batch.

« Back to Glossary Index

Produces results within milliseconds, enabling real-time alerting and personalisation
Processes unbounded event streams without waiting for a batch window to close
Supports windowed aggregations that summarise live data over rolling time periods
Keeps operational dashboards and models continuously up to date

Real World Example

A fraud-detection system uses stream processing to score each card transaction the instant it occurs, blocking suspicious payments in real time instead of catching them in a report the next morning.

FAQs

What is a window in stream processing?

A window groups events over a time span, such as the last five minutes, so the system can compute aggregates over continuous streams.

How does stream processing differ from batch?

It processes each event as it arrives for near-instant results, while batch waits to process accumulated data on a schedule.

What frameworks support stream processing?

Apache Flink, Spark Structured Streaming, and Kafka Streams are widely used stream-processing engines.