Apache Beam Flink Pipeline Engine

    The Flink Runner and Flink are suitable for large scale, continuous jobs, and provide:

    • A runtime that supports very high throughput and low event latency at the same time

    • Fault-tolerance with exactly-once processing guarantees

    • Custom memory management for efficient and robust switching between in-memory and out-of-core data processing algorithms

    • Integration with YARN and other components of the Apache Hadoop ecosystem

    Check the Apache Beam Flink runner docs for more information.