In this tutorial, We will see the comparison between Apache Flink, Apache Spark, and Apache Hadoop framework. All these technologies are very famous and the best in their way. We will map these three technologies on different parameters and share the detailed deference.
The following table shows the detailed comparison of Apache Flink, Apache Hadoop, and Apache Hadoop.
| Different Parameters | Apache Hadoop | Apache Spark | Apache Flink |
|---|---|---|---|
| Technology Birth | Apache Hadoop Birth year is 2005. | Apache Spark Birth year is 2009. | Apache Flink Birth year is 2009. |
| Technology Birth Place | The birthplace of MapReduce is Google and Hadoop is at Yahoo. | The birthplace of Apache Spark is the University of California, Berkeley. | The birthplace of Apache Flink is the Technical University of Berlin. |
| Type of Processing Engine | Apache Hadoop is a batch processing engine. | Apache Spark is a batch processing engine. | Apache Flink is a stream processing engine. |
| Data Processing Speed | Hadoop is Slower than Spark and Flink. | Apache Spark is 100x Faster than Hadoop. | Apache Flink is Faster than Apache Spark. |
| Language Support | Hadoop supports Java, C, C++, Ruby, Groovy, Perl, Python. | Apache Spark supports Java, Scala, Python, and R. | Apache Flink supports Java and Scala. |
| Programming Model Support | Hadoop uses the MapReduce programming model. | Apache Spark uses Resilient distributed Datasets (RDD) programming model | Apache Flink uses Cyclic dataflows programming model. |
| Data Movement | Batch | Batch | Pipelined and Batch |
| Memory Management | Hadoop supports Disk-Based memory management. | Apache Spark supports JVM-based memory management. | Apache Flink supports Active based memory management. |
| Latency Measurement | The latency in Hadoop is Low. | The latency of Apache Spark is Medium. | The Latency of Apache Flink is Low. |
| Throughput Measurement | It provides medium throughput. | It provides high throughput. | It provides high throughput. |
| Optimization Level | The Optimization in Hadoop is Manual. | The Optimization in Spark is Manual. | The Optimization in Flink is Automatic. |
| API Support | Low-level | High-level | High-level |
| Streaming Support | Hadoop does not support streaming. | Apache Spark supports Spark Streaming. | Apache Flink supports the Flink Streaming. |
| SQL Support | Hadoop supports Hive, Impala for SQL. | Apache Spark uses SparkSQL for SQL. | Apache Flink uses the table API and SQL to support SQL. |
| Graph Support | Hadoop does not support Graph. | Apache Spark uses GraphX to support Graph. | Apache Flink uses the Gelly to support Graph. |
| Machine Learning Support | Hadoop does not support Machine Learning | Apache Spark uses SparkML to support Machine Learning. | Apache Flink uses FlinkML to support Machine Learning. |