Note that you can run multiple programs per session.
Start Flink Sessionįollow these instructions to learn how to launch a Flink Session within your YARN cluster.Ī session will start all required Flink services (JobManager and TaskManagers) so that you can submit programs to the cluster. If you have troubles using the Flink YARN client, have a look in the FAQ section. HDFS (Hadoop Distributed File System) (or another distributed file system supported by Hadoop).Users do not have to setup or install anything if there is already a YARN setup. Flink runs on YARN next to other applications. It allows to run various distributed applications on top of a cluster. examples/batch/WordCount.jar Flink YARN SessionĪpache Hadoop YARN is a cluster resource management framework. bin/flink run -m yarn-cluster -p 4 -yjm 1024m -ytm 4096m.
Run a Flink job on YARN # get the hadoop2 package from the Flink download page at # Once the session has been started, you can submit jobs to the cluster using the. We recommend to set the number of slots to the number of processors per machine. Specify the -s flag for the number of processing slots per Task Manager. Start a YARN session where the job manager gets 1 GB of heap space and the task managers 4 GB of heap space assigned: # get the hadoop2 package from the Flink download page at # Quickstart Start a long-running Flink cluster on YARN Build YARN client for a specific Hadoop version.Start a long-running Flink cluster on YARN.We recommend you use the latest stable version. This documentation is for an out-of-date version of Apache Flink.