Tez Sessions – Using the Same ApplicationMaster for Queries

Hive on Tez execution engine allows you to use sessions. This means that you can use the same ApplicationMaster to submit your queries.

Let’s start Hive and run the first query:

set hive.execution.engine=tez;

select count(*) from tab1;

Query ID = v-dtolpeko_20141008091515_ccd90ab1-4771-4515-beb7-6696be762263
Total jobs = 1
Launching Job 1 out of 1

Status: Running (application id: application_1412375486094_99119)

Map 1: -/-      Reducer 2: 0/1
Map 1: 0/1      Reducer 2: 0/1
Map 1: 1/1      Reducer 2: 1/1
Status: Finished successfully
OK

Then let’s start the second query:

select count(*) from tab2;

Query ID = v-dtolpeko_20141008091616_e8f7f0ed-7f64-41b7-ba6e-c0fbe49191d7
Total jobs = 1
Launching Job 1 out of 1

Status: Running (application id: application_1412375486094_99119)

Map 1: -/-      Reducer 2: 0/1
Map 1: 1/1      Reducer 2: 1/1
Status: Finished successfully
OK

You can see that both queries use the same application_id application_1412375486094_99119 that means they use the same ApplicationMaster. In contrast to MapReduce that has to launch a new ApplicationMaster for every query (and even multiple ApplicationMaster per single query), Tez sessions allow you to reduce the job startup time.

Note that ApplicationMaster does not execute jobs, it has to launch one or more Containers to execute the query. To reuse containers consider also specifying:

set hive.prewarm.enabled=true;

Leave a Reply