您可以从 impala 脚本调用 shell 命令行。
作为其工作原理的示例。
script_impala.sql
-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '${hivevar:game}';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '${hivevar:game}';
shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts;
shell cat hex_color.sql
$ impala-shell -f script_impala.sql
output
Starting Impala Shell without Kerberos authentication
Connected to localhost.localdomain:21000
Server version: impalad version 2.10.0-cdh5.13.3 RELEASE (build 15a453e15865344e75ce0fc6c4c760696d50f626)
Variable GAME set to Monopoly
Query: -- return the list of the game
SELECT list_price FROM fun.games WHERE name = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=a94afc7556843cf7:8b19809d00000000
+------------+
| list_price |
+------------+
| 19.99 |
+------------+
Fetched 1 row(s) in 0.15s
Query: -- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = 'Monopoly'
Query submitted at: 2020-06-04 23:29:19 (Coordinator: http://localhost.localdomain:25000)
Query progress can be monitored at: http://localhost.localdomain:25000/query_plan?query_id=c64d1c8950e5e1f8:15d410cb00000000
+-----------+-------+
| shop | price |
+-----------+-------+
| Dicey | 17.99 |
| Board 'Em | 25.00 |
+-----------+-------+
Fetched 2 row(s) in 0.14s
Found 5 items
drwxrwxrwt - mapred hadoop 0 2019-04-29 18:19 /user/history
drwxrwxrwx - hive hive 0 2019-04-29 18:19 /user/hive
drwxr-xr-x - hue hue 0 2019-11-25 10:19 /user/hue
drwxr-xr-x - spark spark 0 2019-04-29 18:19 /user/spark
drwxrwxrwx - training supergroup 0 2020-05-28 11:33 /user/training
--------
Executed in 2.10s
total 56
-rw-rw-r-- 1 training training 61 Sep 25 2019 hex_color.sql
-rw-rw-r-- 1 training training 115 Sep 25 2019 color_from_rgb.sql~
-rw-rw-r-- 1 training training 58 Sep 25 2019 hex_color_impala.sql
-rwxr-xr-x 1 training training 449 Sep 25 2019 email_results.sh
-rw-rw-r-- 1 training training 1166 Sep 25 2019 zero_air_time.csv
-rw-r--r-- 1 training training 261 Sep 26 2019 change_background.sh~
-rwxr-xr-x 1 training training 262 Sep 26 2019 change_background.sh
-rw------- 1 training training 2966 Sep 26 2019 ChangeVMDesktopColor.txt
-rw------- 1 training training 3279 Sep 26 2019 Hive&ImpalaInScripts&Applications.txt~
-rw------- 1 training training 3278 Sep 26 2019 Hive&ImpalaInScripts&Applications.txt
-rw-rw-r-- 1 training training 449 Sep 26 2019 email_resuts.sh
-rw-rw-r-- 1 training training 120 Apr 27 11:38 color_from_rgb.sql
-rw-rw-r-- 1 training training 397 Jun 4 23:24 game_prices.sql~
-rw-rw-r-- 1 training training 395 Jun 4 23:29 game_prices.sql
--------
Executed in 0.00s
I can echo from impala scripts
--------
Executed in 0.00s
SELECT hex FROM wax.crayons WHERE color = '${hivevar:color}'
--------
Executed in 0.01s
正如您所看到的，您只需在脚本中编写
shell <command>;
您甚至可以调用 hdfs dfs 命令
shell hdfs dfs -<command>;
上面的示例将输出打印到控制台，如果要将输出打印到文件，则必须执行以下操作以将所有输出重定向到文件：
$ impala-shell -f script_impala.sql >> /home/..../my_file.csv
其他选项是仅从脚本内部重定向 echo 命令。这将如下所示：
script_impala.sql
-- set a variable containing the of the game
SET hivevar:game=Monopoly;
-- return the list of the game
SELECT list_price FROM fun.games WHERE name = '${hivevar:game}';
-- return the prices of the game ate game shops
SELECT shop, price FROM fun.inventory WHERE game = '${hivevar:game}';
shell hdfs dfs -ls /user;
shell ls -ltr;
shell echo I can echo from impala scripts >> /home/..../my_file.csv;
shell echo How are you? >> /home/..../my_file.csv;
shell echo I am feeling very good, how about you? >> /home/..../my_file.csv;
shell cat hex_color.sql
并且不要忘记行末尾的分号“;”。
我希望它有帮助。
