E-MapReduce集群支持预装Phoenix

E-MapReduce在emr-2.0.0版本开始，在集群中会集成进phoenix-4.7.0，用户在创建集群的软件列表中需要勾选HBase，集群创建后即可使用phoenix。

创建集群

Phoenix使用HBase作为底层存储，所以创建集群时需要勾选HBase软件，如下：

集群空闲后，可以从集群的详情页面查看Zookeeper节点，如下：

phoenix使用上面Zookeeper节点进行相关操作。

Phoenix使用示例

1.示例一

执行:

sudo su hadoop
performance.py localhost 1000

输出(截取部分):

CSV Upsert complete. 1000 rows upserted
Time: 1.298 sec(s)

                                COUNT(1)
----------------------------------------
                                    2000
Time: 0.437 sec(s)

HO
--
CS
EU
NA
Time: 0.2 sec(s)

DOMAIN
----------------------------------------
Apple.com
Google.com
Salesforce.com
Time: 0.079 sec(s)

DAY
-----------------------
2016-06-13 00:00:00.000
Time: 0.076 sec(s)

                                COUNT(1)
----------------------------------------
                                      45
Time: 0.068 sec(s)

2.示例二

执行:

sudo su hadoop
cd ~
echo '100,Jack,Doe' >>example.csv
echo '200,Tony,Poppins' >>example.csv

sqlline.py localhost

0: jdbc:phoenix:localhost> CREATE TABLE example (
    my_pk bigint not null,
    m.first_name varchar(50),
    m.last_name varchar(50)
    CONSTRAINT pk PRIMARY KEY (my_pk));

psql.py -t EXAMPLE localhost example.csv

验证:

sqlline.py localhost

0: jdbc:phoenix:localhost> select * from example;

+--------+-------------+------------+
| MY_PK  | FIRST_NAME  | LAST_NAME  |
+--------+-------------+------------+
| 100    | Jack        | Doe        |
| 200    | Tony        | Poppins    |
+--------+-------------+------------+

3.示例三

执行:

sudo su hadoop
cd ~
echo '100,Jack,Doe' >>example.csv
echo '200,Tony,Poppins' >>example.csv

hadoop dfs -put example.csv /

sqlline.py localhost
0: jdbc:phoenix:localhost> CREATE TABLE example (
    my_pk bigint not null,
    m.first_name varchar(50),
    m.last_name varchar(50)
    CONSTRAINT pk PRIMARY KEY (my_pk));

hadoop jar /opt/apps/phoenix-4.7.0-HBase-1.1-bin/phoenix-4.7.0-HBase-1.1-client.jar org.apache.phoenix.mapreduce.CsvBulkLoadTool -Dyarn.application.classpath='/opt/apps/phoenix-4.7.0-HBase-1.1-bin/*,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*,/opt/apps/extra-jars/*' --input  /example.csv -z localhost -t example

验证:

sqlline.py localhost

0: jdbc:phoenix:localhost> select * from example;

+--------+-------------+------------+
| MY_PK  | FIRST_NAME  | LAST_NAME  |
+--------+-------------+------------+
| 100    | Jack        | Doe        |
| 200    | Tony        | Poppins    |
+--------+-------------+------------+