Hive环境搭建01 – Neohope's Blog

1、安装MySQL

2、建立Hive的Meta库

CREATE DATABASE hive CHARACTER SET latin1;
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';
GRANT ALL PRIVILEGES ON hive.* to 'hive'@'localhost';

如果上面的库字符集不小心弄成了utf8，就和我一样，就会遇到下面的错误：

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes.

修正方法是：

alter database hive character set latin1;

3、下载hive-1.2.1，并解压
（*如果要看后续文章，建议使用hadoop-2.5.2 hbase-1.1.2 hive-1.2.1 spark-2.0.0）

4、下载MySQL的JDBC驱动mysql-connector-java-xxx.jar，放到hive的lib目录下

5、用hive中lib目录下jline-2.12.jar文件，替换掉hadoop下面share/hadoop/yarn/lib/jline-xxx.jar

6、复制hive-default.xml.template为hive-site.xml

7、修改hive-site.xml配置，分为下面几部分
7.1、mysql相关配置

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    <description>password to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseInfoNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <!--value>org.apache.derby.jdbc.EmbeddedDriver</value-->
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>Username to use against metastore database</description>
  </property

7.2、hadoop相关配置（HDFS上要有这些文件）

  <property>
    <name>hive.exec.scratchdir</name>
    <value>hdfs://hadoop-master:9000/hive/tmp</value>
    <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description>
  </property>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>hdfs://hadoop-master:9000/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
  <property>
    <name>hive.user.install.directory</name>
    <value>hdfs://hadoop-master:9000/hive/usr</value>
    <description>
      If hive (in tez mode only) cannot find a usable hive jar in "hive.jar.directory", 
      it will upload the hive jar to "hive.user.install.directory/user.name"
      and use it to run queries.
    </description>
  </property>

7.3、hive相关配置

  <property>
    <name>hive.metastore.uris</name>
    <value>thrift://hadoop-master:9083</value>
    <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
  </property>

7.4、hive本地路径相关配置
把下面的value，按部署实际情况，设置成实际存在的文件夹即可

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>${system:java.io.tmpdir}/${system:user.name}</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>
  <property>
    <name>hive.querylog.location</name>
    <value>${system:java.io.tmpdir}/${system:user.name}</value>
    <description>Location of Hive run time structured log file</description>
  </property>
  <property>
    <name>hive.server2.logging.operation.log.location</name>
    <value>${system:java.io.tmpdir}/${system:user.name}/operation_logs</value>
    <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
  </property>

8.配置完成：）

Leave a Reply Cancel reply