在企業里快速搭建大數據平臺除了Ambria外還可以用Cloudera Manager，這兩種工具相對于純手工搭建確實便捷很多，但是有利也有弊，相對于新手來說就不便于掌握內部原理，不好排錯等,而純手工搭建，出錯較多，不容易成功。一. 規劃：
192.168.3.201 server/agent mysql
192.168.3.202 agent namenode
192.168.3.203 agent namenode（備） resourcemanager
192.168.3.204 agent resourcemanager （備）
192.168.3.205 agent
二. 環境準備
配置IP，hostname,
各個服務器IP配置：# vim /etc/sysconfig/network-script/ifcfg-eth0
service network restart
各個服務器名配置：# vim /etc/sysconfig/network
192.168.3.201 rac201.com
192.168.3.202 rac202.com
192.168.3.203 rac203.com
192.168.3.204 rac204.com
192.168.3.205 rac205.com
3.檢查DNS配置，確保能上網
# vim /etc/resolve.conf
# ping www.baidu.com
4.關閉防火墻和禁用selinux
getenforce 查看selinux的狀態并修改為disable狀態
# vim /etc/sysconfig/selinux
SELINUX=disable
重啟生效
5. 禁用和關閉防火墻
臨時禁用 # iptables -F
# setenforce 0
或設置開機不啟動
# service iptables stop
# chkconfig iptables off
6. 所有服務器卸載自帶的JDK，安裝新版本，比如1.7或1.8版本的
# rpm -qa | grep java
# rpm -e --nodeps tzdata-java-2012j-1.el6.noarch java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.x86_64 java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.x86_64
# mkdir -p /opt/modules
# tar zxf jdk-8u141-linux-x64.gz -C /opt/modules
7. 對所有服務器配置JAVA環境變量
# vi /etc/profile
export JAVA_HOME=/opt/modules/jdk1.8.0_141
export PATH=$PATH:$JAVA_HOME/bin
# source /etc/profile
8. 拷貝配置到其他服務器
# scp -r /opt/modules/jdk1.8.0_141 root@rac202.com:/opt/modules/
# scp -r /opt/modules/jdk1.8.0_141 root@rac203.com:/opt/modules/
# scp -r /opt/modules/jdk1.8.0_141 root@rac204.com:/opt/modules/
# scp -r /opt/modules/jdk1.8.0_141 root@rac205.com:/opt/modules/
檢查： java -version
9. 配置服務器之間免密鑰登錄
9.1 # ssh-keygen
# ssh-copy-id -i ~/.ssh/id_rsa.pub rac201.com
# ssh-copy-id -i ~/.ssh/id_rsa.pub rac202.com
# ssh-copy-id -i ~/.ssh/id_rsa.pub rac203.com
# ssh-copy-id -i ~/.ssh/id_rsa.pub rac204.com
# ssh-copy-id -i ~/.ssh/id_rsa.pub rac205.com
9.2 # ssh rac01.com
# exit #不要忘記連接后要退出
# ssh rac02.com
# exit
# ssh rac03.com
# exit
# ssh rac04.com
# exit
# ssh rac05.com
# exit
10. 服務器時間同步
10.1 啟動ntpd進程服務
# service ntpd start
# chkconfig ntpd on
# chkconfig --list | grep ntpd
10.2 修改時間服務器的配置
# vim /etc/ntp.conf
打開注釋，修改成本服務器的網段
# restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
注釋掉以下幾行，可能是三行也可能是五行
#server 0.centos.pool.ntp.org
#server 1.centos.pool.ntp.org
#server 2.centos.pool.ntp.org
手工添加以下兩行
server 127.127.1.0 # local clock
fudge 127.127.1.0 s
啟動并且配置開機時啟動
# service ntpd start && chkconfig ntpd on && chkconfig --list | grep ntpd
11. 對所有服務器設置能打開的文件最大數量、進程最大數，能使用的內存等
# vi /etc/security/limits.conf
* soft nofile 32728
* hard nofile 1024567
* soft nproc 65535
* hard nproc unlimited
* soft memlock unlimited
* hard memlock unlimited
復制limits.conf到其他服務器中
# scp /etc/security/limits.conf root@rac202.com:/etc/security/limits.conf
# scp /etc/security/limits.conf root@rac203.com:/etc/security/limits.conf
# scp /etc/security/limits.conf root@rac204.com:/etc/security/limits.conf
# scp /etc/security/limits.conf root@rac205.com:/etc/security/limits.conf
重啟生效： ulimit -a 三. Mysql數據庫安裝
在rac201服務器檢查并卸載系統自動安裝的Mysql版本,
# rpm -e --nodeps mysql-libs-5.1.71-1.el6.x86_64采用yum源方式安裝Mysql5.6, 先下載yum源文件,
# rpm -Uvh mysql57-community-release-el6-8.noarch.rpm
修改yum源文件：
vim /etc/yum.repos.d/mysql-community.repo
[mysql56-community]
enabled=1
gpgcheck=1
[mysql57-community]
enabled=0
gpgcheck=1
vim /etc/yum.repos.d/mysql-community-source.repo
[mysql56-community-source]
enabled=1
gpgcheck=1
[mysql-tools-preview-source]
name=MySQL Tools Preview - Source
baseurl=http://repo.mysql.com/yum/mysql-tools-preview/el/6/SRPMS
enabled=1
gpgcheck=1
用yum repolist mysql這個命令查看一下是否已經有mysql可安裝文件
#yum repolist all | grep mysql
yum安裝mysql, 一直yes下去
# yum -y install mysql-community-server
安裝成功后，啟動并初始化
# service mysqld start
# service mysqld status
mysql剛安裝完，mysql的root用戶的密碼默認是空的，需要給root用戶設置密碼密碼，執行下列命令進行安全設置
# mysql_secure_installation
輸入當前用戶：回車
輸入root密碼：123456
production environment.
Remove anonymous users? [Y/n] y
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] n
... skipping.
By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] n
... skipping.
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] y
... Success! 四. Cloudera Manager安裝
在每臺服務器上安裝第三方依賴包，或者在Centos系統安裝時安裝好開發包
yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs redhat-lsb
提前下載好cloudera-manager-el6-cm5.3.6_x86_64.tar.gz
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.3.6_x86_64.tar.gz
在rac201服務器上創建CM的解壓目錄
在rac201服務器上修改修改agent配置
server_host=rac201.com
server_port=7182 拷貝解壓后的CM文件夾到其他服務器
# scp -r /opt/cloudera-manager/ root@rac203.com:/opt/
# scp -r /opt/cloudera-manager/ root@rac204.com:/opt/
# scp -r /opt/cloudera-manager/ root@rac205.com:/opt/在所有服務器上創建CM的用戶
# cat /etc/passwd | grep cloudera-scm設置CM數據庫的權限
mysql> grant all privileges on *.* to 'temp'@'%' identified by 'temp' with grant option;
mysql> grant all privileges on *.* to 'temp'@'rac01.com' identified by 'temp' with grant option;
mysql> flush privileges;
mysql> use mysql;
mysql> select host ,user from user;
進入rac201服務器以下文件夾，執行腳本命令，事先要將mysql-connector-java.jar放入/usr/share/java文件夾中
# cd /opt/cloudera-manager/cm-5.3.6/share/cmf/schema
# ./scm_prepare_database.sh mysql -h rac01.com -utemp -ptemp --scm-host rac01.com scm scm scm 提前下載好CDH源文件
CDH-5.3.6-1.cdh6.3.6.p0.11-el6.parcel
CDH-5.3.6-1.cdh6.3.6.p0.11-el6.parcel.sha
在rac201服務器上創建以下目錄
# mv CDH-5.3.6-1.cdh6.3.6.p0.11-el6.parcel* /opt/cloudera/parcel-repo/
在所有服務器上創建
# mkdir -p /opt/cloudera/parcels
更改parcel目錄的所有人
在rac201服務器上
# chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/
在所有服務器上
# chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcels 啟動CM的進程
在rac201服務器上
# ./cloudera-scm-server start 在所有服務器上啟動agent進程
# ./cloudera-scm-agent start CM的服務器登錄界面
http://rac01.com:7180
登錄賬號和密碼 admin admin
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
