Restart the network service as shown above, this command was not changed, then check the ip.
Each unit performs such configure。
2． hostname change
Make changes to all three separately, and note the name and ip of each one, matching each one.
[root@hodoop1~] vi /etc/sysconfig/network
# configure elements
This change will require a reboot to take effect, or it can take effect immediately by using the following.
[root@hodoop1~] hostname cdh01
cdh01 3． Turn off the firewall
CentOS 7.0 uses firewall by default, and the related operation has changed a lot, and is not at all the same as in 6.5, as follows.
[root@hadoop1~]firewall-cmd --state #View firewall status
[root@hadoop1~]systemctl stop firewalld # Stop the firewall's services
[root@hadoop1~]systemctl disable firewalld # Disable bootup
rm '/etc/systemd/system/basic.target.wants/firewalld.service' 4． selinux shutdown
All nodes need to be closed, as required by the official documentation.
Reboot to take effect, check after reboot.
SELinux status: disabled # indicates that it has been turned off. 5． No Secret Login configure
erect in the course ofmaster The need for individual nodes of theroot No login password， first inmaster Generate public key on：
Do the above for each node separately and verify that you can log in to the node server directly without entering a password by using the ssh ip when you are done.
6． ip harmony hostname (of a networked computer) mapping relationship
The operation is as follows.
# It reads as follows
Content as above， Save to exit， Then you can use thescp The command will take this configure Copy to other hosts。
scp /etc/hosts firstname.lastname@example.org:/etc/hosts 7． NTP server settings
Many blogs online mention this configure， It says to make the nodes uniform in time， But I didn't. configure， I'm also putting this configure Put it here.：
[root@cdh01~]yum -y install ntp #Change the master's node
Comment out all server *.*.* pointers and add a new connectable ntp server (Baidu for ntp server)
server ntp.sjtu.edu.cn iburst
Just point ntp to the master server address on the other nodes (under /etc/ntp.conf)
server 192.168.160.130 iburst All nodes.
2、 Third-party reliance packet 1． Other dependencies
For this step， You can read many blogs online that put this step intoMySQL erect behind the， But after my owntest (machinery etc)， It's better to put this step in front， Because behind erect Software， But where there is dependence， You've done it here， So operate here erect depend packet， And it's all nodes erect。
yum install chkconfig python bind-utils psmisc libxslt zlib sqlite fuse fuse-libs redhat-lsb cyrus-sasl-plain cyrus-sasl-gssapi
Note that this place dependency package must be installed completely, will go through three y/N process, all choose Y, this installation process depends entirely on your Internet speed, basically after knocking this command you can rest for a while.
2． MySQL jar packet
This session only needs to be done on the master node.
atcdh01 on preparationmysql ofjar packet：
mkdir -p /usr/share/java
revisejar packet name， and copy to/usr/share/java/ directory：
cp mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar
Note the above， It's modifiedjar packet name， Because there's an operation behind it that looks for thisjar packet this fixed name， So here's a change of name。
This is the foundation of all services， Each node needs erect。
erect Check it first before， Does your server default to erect finishOpenJDK， If the system comes with， would require uninstallation， The steps are as follows：
# Check if erect
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-126.96.36.199.el7_0-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
# look over need uninstallation of packet
[root@cdh01~]rpm -qa | grep jdk
[root@cdh01~]yum -y remove java-1.7.0-openjdk-188.8.131.52-184.108.40.206.el7_0.x86_64
[root@cdh01~]yum -y remove java-1.7.0-openjdk-headless-220.127.116.11-18.104.22.168.el7_0.x86_64
# Double check.
bash: /usr/bin/java: No such file or directory
If not erect， You can skip this step， My system doesn't come with itjdk。
Choose either of the two methods below, don't do both.
1> rpm or yum installation
Use commands directly erect Can， There's not much to explain here。
2> Use compression packet erect
Upload the prepared installation package, the directory I'm used to is: /home/software, finish uploading and unzip:.
tar -zxvf jdk-8u131-linux-x64.tar.gz
mv jdk-1.8.0 jdk1.8
If you want to erectspark， You can combine the environment variables withScala together configure。
configure environment variable：
Add the following to the document.
After saving the exit, a soft connection needs to be set up here as well:.
ln -s /home/software/jdk1.8 /usr/java/default
If you are usingyum orrpm erect ofjdk These steps can then be omitted。 BecauseCDH platform erect The default look for when thejdk The path is/usr/java。
If you want to erectSpark Must not be omittedScala of erect， if not， You're in.web end-to-end operationCDH when， will give you a mistake， The solution on the web， It's basically all saidjdk No one said an ever-say-wellScala。
This operation is likewise on all nodes.
The directory is still there/home/software。 The operation is as follows.
tar -zxvf scala-2.11.0.tgz
mv scala-2.11.0 scala2.11
And then configure environment variable：
Also add the following at the end.
export PATH=$PATH:$SCALA_HOME/bin 5、 erectmysql
MySQL installation only needs to be done on the master node.
erectMySQL It also needs to be checked， Whether the system comes with it or not，centos7 Comes with itmariadb， Check first， The command is as follows：
[root@cdh01]rpm -qa | grep mariadb
A situation such as the above exists， If present use the following command for uninstallation：
[root@cdh01]rpm -e --nodeps mariadb-libs-5.5.41-2.el7_0.x86_64 2． erect
Place the downloadedMySQLrpm packet Copy to the server， Then unzip it.、 erect。
1> Creating Users and User Groups # Adding the user group mysql.
#Add user mysql, add mysql user group.
useradd -r -g mysql mysql 2> erectMySQL # erectserver：
rpm -ivh MySQL-server-5.6.29-1.linux_glibc2.5.x86_64.rpm
rpm -ivh MySQL-client-5.6.29-1.linux_glibc2.5.x86_64.rpm 3> Add random start
Add mysqld to the system service and start it randomly with the following command.
cp /usr/share/mysql/mysql.server /etc/init.d/mysqld 4> Starting MySQL
The command to start mysqld is as follows.
service mysqld start 5> Change Password
First get the random password for the root user on the mysql installation:.
It can also be viewed using the cat command at
This password can only be used to change the password.
You must change the password of the root user to use mysql, otherwise you can only connect but not operate
mysqladmin -u root -p password root 6> test (machinery etc) #connect into mysql with the following command.
mysql -u root -p
# look overmysql of erect Run path， The command is as follows：
ps -ef|grep mysql 7> MySQL Related Questions
If there is a problem with no permissions, authorize in mysql authorization (executed on the machine where mysql is installed) by executing the following statement.
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'root' WITH GRANT OPTION;
*.*:All tables under all libraries.
%： any IP address or the host can connect。 If% configure Does not take effect， only configure concrete hostname (of a networked computer) call。
3． Create database
The following command is to create the database needed to deploy each service, I personally prefer not to create a good first, when you can directly deploy the service, do not have to come back to the database to create.
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci; III. erectCloudera-Manager
willCloudera Related to the four packet， as follows：
Upload to all servers and perform the following actions on all servers.
unpackcm tar packet to the specified directory， first establish directory， command The operation is as follows.
[root@cdh01 ~]mkdir /opt/cloudera-manager
[root@cdh01 ~]tar -zxvf cloudera-manager-centos7-cm5.7.2_x86_64.tar.gz -C /opt/cloudera-manager
Create cloudera-scm user
[root@cdh01 ~]useradd -r -d /opt/cloudera-manager/cm-5.7.2/run/cloudera-scm-server -M -c "Cloudera SCM User" cloudera-scm
# look over
[root@cdh01 ~]id cloudera-scm 2、 configure 1． configure from a node leader of a group
configure from a nodecloudera-manger-agent Pointing to the master node server， My current cluster planning， is required on each server as follows configure：
willserver_host change intoCMS whereabouts hostname (of a networked computer) namelycdh01
server_host=cdh01 2． configure Warehouse catalog
This operation can be performed only on the master node, where the parcel-repo repository directory is created with the following command.
[root@cdh01 ~]mkdir -p /opt/cloudera/parcel-repo
[root@cdh01 ~]chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
[root@cdh01 ~]cp CDH-5.7.2-1.cdh5.7.2.p0.18-el7.parcel CDH-5.7.2-1.cdh5.7.2.p0.18-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo
Note: where CDH-5.7.2-1.cdh5.7.2.p0.18-el5.parcel.sha1 suffix is to be removed from 1, the operation is as follows.
mv CDH-5.7.2-1.cdh5.7.2.p0.18-el7.parcel.sha1 CDH-5.7.2-1.cdh5.7.2.p0.18-el7.parcel.sha 3． configureCDH from a node directory
Create the parcels directory on all nodes by doing the following.
mkdir -p /opt/cloudera/parcels
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels
Explanation: Clouder-Manager extracts CDH from the /opt/cloudera/parcel-repo directory of the master node and distributes the decompression activation to the /opt/cloudera/parcels directory of each node.
4． Initializing the database
This operation is performed on the master node with the initial script configure database scm_prepare_database.sh with the following action command.
[root@cdh01 ~]/opt/cloudera-manager/cm-5.7.2/share/cmf/schema/scm_prepare_database.sh mysql -hcdh01 -uroot -proot --scm-host cdh01 scmdbn scmdbu scmdbp
Description: This script is the one used to create and configure the database needed by the CMS. The parameters are.
mysql： The database is usedmysql， If erect Used in the processoracle， Then the argument should be changed tooracle。
-hcdh01: The database is built on top of the cdh01 host. That is, above the master node.
-uroot: run mysql as root. -proot: the root password for mysql is root.
--scm-host cdh01：CMS Host， Generally it's a summysql erect The host of is on the same host。
The last three parameters are: database name, database username, and database password.
Execution of the completion order is normal as follows.
This is where it explains why the jar name was changed above.
3、 Start Service 1． Start the server
This command can be executed only on the master node.
Go to the /opt/cloudera-manager/cm-5.7.2/etc/init.d/ directory and execute the following command.
# Or simply execute the following command.
/opt/cloudera-manager/cm-5.7.2/etc/init.d/cloudera-scm-server start 2． Start agent
Start cloudera-scm-agent, which needs to be started on all slave nodes.
Go to the /opt/cloudera-manager/cm-5.7.2/etc/init.d/ directory and execute the following command.
# Or simply execute the following command.
Activation is shown in the following figure.
Just to clarify here, basically both of the above starts will be configured to randomly start, but I experimented with many ways to randomly start and none of them worked, so I'll leave it here for now and add it later when I find a way to randomly start.
Note: The following steps will not cause any problems as long as the above links are done without any problems. IV. serve erect 1、 web login
Type 192.168.75.41:7180 into your browser.
The following login screen appears with the default username and password: admin
This interface appears instructionsCM already erect It worked， Here's itweb Various services are deployed in the interface。
2、 web boot installation
No screens are mentioned in this process, and continue is selected by default.
1． Select the express version first
When logged in, you will be taken to the screen to select the express version, just select free on this screen and continue.
2． configure host
Each of the servers from a node already erect and startedagent， Also on each node configure finishserver point to， So each node'sagent will giveserver Send a message to report， It is possible to find it here in“ Currently managed hosts” Three hosts are seen in， Check all and continue。
Note that ifcloudera-scm-agent Not set to boot up， If the above has restarted here it may not detect other servers。 before configure when， I didn't configure Random start。
3． Select CDH version
Just choose the version you need here. Here I have chosen two items, one of the selection methods uses Parcel; the second CDH version selected is CDH-5.7.2-1 cdh5.7.2p0 18; all others are selected as none.
This step is to distribute parcels to each node, the previous step continues, this step is automatic no action is needed, the only thing needed is to wait. When all are done click Continue.
5． configure verify
This step is for the host configure the correctness of the test。
This is a place to note that two items in this area didn't pass the check, but I didn't do anything about it, I found a method online, but I don't know if it works exactly, in the following way.
The following commands can be used in a cluster
echo 0 > /proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/defrag
Then clicking re-run above will show that all checks passed this time.
I didn't do the above, but the back install didn't affect it either.
6． Select Service
The play is coming， Here's what you want to choose erect The service used，CDH5 By default, six combinations are available as shown， Of course, that can also be customized， Here I choose customization。
7． Role assignment
Here's the service Role assignment To that node， It can be defaulted to， Or you can make your own choices。
8． Database settings selection
This is where the individual databases created earlier are used, and depending on the service you choose, this will ask you to fill in the database used for each service, as well as the username and password.
9． Cluster Audit
Just leave it all as default here, unless you have a plan and can make changes here.
10． commencement erect
If web boot installation Something could go wrong.， Then the problem is all in this step， Questions that will arise here， The problem I'm having， It's basically all described earlier， It is erectjdk harmonyScala when。 Basically nothing else will go wrong。
11． erect finish
At this point the installation is complete, click Finish and you will be taken to the following screen.
Checking the cluster situation, I have quite a lot of alarms here, probably looking at them is basically alarms for memory or storage usage thresholds, since I'm a local VM, these conditions are limited, so I won't deal with them here for now.