oracle 19c rac install 之grid install

一 前置檢查：
安裝最低要求：
Red Hat Enterprise Linux 7.4: 3.10.0-693.5.2.0.1.el7.x86_64 or later
At least 8 GB RAM for Oracle Grid Infrastructure installations. （其實4G也可以安裝完成）
SWAP：4G
安裝目錄：grid 10G,介質安裝完成6.4G
參考:grid install guide
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/server-hardware-checklist-for-oracle-grid-infrastructure.html#GUID-D964EF96-9C9C-4173-A919-F3CAA72180B1
這里環境概述：
IP：和11G準備一樣，5個公網，2個私網；
共享存儲：5GB；
操作系統：RHEL7.6 x64
介質準備：V981624-01.zip，V981627-01.zip，V981623-01.zip三個壓縮文件，比18C小一些；只有for exadata版 本，在edelivery.oracle.com下載，公版沒出來，不過裝在linux應當是沒有問題；介質變化：
1 軟件解壓目錄即安裝目錄
2 自帶JDK 1.8/TFA
3 logfile默認由50調到了200M

二 網絡配置
OS采用最小安裝，安裝過程略，安裝完成后先啟用臨時網絡：

nmcli con modify "System eth0" ipv4.addresses 192.168.56.102/24 
nmcli con up "System eth0" 
ip addr show

設置主機名：
hostnamectl set-hostname rhel6
由于rhel7的網卡名稱命名變化,如需顯示eth0,eth2,需在RHEL 7修改網卡參數：
1、修改/etc/sysconfig/grub或者/etc/default/grub，它們是一個文件，改誰都可以，需要添加參數，給GRUB_CMDLINE_LINUX參數中增加
net.ifnames=0 biosdevname=0
然后：
grub2-mkconfig -o /boot/grub2/grub.cfg
2、配置網卡

mv /etc/sysconfig/network-scripts/ifcfg-enp0s3 /etc/sysconfig/network-scripts/ifcfg-eth0
vi /etc/sysconfig/network-scripts/ifcfg-eth0
修改：
device=eth0
name=eth0

*3、非必須，更改/etc/udev/ruled.d/90-eno-fix.rules，將其中的NAME="enp0s3"改為eth0
最后重啟主機生效：
#禁用dhcp 169.254路由表

echo "NOZEROCONF=yes">>/etc/sysconfig/network
/etc/init.d/network restart

#RHEL 7禁用防火墻

 systemctl stop firewalld
systemctl mask firewalld

三 配置共享磁盤屬性

vi /etc/rc.d/rc.local 
/usr/bin/raw /dev/raw/raw1 /dev/sdb
/usr/bin/chown grid:asmadmin /dev/raw/raw1
/usr/bin/chmod 660 /dev/raw/raw1
chmod +x /etc/rc.d/rc.local

*四 建立用戶
如果先安裝oracle的yum包，會自動建立

$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 
54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),
54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

這種用戶

groupadd -g 501 oinstall
groupadd -g 502 dba
groupadd -g 504 asmadmin
groupadd -g 506 asmdba
groupadd -g 507 asmoper
useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper grid
useradd -u 502 -g oinstall -G dba,asmdba oracle
echo 'oracle123' |passwd oracle --stdin;
echo 'oracle123' |passwd grid --stdin;

--配置ssh信任關系
./oui/prov/resources/scripts/sshUserSetup.sh -user grid -hosts "rhel6 rhel7" -noPromptPassphrase
或者
ssh-keygen生成用戶密鑰,配置互信；

五 配置yum 軟件包
cd /etc/yum.repos.d/
mkdir bak
mv Centos ./bak
直接：
wget http://yum.oracle.com/public-yum-ol7.repo
修改其中的ol7_latest
或者新建一個repo文件：

[ol7_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL7/latest/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=0
enabled=1

由于最小安裝，需補充包:

yum install oracle-database-preinstall-18c
yum -y install gcc*
yun -y install xhost+

當然也可以用傳統的掛載鏡像來做yum源

mount -o loop /tmp/rhel-server-6.6-x86_64-dvd.iso /media
vi /etc/yum.respos.d/dvd.repo
[c6-media]
name=CentOS-$releasever - Media
baseurl=file:///media
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' bc \
binutils \
compat-libcap1 \
compat-libstdc++ \
dtrace-modules \
dtrace-modules-headers \
dtrace-modules-provider-headers \
dtrace-utils \
elfutils-libelf \
elfutils-libelf-devel \
fontconfig-devel \
glibc \
glibc-devel \
ksh \
libaio \
libaio-devel \
libdtrace-ctf-devel \
libX11 \
libXau \
libXi \
libXtst \
libXrender \
libXrender-devel \
libgcc \
librdmacm-devel \
libstdc++ \
libstdc++-devel \
libxcb 

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel|grep -i "not installed"|awk '{print $2}'|xargs yum install -y

如果為RHEL6得升級glibc,libstdc++庫到2.14/4.5以下版本；
檢查版本:```
ldd --version
[grid@rhel6 grid]$ strings /usr/lib64/libstdc++.so.6|grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_FORCE_NEW
GLIBCXX_DEBUG_MESSAGE_LENGTH

**六 配置hosts**

vi /etc/hosts
127.0.0.1 localhost
#eth0 - PUBLIC
192.168.56.101 rhel7
192.168.56.102 rhel6
#VIP
192.168.56.103 rhel7-vip
192.168.56.104 rhel6-vip
#eth2 - PRIVATE
10.0.2.15 rhel7-pvt
10.0.2.16 rhel6-pvt
192.168.56.105 rac-scan

**七 配置 參數**

RHEL7 yum install后可以忽略

vi /etc/sysctl.conf
kernel.shmall = 4294967296
kernel.shmmni =4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
kernel.panic_on_oops=1

#sysctl -p

#調整shm:

tmpfs /dev/shm tmpfs defaults,size=3000M 0 0
mount -o remount /dev/shm

修改limit參數：
/etc/security/limits.conf

grid soft nproc 4096
grid hard nproc 16384
grid soft nofile 4096
grid hard nofile 65536
oracle soft nproc 4096
oracle hard nproc 16384
oracle soft nofile 4096
oracle hard nofile 65536

RHEL6注意可能不生效的情況需修改：
`/etc/security/limits.d/90-nproc.conf`
來源詳細參見：
http://blog.yufeng.info/archives/2568

vi /etc/pam.d/login
session required pam_limits.so

vi /etc/profile
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

#關閉selinux
setenforce 0
vi /etc/sysconfig/selinux，修改target參數為disabled

**8 配置環境變量**

export ORACLE_HOME=/data/grid
export ORACLE_BASE=/u01/grid
export ORACLE_SID=+ASM2
export DISPLAY=192.168.56.1:0.0

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$ORACLE_HOME/bin

**9 安裝**
執行安裝前檢查
` ./runcluvfy.sh stage -pre crsinst -n rhel6,rhel7 -verbose`

>  Verifying DNS/NIS name service ...FAILED
> PRVG-2058 : The 'hosts' entries in the existing "/etc/nsswitch.conf" files are
> inconsistent.
> "hosts" entry was found as "hosts: files dns myhostname" on nodes: rhel7
> "hosts" entry was found as "hosts: files dns" on nodes: rhel6
>  
> Verifying Daemon "avahi-daemon" not configured and running ...FAILED
> Verifying RPM Package Manager database ...INFORMATION
> PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges.

一個是DNS解析文件的順序不一致，一個是avahi-daemon沒配置。
查看avahi解釋：
Avahi允許程序在不需要進行手動網絡配置的情況 下，在一個本地網絡中發布和獲知各種服務和主機。例如，當某用戶把他的計算機接入到某個局域網時，如果他的機器運行有Avahi服務，則Avahi程式自 動廣播，從而發現網絡中可用的打印機、共享文件和可相互聊天的其他用戶，這有點象他正在接收局域網中的各種網絡廣告一樣。
如此看來，可以先忽略；
 運行：  
`  ./gridSetup.sh `
注意：先擇 網絡時選擇asm&private,ausize默認為4MB了,其他安裝和以前基本相同；
查看安裝日志:
` tail -30f /data/grid/install/root_rhel7_2019-02-23_13-05-05-377719388.log`
 查看ohasd日志:

tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd.trc
tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd_orarootagent_root.trc

**10 查看資源狀態**
> [grid@rhel6 bin]$ crsctl stat res -t
> 
> Name           Target  State        Server                   State details       
> 
> Local Resources
> 
> ora.LISTENER.lsnr
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.chad
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.net1.network
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.ons
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> 
> Cluster Resources
> 
> ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.DATA.dg(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.LISTENER_SCAN1.lsnr
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.asm(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    Started,STABLE
>       2        ONLINE  ONLINE       rhel6                    Started,STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.asmnet1.asmnetwork(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.cvu
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.qosmserver
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.rhel6.vip
>       1        ONLINE  ONLINE       rhel6                    STABLE
> ora.rhel7.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.scan1.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> 
在安裝過程中檢查內存使用情況，其實4GB是可以 安裝完成grid的。

[grid@rhel6 bin]$ free -m
total used free shared buff/cache available
Mem: 3789 1339 386 653 2063 1579
Swap: 2047 0 2047

檢查OCR使用量：

ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 4194304 5120 4784 0 4784 0 Y DATA/