Preparing OS for Installing Oracle 11gR1 64bit RDBMS on RHEL 4 UPD4

Regarding "Preparing OS for Installing Oracle 11gR1 32bit RDBMS on OEL 5 & RHEL 5" post, I've attempted the same process on 64Bit so as to take advantage of an existing Linux installation.

I've followed Metalink notes:
437123.1 (Requirements for Installing Oracle 11gR1 RDBMS on RHEL 4 on AMD64/EM64T) Revision 05-NOV-2007
169706.1 Revision 30-JAN-2008

I’ve started the installation on an IBM eServer xSeries x3650 with 2048MB and 2 Intel(R) Xeon(R) 5160 @ 3.00GHz, CPUs. The box has installed a default Red Hat Enterprise Linux ES release 4 (Nahant Update 4) (Linux 2.6.9-42.ELsmp #1 SMP x86_64 x86_64 x86_64 GNU/Linux)

This procedure is meant for those planning/trying to install Oracle Server 11.1.0.x.0 on Redhat 4 on the x86_64 (64-bit) platform. Since it is the expressed goal to keep Oracle Enterprise Linux functionally IDENTICAL to RHEL, this NOTE is also completely applicable to OEL 4 (update 4 or higher).

Info:
Automatic Memory Management.
Starting with Oracle Database 11g, the Automatic Memory Management feature requires more shared memory (/dev/shm) and file descriptors. The shared memory should be sized to be at least the greater of MEMORY_MAX_TARGET and MEMORY_TARGET for each Oracle instance on the computer. To determine the amount of shared memory available, enter the following command: # df -k /dev/shm/
In our case:

[root@xxxxx ~]# df -k /dev/shm/
S.ficheros Bloques de 1K Usado Dispon Uso% Montado en
none 1028096 0 1028096 0% /dev/shm

Note: MEMORY_MAX_TARGET and MEMORY_TARGET cannot be used when LOCK_SGA is enabled or with huge pages on Linux

Required RPMS
I've found a different list of required RPMs on both notes (437123.1, 169706.1)
Some RPMs where placed on system with the OS installation. Next graphic shows a detailed list and result:

TABLE 1 (click for zoom it)

Note: i386 packages might require the --force option during installation if the 64-bit version of the same package is already installed. For example, "rpm -ivh --force and could be found on both Red Hat updates (U4 & U5)

A list of required RPMs:
binutils-2.15.92.0.2
gcc-3.4.5
gcc-c++-3.4.5
gcc-3.4.6-8.i386.rpm
cpp-3.4.6-8.i386.rpm
libgcc-3.4.5 (32-bit)
libgcc-3.4.5
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.97
elfutils-libelf-devel-0.97
glibc-2.3.4-2.19
glibc-2.3.4-2.19 (32 bit)
glibc-common-2.3.4
glibc-devel-2.3.4
glibc-devel-2.3.4 (32-bit)
libaio-0.3.105
libaio-0.3.105 (32 bit)
libaio-devel-0.3.105
libstdc++-3.4.5
libstdc++-3.4.5 (32 bit)
libstdc++-devel 3.4.5
make-3.80
sysstat-5.0.5
glibc-headers-2.3.4-2.19
glibc-kernheaders-2.4-9.1.98.EL

*see Table 1

Note: to distinguish between a 32-bit or 64-bit package
i.e. #rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" | grep libaio-0.3

Setting Kernel parameters and environment.

* Required Environment Variable

umask for the oracle account must be 0022
the "ulimit" stack size value should be at least 32768 (ulimit -s 32768)

* Modify your kernel settings in /etc/sysctl.conf (RedHat) as follows. If the current value for any parameter is higher than the value listed in this table, do not change the value of that parameter. Range values (such as net.ipv4.ip_local_port_range) must match exactly.

kernel.shmall = 2097152
kernel.shmmax = 1/2 of physical RAM. This would be the value 2147483648 for a 4Gb RAM system.
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 512 x processes (for example 65536 for 128 processes)
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 262144

Local values:
# Oracle
kernel.shmall = 2097152
kernel.shmmax = 1052770304
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 76800
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 262144

Set Shell Limits for the oracle User. Assuming that the "oracle" Unix user will perform the installation, do the following:

- Add the following settings to /etc/security/limits.conf

oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536

- Add or edit the following line in the /etc/pam.d/login file, if it does not already exist:
session required pam_limits.so

- Add the following lines to /etc/profile:

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

Note: User settings are best described on "Preparing OS for Installing Oracle 11gR1 32bit RDBMS on OEL 5 & RHEL 5"

Installation note:

I’ve found an OS issue, launching runinstaller from linux.x64_11gR1_database.zip binaries.

./runInstaller

"You are attempting to install 64-bit Oracle on a 32-bit operating system. This is not supported and will not work."

It’s not true due the actual OS

[oracle@xxxxx bin]$ uname -mi

x86_64 x86_64

Problem, getconf shows 32 bit on a 64 bit system (maybe it's concerning Intel Extended Memory 64 Technology (Intel Intel64) that enables servers based on Intel Xeon processors to run both 32-bit and 64-bit applications under 64-bit operating systems):

[oracle@xxxxx bin]$ getconf LONG_BIT

32

Then, I’ve commented the following lines on “RunInstaller” and continue installation without problems:

#if [ `$UNAME` = "Linux" ]; then

# if [ -e $GETCONF ]; then

# value=`$GETCONF LONG_BIT`

# if [ $value != 64 ]; then

# echo "\"You are attempting to install 64-bit Oracle on a 32-bit operating system. This is not supported and will not work.\"";

# exit 0;

# fi

# fi

#fi

References:

Oracle Metallink Note: 437123.1
Oracle Metallink Note: 169706.1

Preparing OS for Installing Oracle 11gR1 32bit RDBMS on OEL 5 & RHEL 5

Everybody knows second releases are better, however I’ll try 11gR1 (Oracle 11gR1 (11.1.0.x) 32-bit on Enterprise Linux 5 x86 Platform) in order to review Oracle 11g top new features and the whole installation process.
Since it is the expressed goal to keep Oracle Enterprise Linux functionally IDENTICAL to RHEL, this NOTE is also completely applicable to RHEL 5 (any update level)

I’ve started with a VMware Server Console 1.0.1 build-29996 on Windows XP Professional 5.1.2600, Service Pack 2 with 1024MB for test purposes.

I’m using an Enterprise Linux 5 virtual machine with 512MB and 1GB swap (Linux version 2.6.18-8.el5 (mockbuild@ca-build14) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP) and I’ve made a default OS installation (without customization during installation).

Required RPMS

compat-libstdc++-33-3.2.3-61
elfutils-libelf-devel-0.125 (requires elfutils-libelf-devel-static-0.125)
glibc-devel-2.5-12 (requires glibc-headers-2.5-12)
gcc-4.1.1-52 (requires libgomp-4.1.1-52)
gcc-c++-4.1.1-52
libaio-devel-0.3.106
libstdc++-devel-4.1.1-52.e15
sysstat-7.0.0
unixODBC-2.2.11
unixODBC-devel-2.2.11

Dependent RPM's:
elfutils-libelf-devel-static-0.125-3.el5
glibc-headers-2.5-12
libgomp-4.1.1-52.el5.2

Other required RPMS's (as per Install Guide, release Notes)
(these get installed during a "Default RPM" installation of the RHEL AS/ES 5 OS. Additional RPMs maybe needed if a "less-than-default-RPMs" installation of RHEL AS/ES 5 is performed.

binutils-2.17.50.0.6-2.el5
elfutils-libelf-0.125-3.el5
glibc-2.5-12
glibc-common-2.5-12
libaio-0.3.106
libgcc-4.1.1-52
libstdc++-4.1.1
make-3.81-1.1

The easy (manual) way to install required RPMS and their dependencies

a) Make a list

[root@localhost ~]# cat rpms.lst
compat-libstdc++
elfutils-libelf-devel-static
elfutils-libelf-devel
glibc-headers
glibc-devel
libgomp
gcc
gcc-c++
libaio-devel
libstdc++-devel
sysstat
unixODBC
unixODBC-devel

b) Find missing packages and copy them to a temporary folder (manually mounting all OS installations CDs)

[root@localhost ~]mount /dev/cdrom /mnt/
[root@localhost ~]# for i in `cat rpms.lst`;do find /mnt/ -name "*$i*" -exec cp {} rpm/ \;;done
[root@localhost ~]umount /mnt/

c) Install RPMS

rpm -ivh elfutils-libelf-devel-0.125-3.el5.i386.rpm elfutils-libelf-devel-static-0.125-3.el5.i386.rpm

rpm -ivh glibc-devel-2.5-12.i386.rpm glibc-headers-2.5-12.i386.rpm gcc-4.1.1-52.el5.i386.rpm libgomp-4.1.1-52.el5.i386.rpm

rpm -ivh compat-libstdc++-296-2.96-138.i386.rpm compat-libstdc++-33-3.2.3-61.i386.rpm

rpm -ivh libstdc++-devel-4.1.1-52.el5.i386.rpm

rpm -ivh gcc-c++-4.1.1-52.el5.i386.rpm

rpm -ivh libaio-devel-0.3.106-3.2.i386.rpm

rpm -ivh sysstat-7.0.0-3.el5.i386.rpm

rpm -ivh unixODBC-2.2.11-7.1.i386.rpm
unixODBC-devel-2.2.11-7.1.i386.rpm

d) Check

[root@localhost ~]# for i in `cat rpms.lst`;do echo "checking $i"; rpm -qa|grep $i;echo "-----------------";done

Setting Kernel parameters

[root@localhost ~]# vi /etc/sysctl.conf

kernel.shmall = 2097152
kernel.shmmax = 1/2 of physical RAM. This would be the value 2147483648 for a 4Gb RAM system. (268435456 in my case)
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 512 * PROCESSES (for example 65536 for 128 processes)
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 262144

[root@localhost ~]# sysctl –p

Adding user Oracle

# useradd oracle
# passwd oracle
# groupadd dba
# groupadd oinstall
# usermod -g oinstall -G dba oracle

Configure Oracle software owner limits

[root@el5 ~]# vi /etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536

[root@el5 ~]# vi /etc/pam.d/login
session required /lib/security/pam_limits.so
session required pam_limits.so

[root@el5 ~]# vi /etc/profile

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

Miscelaneous

In order to avoid errors OUI-10035 and OUI-10033 during installation, take into account Oracle Metalink Note 452780.1 what in a brief, explains that regarding new oraInventory location’s change on 11gR1, Oracle user, needs write permission on $ORACLE_BASE/../ ($ORACLE_BASE/../oraInventory)


If you are installing Oracle RDBMS 11gR1 software onto either the RHEL 5 or the OEL 5 operating system, please set the Linux feature "SELinux" to "Permissive" mode. This issue is described more fully in Metalink Note 454196.1, "./sqlplus: error on libnnz11.so: cannot restore segment prot after reloc".

It talks about issuing the following commands and/or switch SELinux from the default "Enforcing" mode that it is running in, to the "Permissive" mode.

/usr/sbin/semanage fcontext -a -t textrel_shlib_t $ORACLE_HOME/lib/libnnz11.so
/usr/sbin/semanage fcontext -a -t textrel_shlib_t $ORACLE_HOME/lib/libnque11.so
/usr/sbin/semanage fcontext -a -t textrel_shlib_t $ORACLE_HOME/lib/libclntsh.so.11.1

In my case (test scenario) I’ve decided to disable selinux before installation.

[root@el5 ~]# vi /etc/selinux/config

[root@el5 ~]# id oracle

uid=500(oracle) gid=502(oinstall) groups=502(oinstall),501(dba) context=root:system_r:unconfined_t:SystemLow-SystemHigh

After reboot:
[root@el5 ~]# id oracle
uid=500(oracle) gid=502(oinstall) groups=502(oinstall),501(dba)

Login as Oracle software owner:

If the ORACLE_SID, ORACLE_HOME or TNS_ADMIN environment variable is set, then remove/unset them from the environment.
Unset following environment variables (if set) LD_LIBRARY_PATH
Java References (like JAVA_HOME..etc)
Oracle Home references in $PATH

set 'umask 0022' and ulimit stack size to 32768

umask 0022
ulimit -s 32768


Ready to install Oracle 11gR1 32bit RDBMS


References:
Oracle Metallink Note: 438765.1
Oracle Metallink Note: 452780.1
Oracle Metallink Note: 454196.1