3.6 Preparing Oracle VM Server Nodes

An Oracle VM Server can be used as a compute node, and as a Ceph OSD node. This section describes how to prepare an Oracle VM Server as a compute node for OpenStack. Section 4.10.6, “Setting up Oracle VM Server as a Ceph OSD” describes how to prepare an Oracle VM Server as a Ceph OSD node. Section 3.5, “Preparing Oracle Linux Nodes” describes how to prepare an Oracle Linux node.

You can download the installation ISO for Oracle VM Server Release 3.4.2, from the Oracle Software Delivery Cloud at:
https://edelivery.oracle.com/linux

To prepare an Oracle VM Server node:

  1. Install Oracle VM Server for x86 using the instructions in the Oracle VM Installation and Upgrade Guide for Release 3.4.2 at:

    http://docs.oracle.com/cd/E64076_01/E64078/html/vmiug-ovs-install-x86.html

  2. When the operating system is installed, select Alt+F2 to enter the login screen. Log in as the root user.

  3. Increase the dom0 maximum memory allocation to 16GB.

    Any RAM not in use by dom0 is available for use by domUs (virtual machines).

    1. Edit the /etc/default/grub file on the Oracle VM Server and change the dom0_mem parameter to 16384M: 

      GRUB_CMDLINE_XEN="dom0_mem=max:16384M allowsuperpage"

    2. Regenerate the grub.cfg file.

      For example, in a BIOS installation, enter: 

      # grub2-mkconfig -o /boot/grub2/grub.cfg

      In a UEFI installation, enter: 

      # grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

    3. Restart the Oracle VM Server for the changes to take effect: 

      # reboot

  4. Download the latest Oracle Linux Release 6 Yum Server repository file: 

    # curl -L -o /etc/yum.repos.d/public-yum-ol6.repo \
http://yum.oracle.com/public-yum-ol6.repo

  5. Edit the /etc/yum.repos.d/public-yum-ol6.repo file. Enable the following repositories by setting enabled=1 in the following sections:

    • [ol6_addons]

    • [ol6_UEKR4]

    • [ol6_openstack30]

    All other repositories should be disabled by setting enabled=0 in each section.

    This repository contains the required packages for Oracle Linux Release 6, upon which Oracle VM Server is based. This repository also contains the preinstallation packages to set up Oracle OpenStack for Oracle Linux on Oracle VM Server. This repository does not contain updates for Oracle VM Server.

  6. If you access the Yum repositories using a proxy server, update the /etc/yum.conf file with the proxy server address. For more information on using the Oracle Linux Yum Server, see:

    https://docs.oracle.com/cd/E37670_01/E41138/html/ol_yum.html

  7. Use the yum command to check the repository configuration.

    Clean all yum cached files from all enabled repositories. 

    # yum clean all

    List the configured repositories for the system. 

    # yum repolist

  8. The iptables service is not running by default. If you enable the iptables service, you need to make sure all the required ports are open. You can find a list of the required ports by issuing the following command from the master node (after setting up all the required kollacli deployment options). 

    # kollacli property list | grep port

    For more information on using iptables, see the Oracle Linux Administrator's Guide for Release 6:

    https://docs.oracle.com/cd/E37670_01/E41138/html/ol_firewall_sec.html

  9. Install the Oracle OpenStack for Oracle Linux preinstallation package from the Oracle Yum Server: 

    # yum install openstack-kolla-preinstall

  10. If you are using a web proxy, configure the docker service to use the proxy.

    1. Add the following content to the /etc/sysconfig/docker file: 

      HTTP_PROXY=proxy_URL:port                              
http_proxy=$HTTP_PROXY
HTTPS_PROXY=$HTTP_PROXY
https_proxy=$HTTP_PROXY

      Replace proxy_URL and port with the appropriate URLs and port numbers for your web proxy.

      You may also want to specify that local connections do not need to be proxied by setting the NO_PROXY environment variable: 

      NO_PROXY=localhost,127.0.0.1
no_proxy=localhost,127.0.0.1

      Include a line to export these environment variables: 

      export HTTP_PROXY HTTPS_PROXY http_proxy https_proxy NO_PROXY no_proxy

    2. Restart the docker service: 

      # service docker restart

  11. Create a partition for Docker.

    This partition must be at least 64 GB in size. The partition can be created in a number of ways, depending on your hardware and storage configuration.

    For simplicity, this example uses an Oracle VM Server installation on a single disk, accepting all the partitioning defaults in the Oracle VM Server installer. Your installation and configuration may vary.

    Example 3.1 In a UEFI-based installation:

    1. Start the parted utility: 

      # parted
GNU Parted 2.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.

    2. List the existing partitions: 

      (parted) print                                                            
...
Number Start   End     Size     File system     Name  Flags
1      ...     ...     ...      fat16                 boot
2      ...     ...     ...      ext4
3      ...     ...     ...      ext4
4      ...     ...     ...      linux-swap(v1)
5      ...     ...     ...

    3. Remove the last partition: 

      (parted)  rm 5                                             
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sda
(Device or resource busy).  As a result, it may not reflect all of your changes
until after reboot.

    4. Create a new partition of at least 64 GB for Docker, for example: 

      (parted) mkpart docker 100GB 164GB
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sda
(Device or resource busy).  As a result, it may not reflect all of your changes
until after reboot.

    5. Create a partition with the remaining space for the Oracle VM Server local storage, for example: 

      (parted) mkpart storage 164GB 1200GB                                        
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sda
(Device or resource busy).  As a result, it may not reflect all of your changes
until after reboot.

    6. The results should look something like this: 

      (parted) print                                                            
...
Number Start   End   Size     File system     Name      Flags
1      ...     ...   ...      fat16                      boot
2      ...     ...   ...      ext4
3      ...     ...   ...      ext4
4      ...     ...   ...      linux-swap(v1)
5      100GB  160GB  64GB                     docker 
6      160GB   ...   ...                      storage

    7. Reboot the Oracle VM Server: 

      # reboot

    Example 3.2 In a BIOS-based installation:

    1. List the partitions using: 

      # fdisk -l
...
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *         ...         ...         ...   83  Linux
/dev/sda2             ...         ...         ...   83  Linux
/dev/sda3             ...         ...         ...   82  Linux swap / Solaris
/dev/sda4             ...         ...         ...   83  Linux
...

      As shown in this output, there are four primary partitions. A BIOS-based installation is restricted to four primary partitions. To create another partition for Oracle OpenStack for Oracle Linux, delete the last partition, and create a new extended partition, which can include a new logical partition for the Docker images.

    2. Start the fdisk utility. 

      # fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

    3. Remove the last partition created by the Oracle VM Server installer, this case it is the fourth partition: 

      Command (m for help): d
Partition number (1-4): 4

    4. Write the changes to the partition table and exit: 

      Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

    5. Reboot the Oracle VM Server: 

      # reboot

    6. Create an extended partition for the Oracle VM Server local storage: 

      # fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
e
Selected partition 4
First cylinder (...-..., default ...): 
Using default value ...
Last cylinder, +cylinders or +size{K,M,G} (...-..., default ...): 
Using default value ...

    7. Create a logical partition for the Docker images, with a size of at least 64 GB: 

      Command (m for help): n
First cylinder (...-..., default ...): 
Using default value ...
Last cylinder, +cylinders or +size{K,M,G} (...-..., default ...): +64G

    8. The results should look something like this: 

      Command (m for help): p
...
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         ...      ...      83  Linux
/dev/sda2             ...         ...      ...      83  Linux
/dev/sda3             ...         ...      ...      82  Linux swap / Solaris
/dev/sda4             ...         ...      ...       5  Extended
/dev/sda5             ...         ...      ...      83  Linux

    9. Write the changes to the partition table and exit: 

      Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

    10. Reboot the Oracle VM Server: 

      # reboot

  12. Stop the docker service: 

    # service docker stop

  13. Create either a btrfs or an ext4 file system on the partition you created for Docker, mounted on /var/lib/docker.

    The following example formats an existing partition on /dev/sda5, and mounts it on /var/lib/docker using the /etc/fstab file.

    1. Create the file system on an existing device:

      To create a btrfs file system: 

      # mkfs.btrfs -f [-L label] /dev/sda5

      To create an ext4 file system: 

      # mkfs.ext4 [-L label] /dev/sda5

      where -L label is an optional label that can be used to mount the file system.

      More information on using mkfs.btrfs is available in the Oracle Linux Administrator's Guide for Release 6 at:

      https://docs.oracle.com/cd/E37670_01/E41138/html/ol_create_btrfs.html

    2. Use the blkid command to display the UUID of the device and make a note of this value, for example: 

      # blkid /dev/sda5
/dev/sda5: LABEL="docker" UUID="bb6912ae-e6c4-4ffd-bcdc-37c8c19eb7db" 
UUID_SUB="599202a1-fb2d-4352-916f-aee4d034d3bd" TYPE="btrfs"

    3. Edit the /etc/fstab file and add an entry to ensure the file system is mounted when the system boots: 

      UUID=bb6912ae-e6c4-4ffd-bcdc-37c8c19eb7db /var/lib/docker  btrfs  defaults  1 2

      If you created a label for the file system, you can also use the label instead of the UUID, for example: 

      LABEL=label /var/lib/docker  ext4  defaults  1 2

    4. Mount all the file systems listed in /etc/fstab: 

      # mount -a

    5. You can verify the partition is mounted using: 

      # df
Filesystem     1K-blocks    Used Available Use% Mounted on
...
/dev/sda5            ...    ...  ...       1%   /var/lib/docker

  14. Start the docker service: 

    # service docker start

  15. Ensure that Docker is using the correct storage driver, either btrfs or overlay2.

    Docker may not use the correct storage driver for the file system mounted on /var/lib/docker when it starts. Use the docker info command to check the current storage driver.

    For a btrfs file system, the storage driver must be btrfs, for example: 

    # docker info | grep -A 1 Storage
Storage Driver: btrfs
 Build Version: Btrfs v4.2.2

    For an ext4 file system, the storage driver must be overlay2, for example: 

    # docker info | grep -A 1 Storage
Storage Driver: overlay2
 Backing Filesystem: extfs

    If the storage driver is incorrect, configure Docker to use the correct driver, as follows:

    1. Edit the /etc/sysconfig/docker file and add a --storage-driver option to the other_args variable.

      For example: 

      other_args='--storage-driver=driver'

      where driver is either btrfs or overlay2.

    2. Restart the docker service. 

      # service docker restart

    3. Check the correct driver is now loaded.

  16. Synchronize the host time with the controller nodes to avoid errors during OpenStack operations. Time synchronization is discussed in more detail in Section 3.5, “Preparing Oracle Linux Nodes”.

    1. To synchronize the host with the controller nodes, edit the /etc/ntp.conf file to configure the ntpd service. Add the controller nodes to the file, for example: 

      server control1.example.com
server control2.example.com

    2. Create the drift file, which is used to store the system's clock drift value. 

      # touch /var/lib/ntp/drift

      Start the ntpd service and configure it to start following a system reboot. 

      # service ntpd start
# chkconfig ntpd on

    3. You can check the ntpd service is configured and synchronizing correctly using the ntpq and ntpstat commands, for example: 

      # ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*control1       10.169.64.1      3 u   58   64  377    0.132   33.490  22.604
 control2       .INIT.          16 u    -  128    0    0.000    0.000   0.000
# ntpstat
synchronised to NTP server (10.196.134.1) at stratum 4 
   time correct to within 120 ms
   polling server every 64 s

Copyright © 2016, 2017 Oracle and/or its affiliates. All rights reserved. Legal Notices