Storage Gateway Installation
Install the Storage Gateway software on your computer.
Prerequisites
These instructions assume that you are familiar with the administration and configuration commands of the operating system on your host machine. To install Storage Gateway, your host system must meet certain hardware and software requirements.
Hardware Recommendations and Requirements
To run Storage Gateway, the host machine must meet the following requirements:
-
Two dual-core CPUs or better. Oracle recommends 4-core CPUs.
-
Minimum memory requirements:
-
16 GB for required for any Storage Gateway file system.
-
32 GB for file systems up to 50 million files.
-
64 GB for file systems up to 100 million files.
-
-
The recommended local storage disk size is 600 GB, which includes 500 GB for the file system cache, 80 GB for metadata storage, and 20 GB for log storage.
Important
-
Provision local storage before installing Storage Gateway. For best performance, allocate dedicated local storage file systems for the Storage Gateway cache, the metadata, and the logs. The installation script prompts you for the paths to your Storage Gateway file system cache, metadata storage, and log storage locations. Follow the disk size recommendations provided by the installer.
-
Oracle recommends that you use the XFS file system for the file system cache, metadata, and logs. XFS is a 64-bit file system designed for parallel I/O. Parallel I/O allows a system to scale based on the number of I/O threads and file system bandwidth.
-
Software Requirements
The Storage Gateway installation software automatically installs Docker and the NFS protocol.
If you create an Oracle Cloud Infrastructure compute instance to host Storage Gateway, the instance creation wizard provides an option to choose the operating system image.
Here are the Storage Gateway software requirements:
-
Oracle Linux 7.
Note
Oracle Linux 8 is not supported.
-
Docker 1.12.6 or newer. Docker is an open platform for building, shipping, and running distributed applications. For more information, see https://www.docker.com/.
- NFSv4.
Downloading Storage Gateway
You install Storage Gateway by first going to the Storage Gateway
Download page. Click the Download link for all
supported platforms. This page contains a license agreement that you must accept. Then
click the link to download a compressed archive file (approximately 1 GB) to your local
host. Here you can use the SFTP tool of your choice to copy the tar archive into the
/tmp
folder of the host machine.
The Oracle Cloud Infrastructure Storage Gateway download includes:
- An installer script to install the Storage Gateway.
- A readme file with installation information and a configuration overview.
See Installing Storage Gateway for complete installation instructions.
Installing Storage Gateway
You can install Storage Gateway on an Oracle Cloud Infrastructure compute instance or an on-premises host that meets the hardware and software requirements.
To install Storage Gateway:
-
Connect to your compute instance or on-premises host.
For help with connecting to an Oracle Cloud Infrastructure compute instance, see Connecting to an Instance.
-
If your host volume is new, you might need to format and mount the disk.
Tip
This task describes the simplest way to create a functional file system to host a Storage Gateway. It uses one device and file system to host the cache, metadata, and log volumes. You specify the paths to those volumes later in this procedure. To optimize performance for your system, you can:
-
Create a separate device and file system for each of the cache, metadata, and log volumes.
-
Create a single device, but create logical volumes and file systems for the cache, metadata, and log volumes.
To format the disk and create a file system:
-
Run the following command:
sudo fdisk /dev/sdb
(Optional) Press m to view the
fdisk
options. -
Choose command g - create a new empty GPT partition table.
-
Choose command w - write table to disk and exit.
-
Create an XFS (file system).
(Optional) To make the file system volume extensive, Oracle recommends using LVM to create logical partitions that can span across one or more physical hard drives. First, the hard drives are divided into physical volumes. Next, those physical volumes are combined to create the volume group. Finally, the logical volumes are created from volume group. Follow these steps:
-
Create a volume group.
sudo vgcreate volume_group device
-
Create a logical volume.
sudo lvcreate -l 100%FREE -n logical_volume volume_group
-
Create an XFS file system.
sudo mkfs.xfs /dev/volume_group/logical_volume
If you do not want to use LVM to create logical partitions to span across your physical hard drives, skip these steps and proceed to the next section.
-
To mount the formatted volume:
This example uses
/ocisg
as the mount directory. You can specify a different mount directory as needed.-
Create a mount directory:
sudo mkdir /ocisg
-
Mount the drive:
-
When using LVM:
sudo mount /dev//volume_group/logical_volume /ocisg
-
For not using LVM:
sudo mount /dev/sdb /ocisg
-
-
-
Download the Storage Gateway tar archive.
-
Use the SFTP tool of your choice to copy the tar archive into the
/tmp
folder of the host machine. -
On the host machine, change directory to
/tmp
and extract the files from the tar archive:cd /tmp sudo tar xvzf ocisg-version.tar.gz
This command extracts the files from the tar archive into a subdirectory named
ocisg-version
, whereversion
is the Storage Gateway version to which you are installing. -
Change directory to
ocisg-version
and run the installation script assudo
orroot
user:cd ocisg-version sudo ./ocisg-install.sh
Optionally, you can specify the following
ocisg-install.sh
script flags:-
-a Runs the installation in advanced configuration mode, which lets you specify ports and the Docker network mode.
In addition to prompting you for the paths to the metadata storage, cache storage, and log storage, advanced configuration mode also prompts you for:
-
The Docker network mode (host or bridge).
Bridge mode is the default.
Oracle recommends host mode because it improves network performance. If you encounter issues with host mode, try using bridge mode. If switching to bridge mode does not help, contact My Oracle Support.
-
The host port to use for the management console.
-
The host port to use for NFS access.
Tip
For each host port specification, you can designate a port or press Enter to let Storage Gateway dynamically allocate the port. You can use the
ocisg configure port
command to change the ports later. See Getting Started with Storage Gateway for details. -
-
-d Installs Storage Gateway at the location that you specify instead of the default location of
/opt/ocisg
. For example:sudo ./ocisg-install.sh -d /opt/storagegateway
-
-h Displays the installation script help information.
-
-p Specifies that Storage Gateway is running behind a proxy server. You can specify multiple proxy arguments. For example:
./ocisg-install.sh -p http://myproxy.com:80 -p https://mysecureproxy.com:80
-
-q Runs the installation in quiet mode.
If you supply the paths to the Storage Gateway cache, metadata, and log storage locations using -m path_to_metadata_storage, -c path_to_cache_storage, and -l path_to_log_storage, you are not prompted for input. For example:
sudo ./ocisg-install.sh -q -m /ocisg/metadata -c /ocisg/cache -l /ocisg/log
Note
Ignore the
devicemapper
warning message if it appears during the installation.
-
The script guides you through the Storage Gateway installation. Depending on your host machine configuration, some steps can require your input:
-
Docker does not appear to be installed. Do you want to install docker engine with yum? [y/N]
Press y, and then press Enter.
The installation script automatically installs Docker and configures the storage driver for use with Storage Gateway.
Important
If Docker is already installed on your system, the installation script does not automatically configure the storage driver and returns a warning message:
Checking that docker is installed and using the correct version Found docker version Docker version 18.03.1-ol, build 0d51d18 The storage appliance requires to set devicemapper as the docker storage driver. Please follow the setup link below to enable devicemapper and rerun the install.
Manually verify and update the Docker storage driver to be
devicemapper
as required. See Verifying and Updating the Storage Driver in Docker. - When prompted, press Enter accept the default installation location.
-
When prompted, specify the paths to your Storage Gateway cache, metadata, and log storage locations.
The following examples represent paths for a simple system. Your setup might include paths to separate devices and file systems for each location.
-
Enter a path for the file system cache. For example:
/ocisg/sg/cache
-
Enter the path for metadata storage. For example:
/ocisg/sg/metadata
-
Enter the path for log storage. For example:
/ocisg/sg/log
Note
The Storage Gateway installer expects three separate directories on the same volume or three different volumes for each cache, metadata, and log locations.
If you receive warnings about cache, metadata, and log storage existing on the same volume, enter y to proceed with the installation.
-
-
After a successful installation, the script provides the following information:
Starting OCISG [ocisg:version]
Setting up config file port with nfs
Setting up config file port with admin
Setting up config file port with rest
Management Console: https://gv-sg:32770
If you have already configured an OCISG FileSystem via the Management Console,
you can access the NFS share using the following port.
NFS Port: 32771
Security List Requirements for Compute Instance Installations
If you installed Storage Gateway on an Oracle Cloud Infrastructure compute instance, that instance must be able to receive HTTPS connections from other hosts and allow communication with the Storage Gateway management console. To open the necessary port, add an ingress rule to the security list governing the instance's host subnet. To learn about VCN security control, see Security Lists.
Only use open network port access to networks that you trust. Oracle strongly recommends that you do not open network ports to the public internet. Instead, use a private connection to the machine hosting the Storage Gateway management console, for example a VPN or SSH local forward tunnel. See Site-to-Site VPN for more information.
Use the following syntax for SSH local forward tunnel:
ssh -L localHost:localPort:remoteHost:remotePort remoteHost
See https://www.ssh.com/ssh/tunneling/example#local-forwarding for more information.
Hosting Storage Gateway on a Compute Instance
To host Storage Gateway on an Oracle Cloud Infrastructure compute instance, you need:
-
An SSH key pair in PEM format.
-
To create a key pair, see Managing Key Pairs on Linux Instances.
-
If your public key is not in PEM format, use the following command to convert it:
ssh-keygen -f key_name.pub -e -m pem
-
-
An Oracle Cloud Infrastructure user account with an API signing key (the public key from your SSH key pair).
-
If you need to create a user account, see Managing Users.
-
To upload an API signing key to an existing user account, see Managing User Credentials.
-
-
A virtual cloud network (VCN) and related resources. For help with creating a VCN, see VCNs and Subnets.
The following configuration points apply to your VCN:
-
Do not select the Use DNS Hostnames in this VCN check box unless you plan to use DNS hostnames for your Storage Gateway compute instance.
-
After you install the Storage Gateway software your host machine, you must add a security list rule to allow communication with the management console port. More information appears on this page after the Storage Gateway installation instructions.
-
-
A compute instance. See Creating an Instance.
The VM.Standard2.4 compute shape meets the minimum required specifications for Storage Gateway. Large file systems might require an image with more resources.
-
A Block Volume. See Creating a Volume.
-
The recommended minimum disk size is 600 GB. See Configuring the Cache for File Systems for more information.
-
Attach the volume to your compute instance. See Attaching a Volume.
-
If you specify iSCSI as the volume attachment type, you must also connect and mount the volume from the instance for the volume to be usable. For more information, see Overview of Block Volume and Connecting to a Volume.
-
Adding a Security List Rule to Storage Gateway
-
Open the navigation menu, click Networking, and then click Virtual Cloud Networks.
-
Click the name of the cloud network (VCN) that hosts your compute instance.
-
Click Security Lists.
-
Click the name of the security list that governs the subnet hosting your compute instance.
-
Click Edit All Rules.
-
Add an ingress rule:
-
Leave Stateless unmarked.
-
Select TCP as the IP Protocol.
-
Select CIDR for the Source Type and enter a source CIDR block for your on-premises network or the CIDR block where you access the Storage Gateway management console and NFS port. Oracle strongly recommends limiting the rule to the most specific criteria as possible and not opening the rule to all networks.
-
Enter All in the Source Port Range field.
-
Specify your Storage Gateway management console port and NFS port in the Destination Port Range field. For example:
32769
If you do not know the management console port and NFS port for your Storage Gateway installation, run the following command on the host machine:
sudo ocisg info
The management console port appears at the end of the management console URL:
Management Console: https://exampleCompute:32769
The NFS port appears as:
NFS Port: 32770
-
-
Click Save Security List Rules.
You can now connect to the compute instance using the public IP address. See Launching a Linux Instance for more information.
Verifying and Updating the Storage Driver in Docker
To verify the storage driver in Docker:
-
Start Docker:
sudo systemctl start docker
-
Verify the information in Docker:
sudo docker info
-
Look for
Storage Driver
in the output. For example:Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 18.03.1-ol Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: false Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88 runc version: 4fc53a81fb7c994640722ac585fa9ca548971871 init version: 949e6fa Security Options: seccomp Profile: default selinux Kernel Version: 4.1.12-124.15.4.el7uek.x86_64 Operating System: Oracle Linux Server 7.5 OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 13.45GiB Name: ocisg-mahesh ID: OJ2H:QUSK:BWQZ:25L6:VI5V:CXGX:WFXT:NNNP:RK6O:OS4P:4ABE:JWMV Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries: docker.io (secure)
Note
Ignore thedevicemapper
warning message if it appears.If
Storage Driver
is notdevicemapper
, do the following:-
Stop Docker:
sudo systemctl stop docker
-
Look for
/etc/docker/daemon.json
in the host.If the file
daemon.json
does not exist, create it. -
In the
daemon.json
file, set thestorage-driver
variable todevicemapper
:{ "storage-driver": "devicemapper" }
-
Restart Docker:
sudo systemctl start docker
-
Verify the information in Docker:
sudo docker info
Look for
Storage Driver
in the output and verify that the storage driver isdevicemapper
.
-