Note:
- This tutorial is available in an Oracle-provided free lab environment.
- It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.
Use a Container to Create a DNF or ULN Repository Mirror
Introduction
Oracle Linux includes a fully functional reposync tool for DNF, making creating a mirror of any yum repository easy. We can also extend this facility to mirror ULN channels for environments where the majority of your systems do not have direct access to the internet. By creating a yum mirror of your organization’s yum repositories and ULN channels, you can reduce network overhead and improve yum performance across your environment. Yum mirrors are also useful if you are configuring other services for your environment, such as offline Ksplice.
While previous releases of Oracle Linux included a uln-yum-mirror
package that you could use to perform mirroring services, this was not particularly efficient and was relatively complicated to set up.
A ULN or yum mirror service is a typical example of a service that is best run within a set of containers. Using Podman, you can quickly and easily deploy a container that uses the Oracle Linux slim image to handle scheduled synchronization of the yum repositories or ULN channels that you use within your organization. You can also deploy a container that handles the provisioning of the mirrored repositories within a web service that client systems are able to access.
An opensource GitHub project provides the Dockerfiles, scripts, and instructions to set up this kind of service at https://github.com/Djelibeybi/oraclelinux-reposync.
Objectives
In this tutorial, you’ll learn to:
- Install container-tools
- Build a container with the required software
- Create a mirror service using reposync within the container
Prerequisites
-
Minimum of a single Oracle Linux system
-
Each system should have Oracle Linux installed and configured with:
- A non-root user account with sudo access
- Access to the Internet
If you are using Oracle Linux 7, substitute these dnf command instructions with yum, and use Docker to perform those tasks referencing podman or buildah.
Deploy Oracle Linux
Note: If running in your own tenancy, read the linux-virt-labs
GitHub project README.md and complete the prerequisites before deploying the lab environment.
-
Open a terminal on the Luna Desktop.
-
Clone the
linux-virt-labs
GitHub project.git clone https://github.com/oracle-devrel/linux-virt-labs.git
-
Change into the working directory.
cd linux-virt-labs/ol
-
Install the required collections.
ansible-galaxy collection install -r requirements.yml
-
Deploy the lab environment.
ansible-playbook create_instance.yml -e localhost_python_interpreter="/usr/bin/python3.6"
The free lab environment requires the extra variable
local_python_interpreter
, which setsansible_python_interpreter
for plays running on localhost. This variable is needed because the environment installs the RPM package for the Oracle Cloud Infrastructure SDK for Python, located under the python3.6 modules.The default deployment shape uses the AMD CPU and Oracle Linux 8. To use an Intel CPU or Oracle Linux 9, add
-e instance_shape="VM.Standard3.Flex"
or-e os_version="9"
to the deployment command.Important: Wait for the playbook to run successfully and reach the pause task. At this stage of the playbook, the installation of Oracle Linux is complete, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys and any other deployment information needed while running the lab.
Install Required Packages
Open a terminal and connect via SSH to the ol-node-01 instance:
ssh oracle@<ip_address_of_instance>
Install Git so that you can clone the container-reposync repository:
sudo dnf install -y git
Install podman and related utilities:
Oracle Linux 8:
sudo dnf module install -y container-tools:ol8
Oracle Linux 9:
sudo dnf install -y container-tools
Note: If you are using Oracle Linux 7, the buildah and skopeo packages are only available via the unsupported EPEL repositories. For this reason, you may choose to use Docker for the purpose of building and running your container images. More details on using Docker are found in the opensource project README.md.
Clone the container-reposync Repository
Clone the repository:
git clone https://github.com/Djelibeybi/oraclelinux-reposync.git
cd oraclelinux-reposync
Build the Images
Build the two container images included in the repo:
buildah build-using-dockerfile -t ol-repo-sync .
buildah build-using-dockerfile -t ol-repo-web -f Dockerfile.nginx .
Verify the build:
podman images
Create Additional Storage Directories
You need to create two additional directories for the container-reposync service to function correctly:
rhn
: If you intend to mirror ULN channels, you must create a directory to store your ULN registration information to make it persistent across subsequent container restarts. You can map this directory into your container whenever you run it so that it has access to ULN registration data.repo
: You must create a directory to store all the packages and metadata you mirror from ULN or the yum server. The file system hosting this based directory needs enough available disk space to cater to all of the repositories and channels that you intend to mirror. You may mount dedicated storage to this location or map to an alternate location if required.
The location of these directories can be wherever you like, but we advise you to create them alongside the configuration information for this container. For example:
mkdir rhn repo
Set the Configuration Variables for Reposync
The ol-repo-sync image depends on configuration information stored in various configuration files in the config directory.
config/uln.conf
: This file stores ULN access credentials such as Oracle SSO credentials and an active CSI. You must set the file permissions to restrict access. You do not need this file if you don’t intend to mirror from ULN.config/repo-map.json
: The image uses this file to construct repository URLs and to identify how to access different ULN channels and yum repositories. You should not edit this file, but you can refer to it to see what channel names are available for mirroring. Note that this file may need to be updated from time to time to account for new ULN channels. Furthermore, some entries in this file are only available in ULN. It includes all repos containing the wordsbase
,patch
,ksplice
,JavaSE
, andExadata
. ULN may further restrict these repos to specific CSIs.config/repos.json
: This file identifies which ULN channels or yum repositories you’ll mirror. It contains two sections: the first listing ULN channels and the second listing yum repositories. If you only intend to mirror yum repositories, remove the ULN section entirely. The ULN section should only contain channels not available as repositories on https://yum.oracle.com as it is more efficient to mirror directly from the yum server. The majority of channels are also available on the Oracle Linux yum server, so use this section to list your channels whenever possible. If you add entries to the ULN section, you must have a ULN account and register the container image with ULN.
Note: An Oracle Linux support subscription is required to sync from ULN. If you do not have a support subscription, remove the
uln
array completely from theconfig/repos.json
.
For the best sync performance, use the yum source instead of ULN wherever possible, as yum.oracle.com leverages the Akamai CDN and will almost always have much higher download speeds than ULN.
-
(Optional) If you have an active Oracle Linux support subscription:
cp config/uln.sample.conf config/uln.conf
Replace the placeholders with Oracle SSO credentials and an active CSI. To protect the content of this file, run:
chmod 400 config/uln.conf
This setting prevents anyone except yourself from access.
-
Create a
config/repo-map.json
file by running the following command:podman run --rm -it \ --name ol-repo-sync \ -v ${PWD}/config:/config:z \ -v ${PWD}/repo:/repo:z \ ol-repo-sync update
This command can be run again at any time to update the
config/repo-map.json
file with the latest repo configuration. You should at least run it whenever a new update or major version is released to make the new repos available for syncing. -
Copy
repos.json
cp config/repos.sample.json config/repos.json
Add all the repos you want to sync to the uln or yum array.
Example:
Here is a script that syncs the
Oracle Linux 8 Ksplice aware userspace
packages from ULN and theOracle Linux Automation Manager
packages from yum.oracle.com. If you do not have a ULN account during this lab, remove the entire uln block.echo '{ "uln": [ "ol8_x86_64_userspace_ksplice", "ol8_aarch64_userspace_ksplice" ], "yum": [ "ol8_x86_64_automation" ] }' | tee config/repos.json
(Optional) Register Your Container with ULN
If you do not intend to mirror any channels from ULN, you do not need to register your container. However, if you have entered your ULN credentials into the ULN configuration file, created a directory to contain your ULN registration data, and configured at least one ULN channel in the repo configuration file, you must register the container.
Perform the registration by running the following:
podman run --rm -it \
-v ${PWD}/rhn:/etc/sysconfig/rhn:z \
-v ${PWD}/config:/config:z \
-v ${PWD}/repo:/repo:z \
ol-repo-sync register
Note:* This will take a few minutes without output to the terminal but should return to the command prompt when completed. Podman maps the
rhn
andconfig
directories from the current working directory into the container. You only need to perform container registration once so long as the rhn directory is mapped to/etc/sysconfig/rhn
for each subsequent time you run the container.
Populate Your Mirror Repository
To populate the mirror repository with packages from the configured repositories and channels, run the following:
podman run --rm -it \
-v ${PWD}/rhn:/etc/sysconfig/rhn \
-v ${PWD}/config:/config \
-v ${PWD}/repo:/repo:z \
ol-repo-sync
The container automatically adds and subscribes to each channel configured in config/repos.json
and creates an identical hierarchy to that used by the Oracle Linux yum server.
You can schedule this command to run on a recurring schedule using a cronjob or systemd timer.
Note: This step takes a while as the packages are downloaded locally to your system.
Serve the Local Yum Mirror to Client Systems
Use the ol-repo-web container image to serve the yum repositories to your client systems. This container can run permanently, and you can configure it to start at boot:
podman run --detach --name yum-server \
-p 8080:80 \
-v ${PWD}/repo:/var/www/html/repo:ro \
ol-repo-web
Note: The repo directory is mapped into the yum-server container with read-only permissions, allowing the container to continue running and serving clients while the mirrored repositories and channels are updated.
Verify the Local Yum Mirror
Create a new dnf repository entry with the following content:
cat << 'EOF' | sudo tee /etc/yum.repos.d/ol-local.repo > /dev/null
[ol_automation_http_repo]
name=OL_automation ($basearch)
baseurl=http://localhost:8080/repo/OracleLinux/OL8/automation/$basearch/
gpgcheck=0
EOF
Note: If exposing to external systems, change the
baseurl
above to the IP address or hostname of the system running the container.
Also, open the firewall to allow access to port 8080.sudo firewall-cmd --permanent --add-port=8080/tcp sudo firewall-cmd --reload
Then confirm the new entry works:
dnf repolist
dnf info ansible
Next Steps
Using reposync
makes local mirroring of remote repositories easier. By wrapping it in a container just makes it even easier. With your new-found skills, set up a mirror of your own and begin leveraging its benefits. Check out our content on the Oracle Linux Training Station for more learning opportunities related to DNF and containers.
Related Links
More Learning Resources
Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.
For product documentation, visit Oracle Help Center.
Use a Container to Create a DNF or ULN Repository Mirror
F54936-03
September 2024