Overview of Load Balancing
The Oracle Cloud Infrastructure Load Balancing service provides automated traffic distribution from one entry point to multiple servers reachable from your virtual cloud network (VCN). The service offers a load balancer with your choice of a public or private IP address, and provisioned bandwidth.
A load balancer improves resource utilization, facilitates scaling, and helps ensure high availability. You can configure multiple load balancing policies and application-specific health checks to ensure that the load balancer directs traffic only to healthy instances. The load balancer can reduce your maintenance window by draining traffic from an unhealthy application server before you remove it from service for maintenance.
How Load Balancing Works
The Load Balancing service enables you to create a public or private load balancer within your VCN. A public load balancer has a public IP address that is accessible from the internet. A private load balancer has an IP address from the hosting subnet, which is visible only within your VCN. You can configure multiple listeners for an IP address to load balance transport Layer 4 and Layer 7 (TCP and HTTP) traffic. Both public and private load balancers can route data traffic to any backend server that is reachable from the VCN.
Public Load Balancer
To accept traffic from the internet, you create a public load balancer. The service assigns it a public IP address that serves as the entry point for incoming traffic. You can associate the public IP address with a friendly DNS name through any DNS vendor.
A public load balancer is regional in scope. If your region includes multiple availability domains, a public load balancer requires either a regional subnet (recommended) or two availability domain-specific (AD-specific) subnets, each in a separate availability domain. With a regional subnet, the Load Balancing service creates a primary load balancer and a standby load balancer, each in a different availability domain, to ensure accessibility even during an availability domain outage. If you create a load balancer in two AD-specific subnets, one subnet hosts the primary load balancer and the other hosts a standby load balancer. If the primary load balancer fails, the public IP address switches to the secondary load balancer. The service treats the two load balancers as equivalent and you cannot specify which one is "primary".
Whether you use regional or AD-specific subnets, each load balancer requires one private IP address from its host subnet. The Load Balancing service supplies a floating public IP address to the primary load balancer. The floating public IP address does not come from your backend subnets.
If your region includes only one availability domain, the service requires just one subnet, either regional or AD-specific, to host both the primary and standby load balancers. The primary and standby load balancers each require a private IP address from the host subnet, in addition to the assigned floating public IP address. If there is an availability domain outage, the load balancer has no failover.
Private Load Balancer
To isolate your load balancer from the internet and simplify your security posture, you can create a private load balancer. The Load Balancing service assigns it a private IP address that serves as the entry point for incoming traffic.
When you create a private load balancer, the service requires only one subnet to host both the primary and standby load balancers. The load balancer can be regional or AD-specific, depending on the scope of the host subnet. The load balancer is accessible only from within the VCN that contains the host subnet, or as further restricted by your security rules.
The assigned floating private IP address is local to the host subnet. The primary and standby load balancers each require an extra private IP address from the host subnet.
If there is an availability domain outage, a private load balancer created in a regional subnet within a multi-AD region provides failover capability. A private load balancer created in an AD-specific subnet, or in a regional subnet within a single availability domain region, has no failover capability in response to an availability domain outage.
Your load balancer has a backend set to route incoming traffic to your Compute instances. The backend set is a logical entity that includes:
- A list of backend servers.
- A load balancing policy.
- A health check policy.
- Optional SSL handling.
- Optional session persistence configuration.
The backend servers (Compute instances) associated with a backend set can exist anywhere, as long as the associated network security groups (NSGs), security lists, and route tables allow the intended traffic flow.
If your VCN uses network security groups (NSGs), you can associate your load balancer with an NSG. An NSG has a set of security rules that controls allowed types of inbound and outbound traffic. The rules apply only to the resources in the group. Contrast NSGs with a security list, where the rules apply to all the resources in any subnet that uses the list. For more information about NSGs, see Network Security Groups.
If you prefer to use security lists for your VCN, the Load Balancing service can suggest appropriate security list rules. You also can configure them yourself through the Networking service. See Security Lists for more information.
See Security Rules for detailed information comparing NSGs and security lists.
Oracle recommends that you create your load balancer in a regional subnet.
Oracle recommends that you distribute your backend servers across all availability domains within the region.
To create a minimal system with a functioning load balancer, you must:
For a public load balancer, create a VCN with an internet gateway and a public regional subnet.
- For a private load balancer, create a VCN with at least one private subnet.
- Create at least two Compute instances, each in a separate availability domain.
- Create a load balancer.
- Create a backend set with a health check policy.
- Add backend servers (Compute instances) to the backend set.
- Create a listener, with optional SSL handling.
- Update the load balancer subnet security rules so they allow the intended traffic.
Private IP Address Consumption
A public load balancer created in one public subnet consumes two private IP addresses from the host subnet.
A public load balancer created in two public subnets consumes two private IP addresses, one from each host subnet.
A private load balancer created in a single subnet consumes three private IP addresses from the host subnet.
See Getting Started with Load Balancing for step-by-step instructions to create a simple load balancing setup.
The following diagram provides a high-level view of a simple public load balancing system configuration. Far more sophisticated and complex configurations are common.
Load Balancing Concepts
The following concepts are essential to working with Load Balancing.
- BACKEND SERVER
- An application server responsible for generating content in reply to the incoming TCP or HTTP traffic. You typically identify application servers with a unique combination of overlay (private) IPv4 address and port, for example, 10.10.10.1:8080 and 10.10.10.2:8080.
- For more information, see Managing Backend Servers.
- BACKEND SET
- A logical entity defined by a list of backend servers, a load balancing policy, and a health check policy. SSL configuration is optional. The backend set determines how the load balancer directs traffic to the collection of backend servers.
- For more information, see Managing Backend Sets.
- If you use HTTPS or SSL for your listener, you must associate an SSL server certificate (X.509) with your load balancer. A certificate enables the load balancer to terminate the connection and decrypt incoming requests before passing them to the backend servers.
- For more information, see Managing SSL Certificates.
- health check
A health check is a test to confirm the availability of backend servers. A health check can be a request or a connection attempt. Based on a time interval you specify, the load balancer applies the health check policy to continuously monitor backend servers. If a server fails the health check, the load balancer takes the server temporarily out of rotation. If the server subsequently passes the health check, the load balancer returns it to the rotation.
You configure your health check policy when you create a backend set. You can configure TCP-level or HTTP-level health checks for your backend servers.
- TCP-level health checks attempt to make a TCP connection with the backend servers and validate the response based on the connection status.
- HTTP-level health checks send requests to the backend servers at a specific URI and validate the response based on the status code or entity data (body) returned.
The service provides application-specific health check capabilities to help you increase availability and reduce your application maintenance window.
- For more information on health check configuration, see Editing Health Check Policies.
- HEALTH STATUS
- An indicator that reports the general health of your load balancers and their components.
- For more information, see the Health Status section of Editing Health Check Policies.
- A logical entity that checks for incoming traffic on the load balancer's IP address. You configure a listener's protocol and port number, and the optional SSL settings. To handle TCP, HTTP, and HTTPS traffic, you must configure multiple listeners.
- Supported protocols include:
- For more information, see Managing Listeners.
- LOAD BALANCING POLICY
- A load balancing policy tells the load balancer how to distribute incoming traffic to the backend servers. Common load balancer policies include:
- Round robin
- Least connections
- IP hash
- For more information, see How Load Balancing Policies Work.
- PATH ROUTE SET
- A set of path route rules to route traffic to the correct backend set without using multiple listeners or load balancers.
- For more information, see Managing Request Routing.
- REGIONS AND availability domains
- The Load Balancing service manages application traffic across availability domains within a region . A region is a localized geographic area, and an availability domain is one or more data centers located within a region. A region is composed of several availability domains.
- For more information, see Regions and Availability Domains.
- SESSION PERSISTENCE
- A method to direct all requests originating from a single logical client to a single backend web server.
- For more information, see Session Persistence.
- A template that determines the load balancer's total pre-provisioned maximum capacity (bandwidth) for ingress plus egress traffic. Available shapes include 10Mbps, 100 Mbps, 400 Mbps, and 8000 Mbps.
- The 10Mbps shape is Always Free eligible. For more information about Always Free resources, including additional capabilities and limitations, see Oracle Cloud Infrastructure Free Tier.
Pre-provisioned maximum capacity applies to aggregated connections, not to a single client attempting to use the full bandwidth.
- Secure Sockets Layer (SSL) is a security technology for establishing an encrypted link between a client and a server. You can apply the following SSL configurations to your load balancer:
- SSL TERMINATION
- The load balancer handles incoming SSL traffic and passes the unencrypted request to a backend server.
- END TO END SSL
- The load balancer terminates the SSL connection with an incoming traffic client, and then initiates an SSL connection to a backend server.
- SSL TUNNELING
- If you configure the load balancer's listener for TCP traffic, the load balancer tunnels incoming SSL connections to your application servers.
- Load Balancing supports the TLS 1.2 protocol with a default setting of strong cipher strength. The default supported ciphers include:
- For more information, see Managing SSL Certificates.
- A subdivision you define in a VCN, such as 10.0.0.0/24 and 10.0.1.0/24. A subnet can span a region or exist within in a single availability domain. A subnet consists of a contiguous range of IP addresses that do not overlap with other subnets in the VCN. For each subnet, you specify the routing and security rules that apply to it.
- For more information on subnets, see VCNs and Subnets and Public IP Address Ranges.
You can apply tags to your resources to help you organize them according to your business needs. You can apply tags at the time you create a resource, or you can update the resource later with the wanted tags. For general information about applying tags, see Resource Tags.
- VIRTUAL HOSTNAME
- A virtual server name applied to a listener to enhance request routing.
- For more information, see Managing Request Routing.
- VIRTUAL CLOUD NETWORK (VCN)
- A private network that you set up in the Oracle data centers, with firewall rules and specific types of communication gateways that you can choose to use. A VCN covers a single, contiguous IPv4 CIDR block of your choice in the allowed IP address ranges.
- You need at least one virtual cloud network before you launch a load balancer.
- For information about setting up virtual cloud networks, see Networking Overview.
- Specifies whether your load balancer is public or private.
- A public load balancer has a public IP address that clients can access from the internet.
- A private load balancer has a private IP address from a VCN local subnet. Clients can access the private load balancer using methods and technology that can provide access to a private IP, such as:
Cross-VCN (via LPG peering)
From another region (via RPC)
From on-prem (via FC private peering)
- For more information, see Managing Load Balancers.
- WORK REQUEST
- An object that reports on the current state of a Load Balancing request.
- The Load Balancing service handles requests asynchronously. Each request returns a work request ID (OCID) as the response. You can view the work request item to see the status of the request.
- For more information, see Viewing the State of a Work Request.
Troubleshooting an HTTP 502 Bad Gateway
- When you browse to a load-balanced IP address, you see 502 Bad Gateway error.
- You can confirm this behavior by running a curl test:
curl -v http://ociexample.com
> GET / HTTP/1.1 > Host: 220.127.116.11 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 502 Bad Gateway < Content-Type: text/html < Content-Length: 161 < Connection: keep-alive
- Use the OCI Logging Service Load Balancer Logs and check for lbStatusCode: "502".
- Check for
You will then correlate this status code with one of the following scenarios below.
502 from Backend
- Backend application is returning a 502
- Issue is misconfigured backend
- Backend is likely another reverse proxy or LB
504 from Backend
- Examine upstream proxy logs to determine why it is returning 502
- Resolve any issues on the ultimate backend that is causing the upstream proxy to return a 502
- When receiving a 504 from the Backend it typically means that the backend is another proxy or LBaaS instance. This code is typically returned when a proxy is unable to connect to an upstream server in a specified time.
- It is recommended to examine the logs of the upstream system and see what is causing that upstream proxy from connection to its backend.
- Increase the connection timeout.
- Determine why backend is taking longer time to respond than usual using a utility such as tcpdump, and built-in application tools.
500 from Backend
- Receiving a 500 from the backend is typically indicative of a server side error, commonly known as an "Internal Server Error". This typically is a result of the backend application.
- Resolve application level issue that is causing
- Common causes, inability to connect to upstream resources such as databases, APIs, services, etc.
No Status Code from the Backend
- You typically see no backend status code accompanied by a 502 lbStatusCode, when no Backend is available to send the connections.
- You will also notice "No healthy backends available in associated backendSet" in the OCI Logging Load Balancer Error Logs.
- It is recommended to ensure the backends are healthy, if they are healthy you will need to confirm that your health check is properly configured.
Most types of Oracle Cloud Infrastructure resources have a unique, Oracle-assigned identifier called an Oracle Cloud ID (OCID). For information about the OCID format and other ways to identify your resources, see Resource Identifiers.
Ways to Access Oracle Cloud Infrastructure
You can access Oracle Cloud Infrastructure using the Console (a browser-based interface) or the REST API. Instructions for the Console and API are included in topics throughout this guide. For a list of available SDKs, see Software Development Kits and Command Line Interface.
To access the Console, you must use a supported browser. To go to the Console sign-in page, open the navigation menu at the top of this page and click Infrastructure Console. You will be prompted to enter your cloud tenant, your user name, and your password.
For general information about using the API, see REST APIs.
You can monitor the health, capacity, and performance of your Oracle Cloud Infrastructure resources by using metrics, alarms, and notifications. For more information, see Monitoring Overview and Notifications Overview.
For information about monitoring the traffic passing through your load balancer, see Load Balancing Metrics.
Authentication and Authorization
Each service in Oracle Cloud Infrastructure integrates with IAM for authentication and authorization, for all interfaces (the Console, SDK or CLI, and REST API).
An administrator in your organization needs to set up groups , compartments , and policies that control which users can access which services, which resources, and the type of access. For example, the policies control who can create new users, create and manage the cloud network, launch instances, create buckets, download objects, etc. For more information, see Getting Started with Policies. For specific details about writing policies for each of the different services, see Policy Reference.
If you’re a regular user (not an administrator) who needs to use the Oracle Cloud Infrastructure resources that your company owns, contact your administrator to set up a user ID for you. The administrator can confirm which compartment or compartments you should be using.
Limits on Load Balancing Resources
Other limits include:
- You cannot convert an AD-specific load balancer to a regional load balancer or the reverse.
- The Load Balancing services supports IPv6 addresses for load balancers in the US Government Cloud only. IPv6 support is only for the load balancer itself, and not the backend.
The maximum number of concurrent connections is limited when you use stateful security rules for your load balancer subnets. In contrast, there is no theoretical limit on concurrent connections if you use stateless security rules. The practical limitations depend on various factors. The larger your load balancer shape, the greater the connection capacity. Other considerations include system memory, TCP timeout periods, TCP connection state, and so forth.Tip
To accommodate high-volume traffic, Oracle strongly recommends that you use stateless security rules for your load balancer subnets.
Each load balancer has the following configuration limits:
- One IP address
- 16 backend sets
- 512 backend servers per backend set
- 1024 backend servers total
- 16 listeners