Instance Connectivity

This section discusses different aspects of instance connectivity.

Virtual Network Interface Cards (VNICs)

The compute nodes in the Private Cloud Appliance have physical network interface cards (NICs). When you create and launch a compute instance on one of the servers, the Networking service ensures that a VNIC is created on top of a physical interface, so that the instance can communicate over the network. Each instance has a primary VNIC that is automatically created and attached during launch. The primary VNIC resides in the subnet you specify when creating the instance. It cannot be removed from the instance.

A VNIC enables an instance to connect to a VCN and determines how the instance communicates with endpoints inside and outside of the VCN. Each VNIC resides in a subnet within a VCN and includes these items:

• One primary private IPv4 address from the subnet the VNIC is in.

• Up to 31 optional private IPv4 addresses from the subnet the VNIC is in.

• An optional public IPv4 address for each private IP, assigned at your discretion.

• An optional host name for DNS for each private IP address.

• A MAC address.

• A flag to enable or disable the source/destination check on the VNIC's network traffic.

• Optional membership of one or more network security groups (NSGs).

• An Oracle-assigned identifier (OCID).

• An optional friendly name you can choose and assign.

You can add secondary VNICs to an instance after it is launched. To be able to use a secondary VNIC, you must also configure the instance OS for it. The maximum number of VNICs for an instance varies by shape. Each secondary VNIC can be in a different subnet than the primary VNIC, either within the same VCN or a different one. A secondary VNIC can also be in the same subnet as the primary VNIC. However, attaching multiple VNICs from the same subnet CIDR block to an instance can introduce asymmetric routing, especially on Linux instances.

Note:

To avoid asymmetric routing in a configuration with multiple IP addresses from the same subnet, Oracle recommends assigning multiple private IP addresses to one VNIC, or using policy-based routing.

If traffic comes in to a service on the instance through a secondary VNIC and the service replies, the reply packets automatically have that VNIC's IP address as the source IP address. Policy-based routing is required for that reply to go back out on the same interface and find the correct default gateway.

Secondary VNICs must always be attached to an instance and cannot be moved. The process of creating a secondary VNIC automatically attaches it to the instance. The process of detaching a secondary VNIC automatically deletes it. They are automatically detached and deleted when you terminate the instance. An instance's bandwidth is fixed regardless of the number of VNICs attached. You cannot specify a bandwidth limit for a particular VNIC on an instance.

By default, every VNIC performs the source/destination check on its network traffic. The VNIC looks at the source and destination listed in the header of each network packet. If the VNIC is not the source or destination, then the packet is dropped. If the VNIC needs to forward traffic – for example, if it needs to perform Network Address Translation (NAT) – you must disable the source/destination check on the VNIC.

VNICs reside in a subnet but attach to an instance. The VNIC's attachment to the instance is a separate object from the VNIC or the instance itself. The VNIC and subnet always exist together in the same compartment, but the VNIC's attachment to the instance always exists in the instance's compartment. This distinction affects your IAM policies if you set up an access control scenario where network administrators manage the network and other users manage instances.

VNICs are rate limited to prevent very active applications from reducing the available bandwidth of other applications to unacceptable levels. Rate limiting applies to each instance’s VNIC interface.

The rate is limited to a value below the maximum network bandwidth associated with each instance shape. Generally, the bandwidth limit increases with the maximum number of VNICs a shape uses, but there is no maximum bandwidth guarantee.

Users cannot specify the bandwidth limits directly. Rate limiting is a system function. The rate applied to each VNIC is based on the shape option during instance launch.

IP Addressing

Instances use IP addresses for communication. Each instance has at least one private IP address and optionally one or more public IP addresses. A private IP address enables the instance to communicate with other instances inside the VCN, or with hosts in your on-premises network. A public IP address enables the instance to communicate with hosts outside of the Private Cloud Appliance network environment.

Certain types of resources in your tenancy are designed to be directly reachable from outside the secure appliance network environment, and therefore automatically come with a public IP address. For example: a NAT gateway. Other types of resources are directly reachable only if you configure them to be. For example: specific instances in your VCN. Direct public connectivity also requires that the VCN has an internet gateway and that the public subnet has correctly configured route tables and security lists.

Primary and Secondary IP Addresses

IP addresses are assigned to a network interface. In some networking models, this is the “IP layer” and in others it’s “Layer 3,” but any network interface needs an IP address to communicate. An IP packet always includes a source and destination IP address. If the source and destination IP addresses are on the same IP subnet, packet delivery is direct. If the source and destination IP addresses are on different IP subnets, packet delivery is through one or more routers or gateways.

Often, especially in a virtual cloud environments, a network interface needs more than one IP address. Also called “aliasing,” a network interface with one IP address already can be given several others, the limit determined mainly by implementation. The initial IP address for the interface is typically the primary IP interface addresses (this can be changed) and all the others become secondary IP interface addresses.

Secondary IP addresses are useful in many networking scenarios. All layered protocols allow more than one process to run on top of a particular layer. In TCP/IP, the IP layer can support more than one TCP layer process and the applications that run on top of the TCP layer. In other models, this means that Layer 3 can support more than one Layer 4 and above.

The advantage, which is very useful in virtual environments, is that applications that require their own IP addresses, such as web sites using SSL certificates, can share a single IP primary address by linking the web site to a secondary address.

With secondary addresses, a single network interface can support multiple connections to various applications running above the IP layer. Whenever the interface receives a packet addressed to one of its multiple secondary IP addresses, the matching of packet destination address to the list of secondary IP addresses assigned to the interface means, “This content is important. Process it further.”

If that is the case, and all secondary address destination packets are processed by the interface, why does one IP address have to be the primary? Because when packets are sourced, not from an application above the IP layer, but by a service for the IP layer itself, the packets always use the primary IP address. For example, an interface reporting a problem or notifying a router of an interface change in MTU or other parameter, always uses the primary address as the packet source address.

The primary address is the “stable” address for the interface. Secondary addresses often come and go, but changes to the primary IP address require careful consideration.

Private IPs

The Networking service defines a private IP object, identified by an Oracle-assigned OCID, which consists of a private IPv4 address and optional host name for DNS. Each instance receives a primary private IP object during launch. The Networking service uses DHCP to assign a private IP address. This address does not change during the instance's lifetime and cannot be removed from the instance. The private IP object is terminated when the instance is terminated.

If an instance has any secondary VNICs attached, each of those VNICs also has a primary private IP. A private IP can have a public IP assigned to it at your discretion. It is not supported to use a private IP as the target of a route rule in your VCN.

About Secondary Private IPs

After an instance is launched, you can add a secondary private IP to one of its VNICs. You can add it to either the primary VNIC or a secondary VNIC on the instance. The secondary private IP address must be in the subnet to which the VNIC belongs. You can move a secondary private IP from a VNIC on one instance to a VNIC on another instance on condition that both VNICs belong to the same subnet.

Typical use cases for a secondary private IP include:

• Instance failover: You assign a secondary private IP to an instance. If a problem occurs with the instance, you can easily reassign its secondary private IP to a standby instance in the same subnet. In addition, if the secondary private IP has an associated public IP, that public IP moves along with the private IP.
• Running multiple services or endpoints on a single instance: For example, you could have an instance that runs multiple container pods, where each pod uses its own IP address from the VCN's CIDR. The containers have direct connectivity to other instances and services in the VCN. Another example: you could run multiple SSL web servers with each one using its own IP address.

Secondary private IP addresses have the following properties:

• They are supported for all shapes and OS types.
• They can be assigned only after the instance is launched, or the secondary VNIC is created and attached.
• They are automatically deleted when you terminate the instance or detach and delete the secondary VNIC.
• Deleting a secondary private IP from a VNIC returns the address to the pool of available addresses in the subnet.
• A VNIC can have a maximum of 31 secondary private IPs.
• The instance's bandwidth is fixed regardless of the number of private IP addresses attached. You cannot specify a bandwidth limit for a particular IP address on an instance.
• A secondary private IP can have a reserved public IP associated with it at your discretion.

Note:

After assigning a secondary private IP to a VNIC, you must configure the instance OS to use it. For Linux and Microsoft Windows instructions, refer to the Oracle Private Cloud Appliance User Guide.

Public IPs

In a private cloud model, public IP addresses are not true unique and publicly routable internet IPs; they are just addresses external to the VCNs in the cloud environment. During initial setup of the appliance, a pool of addresses from the on-premises network is reserved for use as public IPs. A small set of these public IPs is required for system use.

You can assign a public IP address to an instance to enable external communication. The instance is assigned a public IP address from an address pool. Technically, the assignment is to a private IP object on the instance, and the VNIC that the private IP is assigned to must reside in a public subnet. A given instance can have multiple secondary VNICs; each with a (primary) private IP. Consequently, you can assign a given instance multiple public IPs across their VNICs.

The Networking service defines a public IP object, identified by an Oracle-assigned OCID, which consists of a private IPv4 address and additional properties that further define the public IP's type and behavior. There are two types of public IPs:

• An ephemeral public IP is temporary and exists for the lifetime of the instance.

• A reserved public IP is persistent and exists beyond the lifetime of the instance it is assigned to. It can be unassigned and then reassigned to another instance.

The following table summarizes the differences between both types:

Characteristics	Ephemeral Public IP	Reserved Public IP
Allowed assignment	to a VNIC's primary private IP limits: one per VNIC, two per instance	to a VNIC's primary private IP limit: one per VNIC
Creation	Optionally created and assigned during instance launch or secondary VNIC creation. You can create and assign one later if the VNIC doesn't already have one.	You create one at any time. You can then assign it when you like.
Unassignment	You can unassign it at any time, which deletes it. You might do this if whoever launched the instance included a public IP, but you do not want the instance to have one. When you stop an instance, its ephemeral public IPs remain assigned to the instance.	You can unassign it at any time, which returns it to your tenancy's pool of reserved public IPs.
Moving to a different resource	You cannot move an ephemeral public IP to a different private IP.	You can move it at any time by unassigning and then reassigning it to another private IP. Can be in a different VCN or availability domain.
Automatic deletion	Its lifetime is tied to the private IP's lifetime. Automatically unassigned and deleted when: its private IP is deleted its VNIC is detached or terminated its instance is terminated	Never. Exists until you delete it.
Scope	Availability domain	Regional (can be assigned to a private IP in any availability domain in the region)
Compartment and availability domain	Same as the private IPs	Can be different from the private IPs

When you launch an instance in a public subnet, the instance gets a public IP by default, unless you specify otherwise. After you create a given public IP, you cannot change which type it is. For example, if you launch an instance that is assigned an ephemeral public IP with address 203.0.113.2, you cannot convert the ephemeral public IP to a reserved public IP with address 203.0.113.2.

Resources that are designed to be directly publicly reachable automatically get a public IP address assigned from a pool upon creation. For NAT gateways, the assigned address is a regional ephemeral public IP. Like other ephemeral public IPs, it is automatically unassigned and deleted when you terminate its assigned resource. However, unlike other ephemeral public IPs, you cannot edit it or unassign it yourself.

DHCP Options

The Networking service uses DHCP to automatically provide configuration information to instances when they boot up. Although DHCP lets you change some settings dynamically, others are static and never change. For example, when you launch an instance, either you specify the instance's private IP address or the system chooses one for you. Each time the instance boots up or you restart the instance's DHCP client, DHCP passes that same private IP address to the instance. The address never changes during the instance's lifetime.

The Networking service provides DHCP options to let you control certain types of configuration on the instances in your VCN. You can change the values of these options at your discretion, and the changes take effect the next time you restart a given instance's DHCP client or reboot the instance.

Each subnet in a VCN can have a single set of DHCP options associated with it. That set of options applies to all instances in the subnet. Each VCN comes with a default set of DHCP options with initial values that you can change. Unless you create and assign a different set of DHCP options, every subnet uses the VCN's default set.

These are the available DHCP options you can configure:

• Domain Name Server

  The default setting is DNS Type = Internet and VCN Resolver.

  If instead you set DNS Type = Custom Resolver, you can specify up to three DNS servers of your choice.

• Search Domain

  If you set up your VCN with a DNS label, the default value for the Search Domain option is the VCN domain name: <VCN_DNS_label>.oraclevcn.com. Otherwise, the Search Domain option is not present in the default set of DHCP options.

  In general, when any set of DHCP options is initially created – the default set or a custom set –, the Networking service automatically adds the Search Domain option and sets it to the VCN domain name (<VCN_DNS_label>.oraclevcn.com) if all of these are true:

  • The VCN has a DNS label.

  • DNS Type = Internet and VCN Resolver.

  • You did NOT specify a search domain of your choice during creation of the set of DHCP options.

  After the set of DHCP options is created, you can always remove the Search Domain option or set it to a different value. You can specify only a single search domain in a set of DHCP options.

Important notes about your instances and DHCP options:

• Whenever you change the value of one of the DHCP options, you must either restart the DHCP client on the instance, or reboot the instance for the changes to take effect. This applies to all existing instances in the subnets associated with that set of DHCP options.

• Keep the DHCP client running so you can always access the instance. If you stop the DHCP client manually or disable NetworkManager, the instance cannot renew its DHCP lease and will become inaccessible when the lease expires; typically within 24 hours. Do not disable NetworkManager unless you use another method to ensure renewal of the lease.

• Stopping the DHCP client might remove the host route table when the lease expires. Also, loss of network connectivity to your iSCSI connections might result in loss of the boot drive.

• Any changes you make to the /etc/resolv.conf file are overwritten whenever the DHCP lease is renewed or the instance is rebooted.

• Changes you make to the /etc/hosts file are overwritten whenever the DHCP lease is renewed or the instance is rebooted. To persist your changes to the /etc/hosts file in Oracle Linux or CentOS instances, add the following line to /etc/oci-hostname.conf: PRESERVE_HOSTINFO=2. If the /etc/oci-hostname.conf file does not exist, create it.

Name Resolution

The Domain Name System (DNS) translates human-readable domain names to machine-readable IP addresses. For example, when you type a domain name into your browser, your operating system queries several DNS name servers until it finds the authoritative name server for that domain. The authoritative name server then responds with an IP address or other requested record data; the answer is relayed back to your browser and the DNS record is resolved to the web page.

For DNS name resolution within your VCN, there are two options. You choose an option for each subnet in the VCN, using the subnet's set of DHCP options.

The default choice is the Internet and VCN Resolver, an Oracle-provided solution. It consists of two parts: the Internet Resolver and the VCN Resolver. The Internet Resolver lets instances resolve host names that are publicly published on the internet, without requiring the instances to have internet access by way of either an internet gateway or a connection to the on-premises network. The VCN resolver lets instances resolve host names, which you can assign, of other instances in the same VCN.

Alternatively, you can use a Custom Resolver. It allows you to use up to three DNS servers of your choice for name resolution. These could be DNS servers that are available through the internet, in your VCN, or in your on-premises network, which is connected to your VCN through a DRG.

Domains and Host Names

When you create a VCN and subnets, you can specify DNS labels for each. Subnet DNS labels can only be set if the VCN itself is created with a DNS label. The labels, along with the parent domain of oraclevcn.com form the VCN domain name and subnet domain name. Next, when you launch an instance, you can assign a host name. It is assigned to the primary VNIC that is automatically created during instance launch. Along with the subnet domain name, the host name forms the instance's fully qualified domain name (FQDN).

• VCN domain name: <VCN_DNS_label>.oraclevcn.com

• Subnet domain name: <subnet_DNS_label>.<VCN_DNS_label>.oraclevcn.com

• Instance FQDN: <host_name>.<subnet_DNS_label>.<VCN_DNS_label>.oraclevcn.com

The FQDN resolves to the instance's private IP address. The Internet and VCN Resolver also enables reverse DNS lookup, which lets you determine the host name corresponding to the private IP address.

If you add a secondary VNIC to an instance, you can specify a host name. The resulting FQDN resolves to the VNIC's primary private IP address. The resulting FQDN resolves to that private IP address.

DNS labels and host names must meet these requirements:

• VCN and subnet labels must start with a letter and have a maximum length of 15 alphanumeric characters. Hyphens and underscores are not allowed. The value cannot be changed later.

• Host names can be up to 63 characters in length and must be compliant with RFCs 952 and 1123. The value can be changed later.

• VCN DNS labels must be unique across the VCNs in your tenancy. Subnet DNS labels must be unique within the VCN, and host names must be unique within the subnet.

Host Name Validation and Generation

If you set DNS labels for VCN and subnet, the host name is validated for compliance during instance launch. If you have not specified a host name, the system attempts to use the instance display name. If the display name does not pass validation, or if you did not specify one, the system generates a DNS-compliant host name. The generated host name is displayed on the instance’s detail page in the Compute Web UI.

If you launch an instance using the CLI or SDK, and you have not specified a host name or display name, the system does not generate them for you. This means the instance will not be resolvable using the Internet and VCN Resolver.

If you add a secondary VNIC to an instance, the system never generates a host name. You must provide a valid host name if you want the private IP address to be resolvable using the Internet and VCN Resolver.

Public DNS Zones

To enable DNS name resolution for instances deployed inside your VCNs, from outside the appliance network environment, Private Cloud Appliance provides configuration support for public DNS zones. Within your tenancy you create the DNS zones, or sections of the DNS namespace, you intend to manage through the Compute Web UI. The edge network of the appliance handles all DNS queries for the domain(s) in question.

When creating a DNS zone, you specify the name of the domain it manages – for example: example.com. You select whether the zone is primary or secondary. A primary zone contains its own DNS records, while a secondary zone retrieves its records from another zone. To access the external zone's records, the secondary zone needs at least one server IP address for the external zone. In addition, a TSIG (Transaction Signature) key may be required. TSIG keys are shared secrets used for authentication of secondary DNS zones. You can store these keys in the compartment of your choice.

Each DNS zone you create automatically contains two essential records:

• The SOA (Start of Authority) record specifies authoritative information about the DNS zone. This information includes the primary name server, the domain administrator email address, the domain serial number, and several timers related to refreshing the zone. For more information about SOA records, see RFC 1035.

• The NS (Name Server) record lists the authoritative name servers for a zone. For more information about NS records, see RFC 1035.

You configure the DNS zone by adding specific domain information in the form of resource records. For example, using an address record, you make a domain name resolve to the public IP address of an instance in a public subnet of a VCN. The public DNS zones in Private Cloud Appliance support the resource record types described in the table below.

Resource Record Type	Description
A	An address record used to point a host name to an IPv4 address. For more information about A records, see RFC 1035.
AAAA	An address record used point a host name at an IPv6 address. For more information about AAAA records, see RFC 3596.
ALIAS	A private pseudo-record that allows CNAME functionality at the apex of a zone.
CAA	A Certification Authority Authorization record allows a domain name holder to specify one or more Certification Authorities authorized to issue certificates for that domain. For more information about CAA records, see RFC 6844.
CDNSKEY	A Child DNSKEY moves a CDNSSEC key from a child zone to a parent zone. The information provided in this record must match the CDNSKEY information for your domain at your other DNS provider. This record is automatically created if you enable DNSSEC on a primary zone in Private Cloud Appliance DNS. For more information about CDNSKEY, see RFC 7344.
CDS	A Child Delegation Signer record is a child copy of a DS record, for transfer to a parent zone. For more information about CDS records, see RFC 7344.
CERT	A Certificate record stores public key certificates and related certificate revocation lists in the DNS. For more information about CERT records, see RFC 2538 and RFC 4398.
CNAME	A Canonical Name record identifies the canonical name for a domain. For more information about CNAME records, see RFC 1035.
CSYNC	A Child-to-Parent Synchronization record syncs records from a child zone to a parent zone. For more information about CNAME records, see RFC 7477.
DHCID	A DHCP identifier record provides a way to store DHCP client identifiers in the DNS to eliminate potential host name conflicts within a zone. For more information about DHCID, see RFC 4701.
DKIM	A Domain Keys Identified Mail is a special TXT record set up specifically to supply a public key used to authenticate arriving mail for a domain. For more information about DKIM records, see RFC 6376.
DNAME	A Delegation Name record has similar behavior to a CNAME record, but allows you to map an entire subtree beneath a label to another domain. For more information about DNAME records, see RFC 6672.
DNSKEY	A DNS Key record documents public keys used for DNSSEC. The information in this record must match the DNSKEY information for your domain at your other DNS provider. For more information about DNSKEY records, see RFC 4034.
DS	A Delegation Signer record resides at the top-level domain and points to a child zone's DNSKEY record. DS records are created when DNSSEC security authentication is added to the zone. For more information about DS records, see RFC 4034.
IPSECKEY	An IPSec Key record stores public keys for a host, network, or application to connect to IP security (IPSec) systems. For more information on IPSECKEY records, see RFC 4025.
KEY	A Key record stores a public key that is associated with a domain name. Currently only used by SIG and TKEY records. IPSECKEY and DNSKEY have replaced key for use in IPSec and DNSSEC, respectively. For more information about KEY records, see RFC 4025.
KX	A Key Exchanger record identifies a key management agent for the associated domain name with some cryptographic systems (not including DNSSEC). For more information about KX records, see RFC 2230.
LOC	A Location record stores geographic location data of computers, subnets, and networks within the DNS. For more information about LOC records, see RFC 1876.
MX	A Mail Exchanger record defines the mail server accepting mail for a domain. MX records must point to a host name. MX records must not point to a CNAME or IP address. For more information about MX records, see RFC 1035.
NS	A Nameserver record lists the authoritative nameservers for a zone. NS records are automatically generated at the apex of each new primary zone you create. For more information about NS records, see RFC 1035.
PTR	A Pointer record reverse maps an IP address to a hostname. This behavior is the opposite of an A Record, which forward maps a hostname to an IP address. PTR records are commonly found in reverse DNS zones. For more information about PTR records, see RFC 1035.
PX	A resource record used in X.400 mapping protocols. For more information about PX records, see RFC 822 and RFC 2163.
SOA	A Start of Authority record specifies authoritative information about a DNS zone, including: the primary nameserver the email of the domain administrator the domain serial number several timers relating to refreshing the zone An SOA record is generated automatically when a zone is created. For more information about SOA records, see RFC 1035.
SPF	A Sender Policy Framework record is a special TXT record used to store data designed to detect email spoofing. For more information about SPF records, see RFC 4408.
SRV	A Service Locator record allows administrators to use several servers for a single domain. For more information about SRV records, see RFC 2782.
SSHFP	An SSH Public Key Fingerprint record publishes SSH public host key fingerprints using the DNS. For more information about SSHFP records, see RFC 6594.
TLSA	A Transport Layer Security Authentication record associates a TLS server certificate, or public key, with the domain name where the record is found. This relationship is called a TLSA certificate association. For more information about TLSA records, see RFC 6698.
TXT	A Text record holds descriptive, human readable text, and can also include non-human readable content for specific uses. It is commonly used for SPF records and DKIM records that require non-human readable text items. For more information about TXT records, see RFC 1035.

Traffic Management Steering Policies

Steering Policies enable you to configure policies to serve intelligent responses to DNS queries, meaning different answers (endpoints) may be served for the query depending on the logic defined in the policy. Private Cloud Appliance supports these types of traffic management steering policies:

Policy Type	Description
Load Balancer	Load Balancer policies allow distribution of traffic across multiple endpoints. Endpoints can be assigned equal weights to distribute traffic evenly across the endpoints, or custom weights may be assigned for ratio load balancing.
IP Prefix Steering	IP Prefix steering policies enable you to steer DNS traffic based on the IP Prefix of the originating query. You can divide users into groups based on the subnets from where requests originate, and steer them to specific resources based on the subdivision you made. For example, you can serve different responses for your internal users as opposed to external users.

A steering policy contains rules to answer DNS queries. You use those rules to filter answers based on the properties of the DNS request. When multiple answers are served in response to DNS queries, that group of answers is called an answer pool. The answers within a pool are sorted by priority and are marked eligible or ineligible. Ineligible answers are omitted from the response.

When attached to a DNS zone, a steering policy takes precedence over all resource records of the zone it covers, and causes DNS responses to be constructed from the steering policy rules instead. For example, if a steering policy attached to the example.com DNS zone contains a rule covering the domain application.example.com and an answer for the A (address) record type, then the steering policy responds with that answer regardless of any relevant A records that exist in the zone. If a steering policy has no answer for the record type being requested, the DNS request is passed on to the next steering policy or the base zone records.

Steering policies only support records of the types A, AAAA, CNAME. A domain can have at most one steering policy attachment covering a given record type. This means a DNS zone (example.com) may have multiple attached steering policies covering different record types for a given domain – for example, one A record policy and one CNAME record policy for application.example.com. A DNS zone may also have multiple attached steering policies covering a given record type for different domains – for example, an A record policy for application.example.com and an A record policy for database.example.com. However, multiple steering policies for the same domain and record type are not supported, because the answer can be provided by only one policy.