Troubleshoot network issues with VTAP for OCI and Wireshark
Virtual Test Access Point (VTAP) for Oracle Cloud Infrastructure (OCI) provides insights into your network traffic and captures the data required for in-depth network analysis. VTAP for OCI enables OCI Network packet capture and inspection out of band to facilitate troubleshooting, security analysis, and data monitoring without impacting performance.
- A single compute instance
- An Application Load Balancer-as-a-Service (LBaaS) (layer 7, proxy load balancer)
- A Database-as-a-Service (DBaaS)
- An Exadata VM Cluster
- An Autonomous Data Warehouse instance using a private endpoint
The target is the resource that receives traffic mirrored from a VTAP. VTAP targets can be:
- A Network Load Balancer (layer 4, non-proxy load balancer)
Architecture
This reference architecture shows how VTAP for OCI and Wireshark can be used to monitor network traffic at various points in your OCI infrastructure and implement a cost-effective solution to troubleshoot network-related issues in your OCI tenancy.
The following diagram illustrates this reference architecture.
oci-vtap-network-wireshark-arch-oracle.zip
- LBaaS (layer 7, proxy load balancer) in a public subnet
- VNICs from application servers that the load balancer is directing traffic to
- Exadata clusters in a private subnet
- DBaaS systems in a private subnet
- Autonomous databases accessed via private endpoint (PE)
Next, you must navigate from the OCI console to the VTAP with the source you want to capture traffic details from and apply a capture filter. The capture filter allows you to capture only the required traffic based on the following factors:
- Traffic direction
- Ingress
- Egress
- IPv4 CIDR or IPv6 prefix
- Source
- Destination
- IP protocol
- All traffic
- ICMP: ICMP type, ICMP code
- TCP: Source port range, Destination port range
- UDP
- Source port range
- Destination port range
- ICMPv6
- ICMPv6 type
- ICMPv6 code
The architecture has the following components:
- Virtual Test Access Point (VTAP)
A Virtual Test Access Point (VTAP) provides a way to mirror traffic from a designated source to a selected target to facilitate troubleshooting, security analysis, and data monitoring. The VTAP uses a capture filter, which contains a set of rules governing what traffic a VTAP mirrors.
- Wireshark host
Wireshark is a free and open-source packet analyzer that allows traffic to be captured and analyzed. In this reference architecture, the Wireshark host captures network traffic and can present captured packet data in as much detail as possible. Traffic can be either analyzed from the command line or downloaded to a system with the Wireshark UI for graphical analysis.
- Load Balancer as a Service (LBaaS) (layer 7, proxy load
balancer)
A Oracle Cloud Infrastructure Load Balancing service (layer 7, proxy load balancer) provides automated traffic distribution from a single entry point to multiple servers in the back end.
- Network Load Balancer (layer 4, non-proxy load balancer)
A Network Load Balancer (layer 4, non-proxy load balancer) provides automated traffic distribution from one entry point to multiple backend servers in your virtual cloud networks. It operates at the connection level and load balances incoming client connections to healthy backend servers based on Layer3/Layer4 (IP protocol) data.
- Bastion service
Oracle Cloud Infrastructure Bastion service provides restricted and time-limited secure access to resources that don't have public endpoints and that require strict resource access controls, such as bare metal and virtual machines. In this reference architecture, the Bastion service allows secure access to the backend Wireshark hosts even when the Wireshark hosts do not have public endpoints.
- Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
- Availability domains
Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain is unlikely to affect the other availability domains in the region.
- Fault domains
A fault domain is a grouping of hardware and infrastructure within an availability domain. Each availability domain has three fault domains with independent power and hardware. When you distribute resources across multiple fault domains, your applications can tolerate physical server failure, system maintenance, and power failures inside a fault domain.
- Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you complete control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
- Autonomous database
Oracle Cloud Infrastructure autonomous databases are fully managed, preconfigured database environments that you can use for transaction processing and data warehousing workloads. You do not need to configure or manage any hardware, or install any software. Oracle Cloud Infrastructure handles creating the database, as well as backing up, patching, upgrading, and tuning the database.
- Exadata DB system
Exadata Cloud Service enables you to leverage the power of Exadata in the cloud. You can provision flexible X8M systems that allow you to add database compute servers and storage servers to your system as your needs grow. X8M systems offer RoCE (RDMA over Converged Ethernet) networking for high bandwidth and low latency, persistent memory (PMEM) modules, and intelligent Exadata software. You can provision X8M systems by using a shape that's equivalent to a quarter-rack X8 system, and then add database and storage servers at any time after provisioning.
- Application Server
Application servers use a secondary peer that, like the database, will take over processing in the event of a disaster. Application servers use configuration and metadata that is stored both in the database and the file system. Application server clustering provides protection in the scope of a single region but ongoing modifications and new deployments need to be replicated to the secondary location on an ongoing basis for a consistent disaster recovery.
- Route table
Virtual route tables contain rules to route traffic from subnets to destinations outside a VCN, typically through gateways.
- Security list
For each subnet, you can create security rules that specify the source, destination, and type of traffic that must be allowed in and out of the subnet.
Recommendations
- VCN
When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.
Select CIDR blocks that don't overlap with any other network (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider) to which you intend to set up private connections.
After you create a VCN, you can change, add, and remove its CIDR blocks.
When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.
- LBaaS (layer 7, proxy load balancer) bandwidth
While creating the load balancer, you can either select a predefined shape that provides a fixed bandwidth, or specify a custom (flexible) shape where you set a bandwidth range and let the service scale the bandwidth automatically based on traffic patterns. With either approach, you can change the shape at any time after creating the load balancer.
- Network Load Balancer (layer 4, non-proxy load
balancer)
Specify whether you want a private or public Network Load Balancer and then create the listener and backend sets. It is recommended to set the listener for UDP port 4789, which is the port for VXLAN (VTAP mirrored traffic is encapsulated in VXLAN). This will ensure that the Network Load Balancer will listen for and load balance all mirrored traffic.
Considerations
Consider the following options when deploying this reference architecture.
- Cost
There is no cost for the creation of VTAP if you have an OCI subscription. The Network Load Balancer used as the target for VTAP is part of the free tier. The hosted VM for Wireshark can also be in the free tier.