Note:
Before attempting this procedure, set up console access to the systems involved. This is required because running a wrong command can take a network offline, possibly severing any connection you have to the server. In this scenario, the console access is required to repair the network.
The RUEI User's Guide describes how to locate your installation within a network. GRE Tunnelling allows you to locate the Collector Engine anywhere in your network as long as the tunnel endpoints can communicate with each other. While GRE tunnelling is efficient, the network throughput can decrease because of network throughput overhead caused by the additional headers added to the packets and the CPU time overhead caused by encapsulation and decapsulation of those packets.
Linux supports GRE Ethernet tunneling from kernel version 2.6.28, and requires an up-to-date version of the iproute package containing the utilities (specifically the IP utility) to set up and configure GRE Ethernet tunnel (gretap) interfaces.
This procedure uses Oracle Linux 6.4 as a base for setting up the GRE Ethernet tunnels, as Oracle Linux version 6 provides a UEK kernel (Linux version 2.6.39-400.109.1.el6uek at time of writing) capable of setting up GRE Ethernet tunnels, as well as the correct version of the iproute package (iproute-2.6.32-23.el6.x86_64 for Oracle Linux 6.4) required to add, delete, or modify GRE Ethernet tunnels.
Oracle Linux 5 is not supported, even if it supports GRE Ethernet tunnels in its UEK kernel (Linux version 2.6.39-400.21.1.el5uek for OEL5.9). As it needs a newer version (2.6.28 or higher) of the iproute package capable of setting up the GRE Ethernet tunnels. The required version of the iproute package is not officially supported. Hence, is not covered in this procedure.
For more information about installing and configuring a collector, see Oracle Real User Experience Insight Installation Guide.
While this appendix contains details on various aspects of taps and GRE tunnels, the following outlines the process that must be completed:
Perform either the manual or scripted GRE setup. See Manual Setup for Basic RUEI Tap and GRE Tunnel and Scripted Setup for Basic RUEI Tap and GRE Tunnel.
Ensure that the destination endpoint is only receiving traffic. See Making a Tunnel Unidirectional.
Add a tap. See Adding a Virtual Tap.
Configure the collector. See Configuring a Collector for GRE Tunnelling.
Test the setup. See Testing a GRE Tunnel.
Ensure that your configuration is applied even after a reboot. See Making GRE Tunnel Environment Changes Permanent.
This section describes creating a single GRE tunnel, and how to set up either endpoint (assuming they are both Oracle Linux version 6 machines) to be able to aggregate either tap (source) traffic or GRE tunnel output (destination) traffic. With this process you can add one or more taps on one machine to the GRE tunnel, have the collector listen to one or many incoming GRE tunnels. This section contains the following sections:
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
bridge - support for bridges
veth - support for virtual ethernet interfaces
This section describes manually setting up two Oracle Linux version 6 systems, one as a source and the other as destination (with the RUEI Collector). It is an alternative process to Scripted Setup for Basic RUEI Tap and GRE Tunnel.
To prepare both systems, do the following:
Install a GRE tunnel between the source and destination systems by following the instructions in Configuring a GRE Tunnel Manually.
Install a bridge (BRTUN) on each of the source and destination systems by following the instructions in Creating and Setting Up a Linux Bridge.
On both the source and destination systems, add the local GRE tunnel endpoint interface (GRETUN) to the bridge (BRTUN) by following the instructions in Adding and Removing Bridge Interfaces.
By following the instructions in Testing a GRE Tunnel you should be able to view the generated test traffic from the source coming through the tunnel, both on the GRE tunnel interfaces as well as on the bridge interfaces on both ends.
Go to section Making a Tunnel Unidirectional.
This section describes setting up two Oracle Linux version 6 systems using a script, one as a source and the other as destination (with the RUEI Collector). It is an alternative process to Manual Setup for Basic RUEI Tap and GRE Tunnel
To prepare both the systems, use the tunnelctl script to create a bridged GRE tunnel as described in Configuring a GRE Tunnel Using the tunnelctl Script.
Both the source and destination systems are set up, but no traffic flows through the bridges or the tunnel. Before we connect any taps to the tunnel source, we need to make sure that the destination GRE tunnel endpoint (GRETUN on the destination system) can only receive traffic, not send any over the tunnel.
Also, we need to ensure the GRE tunnel is unidirectional as we only want to monitor traffic, not take part in it. We will use linux traffic shaping to block outgoing traffic for the GRETUN interface on the destination endpoint system.
Do the following steps:
Select a handle (HANDLE) to be used for this qdisc. For example, reuse the GRE tunnel id (ID).
Replace the root qdisc of GRETUN with one (prio) that can filter the outgoing traffic, by running the following command as root:
tc qdisc replace dev GRETUN parent root handle HANDLE: prio
Add a filter to pass all outgoing GRE traffic from the machine so that it does not get mirrored. Run the following commands:
tc filter add dev GRETUN parent HANDLE: \ protocol all prio 1 u32 \ match u32 0 0 flowid HANDLE:1 \ action drop
The GRE tunnel is now unidirectional. You can test this by generating traffic on one system by using the ping option and viewing it on the other system by using the tcpdump as described in Testing a GRE Tunnel, and then re-doing the test in the other direction. You should see traffic flowing from the source (tap) to destination (RUEI Collector), but no traffic from the destination (RUEI Collector) to source (tap).
We will now create a virtual tap for one of the local interfaces on the source system, by performing the following steps:
Choose an interface (ETH) on the source system whose traffic you want to monitor on the destination RUEI system.
Create a virtual tap for the interface chosen in step 1, by using the instructions in Creating a Virtual Tap.
On the source system, add the created tap interface (TAP) to the bridge (BRTUN), by using the instructions in Adding and Removing Bridge Interfaces.
On the destination system, test the incoming GRE tunnel traffic, by running the following command as root:
tcpdump -i GRETUN -c 100 -n
Note:
As described in the Testing the Tap, the traffic you see on the bridge interface on the destination system should now be the same as the traffic on the bridge interface on the source system.
After we have a GRE tunnel set up and tested, the collector can be configured to listen to the traffic on the GRE Tunnel. To enable the RUEI Collector Engine to listen to the GRE Ethernet tunnel, do the following:
Using RUEI (Configuration > Security > Collector profiles), note the collector profile that you want to configure. The default network based profile is named System network data collectors. If necessary create a new profile. In the following steps the chosen profile will be referred to as PROFILE.
Make sure the PROFILE Collector you want to configure is governed by the chosen Collector profile.
Log in to the RUEI Reporter system as the $RUEI_USER user.
Run the following command:
execsql config_set_profile_value PROFILE config ForceInterface add greID
Where,
greID is the tunnel interface that you created earlier.
PROFILE is the profile that you choose in step 1.
The new configuration should now automatically propagate to the Collector. The collector must be forced to listen to the interface, since it is not a physical interface and lacks certain internal signals used by the collector to decide if the interface is up or down. The collector would otherwise not use the interface.
Make sure there is not a firewall filtering any packets coming through the interface. It is outside the scope of this document to explain how to perform this task. Here are few tips:
The firewall should be set up to totally ignore the interface, not set up to route everything to a single other interface in the GRE tunnel network. This is because any filtering causes CPU overhead, which can have a negative effect on throughput.
Any generic firewall rules, that is rules covering all interfaces can also apply to the interface currently being configured, must be altered not to cover this interface. As a workaround, add new rules to ignore this interface.
Disable any network throttling that might affect the interface. It is outside the scope of this document to explain how to perform this task.
This section describes how to create a GRE tunnel using the tunnelctl script provided with RUEI.
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
bridge - support for bridges
veth - support for virtual ethernet interfaces
The two endpoints are able to reach each other (for example, tested using ping). The relevant ports must have been opened in any firewalls, both on the endpoints as well as on any router in between.
The two endpoints must have an executable copy of the tunnelctl script.
root user access is available on both endpoints.
Perform the following steps to set up the first endpoint:
Note the IP address of the local and remote endpoints.
Create a numeric Identifier (ID) to be used for both endpoints, for example 123. This ID will be used to identify the tunnel on both sides.
Log in as root using ssh and run the following command:
tunnelctl create gre Local_IP Remote_IP ID
Where,
Local_IP is the address of the current server.
Remote_IP is the address of the remote server.
ID is the identifier you created in the previous step.
Check that the tunnel has been created:
tunnelctl list
An interface named greID should be listed.
To set up a tunnel both endpoints must be configured. The 'other' endpoint can be a switch or router capable of duplicating streams and sending them out through a GRE Ethernet tunnel, or it may be another Linux server where any duplication/streaming can be set up.
If the other endpoint is a router or switch capable of duplicating streams and sending them out through a GRE Ethernet tunnel, refer to the product documentation for any steps that might be necessary.
If the other endpoint is a Linux server, repeat the steps in Setting Up a Tunnel Endpoint on the second endpoint (noting that you need to reverse the local and remote IP addresses when creating the tunnel).
This section describes how to create a GRE tunnel manually.
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
The two endpoints are able to reach each other (for example, tested using ping). The relevant ports must have been opened in any firewalls, both on the endpoints as well as on any router in between.
The two endpoints must have an executable copy of the tunnelctl script.
root user access is available on both endpoints.
Perform the following steps to set up the first endpoint:
Note:
Do not bring the interface up until completing this procedure.
Note the IP address of the local and remote endpoints.
Create a numeric Identifier (ID) to be used for both endpoints, for example 123. This ID will be used to identify the tunnel on both sides.
Log in as root using ssh and enter the following command to load the GRE modules in the Linux kernel:
modprobe ip_gre
Run the following command to check that the GRE modules are loaded in the Linux kernel:
lsmod | grep gre
Log in as root using ssh and run the following command:
ip link add ID type gretap local Local_IP remote Remote_IP
Where,
Local_IP is the address of the current server.
Remote_IP is the address of the remote server.
ID is the identifier you created in the step 2.
Check that the tunnel has been created:
ip link show
An interface named greID should be listed.
Configure the kernel not to route anything coming from the tunnel interface by performing the steps in Configuring an Interface for Mirrored Traffic, taking care to swap IFACE with the interface name you are currently preparing (for example greID).
To set up a tunnel both endpoints must be configured. The 'other' endpoint can be a switch or router capable of duplicating streams and sending them out through a GRE Ethernet tunnel, or it may be another Linux server where any duplication/streaming can be set up.
If the other endpoint is a router or switch capable of duplicating streams and sending them out through a GRE Ethernet tunnel, refer to the product documentation for any steps that might be necessary.
If the other endpoint is a Linux server, repeat the steps in Setting Up a Tunnel Endpoint Manually on the second endpoint (noting that you need to reverse the local and remote IP addresses when creating the tunnel).
This section describes how to create and set up a linux bridge which will act as a layer 2 hub for mirrored data. You can add virtual taps and GRE tunnels to the bridge to create the required configuration. Setting up multiple bridges is also possible, but such a configuration is beyond the scope of this document.
The following components are required:
The following packages must be installed:
iproute2
tcpdump
bridge-utils
The following kernel module must be present and loaded:
bridge
Create a bridge by completing the following steps:
Log in as root using ssh and run the following command:
brctl addbr BRTUN
Where,
BRTUN is the name of the bridge.
Run the following command to check the bridge was created:
brctl show
Run the following command to configure the bridge to act as a (dumb) hub instead of a switch:
brctl setfd BRTUN 0brctl setageing BRTUN 0
Run the following command to configure the bridge to be silent:
brctl stp BRTUN off
Configure the kernel not to route anything coming from the bridge interface by performing the steps in Configuring an Interface for Mirrored Traffic, taking care to swap IFACE with the interface name you are currently preparing (for example BRTUN>).
Run the following command to activate the bridge and set it to accept all traffic:
ip link set BRTUN promisc on arp off up
To add an interface (IFACE) to a bridge, run the following command:
brctl addif BRTUN IFACE
To remove an interface (IFACE) from a bridge, run the following command:
brctl delif BRTUN IFACE
To view the current configuration of the bridge, run the following command:
brctl show
After we have a GRE tunnel set up between two endpoints, and an interface for mirrored traffic to ensure that no mirrored traffic is routed on the linux (virtual) machine, an unused GRE Ethernet tunnel can be tested by running ping on one end and tcpdump on the other to see the GRE tunnel traffic. In the following steps, the two endpoints are referred to as the source and the destination, where the source signifies the endpoint where ping is running, and the destination is where tcpdump is used to verify the traffic:
Make sure the GRE tunnel interface on either endpoint system is up, by running the following command as root on both systems:
ip li set GRETUN up
Send ICMP packets through the tunnel, by running the following command as root on the source system:
ping -I GRETUN 127.1.1.1
The IP address has specifically been chosen so that it does not get inadvertently routed anywhere, as it is a local address. Using ping -I means that the ICMP packets only get sent over the GRE tunnel, restricting the visibility to the destination endpoint.
Check that the GRE encapsulated tunnel traffic was received, by running the following command as root on the destination system. Where, ETH is the interface the tunnel is routed over (the local endpoint, typically eth0), not the tunnel interface itself.
tcpdump -i ETH -c 100 proto gre
You should see ARP and/or ICMP requests for the above IP address wrapped in GRE packets (GREv0) similar to the following:
… IP server_A > server_B: GREv0, length 46: ARP, Request who-has 127.1.1.1 tell server_A, length 28 … IP server_A > server_B: GREv0, length 102: IP server_A > 127.1.1.1: ICMP echo request, id 62057, seq 1, length 64
This section describes a generic method of creating a “tap" network interface that will provide mirrored traffic from any other live interface on the Oracle Linux version 6 machine. This method uses linux traffic shaping to mirror incoming and outgoing data from an interface and copy that network traffic to a set of newly created virtual ethernet interfaces.
A set of two virtual ethernet interfaces are connected to each other in such a way that any data flowing into one will flow out of the other, in this sense they act as a virtual NIC cable. These virtual ethernet interfaces are commonly used in virtual networking.
Any local interface can be mirrored by using this method, including the interface the controlling ssh connection and the interface carrying GRE tunnel traffic. This is possible because GRE traffic will be filtered out of any mirrored traffic by one of the traffic shaping rules in this chapter.
This procedure creates a pair of virtual interfaces, one called “ETHmirror" and the other called “ETHtap". For example, if you want to tap interface eth0, you first create a set of virtual interfaces called eth0mirror and eth0tap. The interfaces are named this way to help keep them apart from any other mirroring setups on the system, since this method allows us to mirror more than one local interface into the GRE tunnel. From now on we will reference them as ETH, MIRROR and TAP.
The ETH interface will have its traffic mirrored on the MIRROR interface. All traffic flowing through the MIRROR interface will also be seen on the TAP interface since they are a virtual ethernet pair, so that you can use that TAP interface in any network configuration (directly or in a bridge) that you want.
The following components are required:
The tap is to be created on an Oracle Linux version 6 system
The following packages must be installed:
iproute2
tcpdump
The following kernel module must be present and loaded:
veth - support for virtual ethernet interfaces
A live interface to be mirrored exists, this interface will be referred to from now on as ETH.
Complete the following steps to create the mirror/tap virtual interfaces:
Create a pair of virtual interfaces by running the following command as root:
ip li ad TAP type veth peer name MIRROR
Activate the interfaces, by running the following commands as root:
ip li set dev TAP up promisc on arp off ip li set dev MIRROR up promisc on arp off
Traffic shaping enables the copying of all incoming and outgoing traffic for a given interface (ETH) to the newly created MIRROR virtual interface. How traffic shaping works is not explained in this document, though individual steps will be annotated.
The mirror setup is simple, though you do need to add an extra filter to prevent any GRE traffic (packet type GREv0, see Testing a GRE Tunnel) from being mirrored. This must be done to ensure that if you are mirroring the interface the GRE tunnel is transported over, you will not force the GRE tunnel to carry its own traffic (a loop) as that would cause the network to fail, and the server to fail.
To mirror the incoming traffic, do the following:
Add an ingress qdisc to ETH by running the following command as root:
tc qdisc add dev ETH ingress
Add a filter to pass all incoming GRE traffic to the machine so that it is not mirrored. Run the following commands:
tc filter add dev ETH parent ffff: \
protocol all prio 1 u32 \
match ip protocol 47 0xff flowid 1:1 \
action pass
Add a filter to mirror all remaining traffic to our MIRROR interface. Run the following commands:
tc filter add dev ETH parent ffff: \ protocol all prio 2 u32 \ match u32 0 0 flowid 1:2 \ action mirred egress mirror dev MIRROR
To mirror all outgoing traffic, do the following:
Replace the root qdisc of ETH with one (prio) that can filter the outgoing traffic by running the following command as root:
tc qdisc replace dev ETH parent root handle 10: prio
Add a filter to pass all outgoing GRE traffic to the machine so that it is not mirrored. Run the following commands:
tc filter add dev ETH parent 10: \
protocol all prio 1 u32 \
match ip protocol 47 0xff flowid 10:1 \
action pass
Add a filter to mirror all remaining traffic to our MIRROR interface. Run the following commands:
tc filter add dev ETH parent 10: \ protocol all prio 2 u32 \ match u32 0 0 flowid 10:2 \ action mirred egress mirror dev MIRROR
Note:
If you deactivate the MIRROR interface after completing this procedure, it will disrupt the ETH network traffic. Leave the MIRROR interface active, since you will only be using the TAP interface in the remaining setup, and that interface can be de-activated without any consequences
At this point you have two new interfaces, MIRROR and TAP. The MIRROR is used by the traffic shaping rules to mirror the network traffic from ETH to, and TAP is the virtual interface counterpart of MIRROR. Leave MIRROR active from now on, and use TAP in our networking setup, as long as you make sure it's data is not being routed by the system. To test this, look at the traffic on the TAP interface. That traffic should be the same as the traffic on ETH, minus the GRE traffic.
View the traffic on the TAP interface by running the following command as root:
tcpdump -i TAP -c 100 -n
Compare the output of step 1 with the output of the following command, which is the traffic on ETH with the GRE traffic filtered out:
tcpdump -i ETH -c 100 -n ! proto gre
Note:
To see the same output you should run both commands simultaneously. If you run the previous steps simultaneously you will probably see that the output does not line up, but after finding where they align you should see that they are the same.
This section describes how to ensure that the Linux kernel does not route or filter any packets going through a specific interface.
In the following steps IFACE denotes the interface that is being set up to accept any packets without routing them.
Configure the interface to accept all traffic without responding to arp or multicast packets by running the following command as root:
ip link set IFACE down promisc on arp off multicast off
This command also brings the interface down if it was not down already, so that you are not inadvertently routing any data. Do not bring the interface up again until all steps are completed.
To make sure the interface will not have an IPv6 address automatically assigned, run the following commands:
sysctl -w net.ipv6.conf.IFACE.autoconf=0 sysctl -w net.ipv6.conf.IFACE.accept_ra=0
Check if the interface has any IPv4 or IPv6 addresses already:
ip address show IFACE
Remove all addresses listed starting with “inet" (IPv4) or “inet6" (IPv6) in the output from the above command (where IP is the address you want to remove):
ip address delete IP dev IFACE
Make sure the interface only respond to ARP requests for its own IP addresses (which it does not receive, so it will never respond):
sysctl -w net.ipv4.conf.IFACE.arp_ignore=1
Turn off reverse path filtering to ensure that the incoming packets are not dropped:
sysctl -w net.ipv4.conf.IFACE.rp_filter=0
Choose an empty routing table number so we can set up (no) routing specifically for the tunnel.
In this example, we use table number 200. Ensure that the table is empty, by running the following command:
ip route show table 200
Set up multiple interfaces on one system by using these steps they can all use the same table, as it will remain empty.
Create a routing table rule to have the kernel use the empty table to look up routing information for this interface:
ip rule add iif IFACE table 200
Check that the rule was added by running the following command:
ip rule show
Following is an example output:
0: from all lookup local
32765: from all iif IFACE lookup 200
32766: from all lookup main
32767: from all lookup default
If a firewall is active on the system, ensure that it is not filtering any packets coming through the interface. It is outside the scope of this document to explain how to do this as there are too many different firewall applications to list here. Here is a short list of pitfalls to take into account:
The firewall should be set up to totally ignore the interface, not set up to route everything to a single other interface in the GRE tunnel network. This is because any filtering causes CPU overhead, which can have a negative effect on throughput.
Any generic firewall rules, i.e. rules covering all interfaces can also apply to the interface currently being configured. These rules must be altered not to cover this interface, or new rules should be added to ignore this interface.
Some systems have network throttling enabled, this must be removed or turned off for the interface being configured, otherwise some packets of the copied or mirrored network may be dropped. How to change the configuration for network throttling fall outside the scope of this document. Though traffic shaping is used, you should be cautious with respect to the traffic shaping rules introduced in this document (also see,Configuring the Mirror, Configuring the Mirror, and Configuring a GRE Tunnel Using the tunnelctl Script).