This appendix describes how to set up a GRE Ethernet (Layer 2) tunnel to a RUEI Collector Engine and how to use a tap with this configuration
Note:
Before attempting this procedure set up console access to the systems involved. This is required because issuing a wrong command can take a network offline, possibly severing any connection you have to the server. In that case console access is needed to repair the network.The RUEI User's Guide describes how to locate your installation within a network. GRE Tunnelling allows you to locate the Collector Engine anywhere in your network as long as the tunnel endpoints can communicate with each other. Note that while GRE tunnelling is efficient, the network throughput can decrease because of network throughput overhead caused by the additional headers added to the packets and the CPU time overhead caused by encapsulation and decapsulation of those packets.
GRE Ethernet tunneling has been supported in Linux since kernel version 2.6.28, and requires an up to date version of the iproute package containing the utilities (specifically the IP utility) to set up and configure GRE Ethernet tunnel (gretap) interfaces.
This procedure uses Oracle Linux 6.4 as a base for setting up the GRE Ethernet tunnels, as Oracle Linux version 6 provides a UEK kernel (Linux version 2.6.39-400.109.1.el6uek at time of writing) capable of setting up GRE Ethernet tunnels, as well as the correct version of the iproute package (iproute-2.6.32-23.el6.x86_64 for Oracle Linux 6.4) needed to add, delete or change GRE Ethernet tunnels.
Oracle Linux 5 is not supported even though it could support GRE Ethernet tunnels in its UEK kernel (Linux version 2.6.39-400.21.1.el5uek for OEL5.9) but needs a newer version (2.6.28 or higher) of the iproute package capable of setting up GRE Ethernet tunnels. Such a version of the iproute package is currently not officially supported and as such it is not covered by this procedure.
Installing and configuring a Collector is described in the Oracle Real User Experience Insight Installation Guide.
While this appendix contains details on various aspects of taps and GRE tunnels, the following outlines the process that must be completed:
Perform either the manual or scripted GRE setup described in Section B.2.2, "Manual Setup for Basic RUEI Tap and GRE Tunnel" and Section B.2.3, "Scripted Setup for Basic RUEI Tap and GRE Tunnel".
Make sure that the destination endpoint is only receiving traffic as described in Section B.2.4, "Making a Tunnel Unidirectional".
Add a tap as described in Section B.2.5, "Adding a Virtual Tap".
Configure the collector as described in Section B.3, "Configuring a Collector for GRE Tunnelling".
Test the setup as described in Section B.7, "Testing a GRE Tunnel".
Make sure that your configuration survives a reboot as described in Section B.10, "Making GRE Tunnel Environment Changes Permanent".
This section describes creating a single GRE tunnel, and how to set up either endpoint (assuming they are both Oracle Linux version 6 machines) to be able to aggregate either tap (source) traffic or GRE tunnel output (destination) traffic. With this process you can add one or many taps on one machine to the GRE tunnel, have the collector listen to one or many incoming GRE tunnels. The following topics are covered:
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
bridge - support for bridges
veth - support for virtual ethernet interfaces
This section describes setting up two Oracle Linux version 6 systems ”manually”, one as a source and the other as destination (with the RUEI Collector). It is an alternative process to Section B.2.3, "Scripted Setup for Basic RUEI Tap and GRE Tunnel"
To prepare both systems perform the following steps:
Install a GRE tunnel between the source and destination systems by following the instructions in Section B.5, "Configuring a GRE Tunnel Manually"
Install a bridge (BRTUN) on each of the source and destination systems by following the instruction in Section B.6, "Creating and Setting Up a Linux Bridge".
On both source and destination systems, add the local GRE tunnel endpoint interface (GRETUN) to the bridge (BRTUN) by following Section B.6.3, "Adding and Removing Bridge Interfaces".
By following Section B.7, "Testing a GRE Tunnel" you should be able to see generated test traffic from the source coming through the tunnel, both on the GRE tunnel interfaces as well as on the bridge interfaces on both ends.
Skip to section Section B.2.4, "Making a Tunnel Unidirectional".
This section describes setting up two Oracle Linux version 6 systems using a script, one as a source and the other as destination (with the RUEI Collector). It is an alternative process to Section B.2.2, "Manual Setup for Basic RUEI Tap and GRE Tunnel"
To prepare both systems use the tunnelctl script to create a bridged GRE tunnel as described in Section B.4, "Configuring a GRE Tunnel Using the tunnelctl Script".
At this point both the source and destination systems are set up, but no traffic is flowing through the bridges or the tunnel. Before we connect any taps to the tunnel source we need to make sure that the destination GRE tunnel endpoint (GRETUN on the destination system) can only receive traffic, not send any over the tunnel.
In effect we need to ensure the GRE tunnel is unidirectional as we only want to monitor traffic, not take part in it. We will use linux traffic shaping to block outgoing traffic for the GRETUN interface on the destination endpoint system.
Perform the following steps:
Select a handle (HANDLE) to be used for this qdisc, for example you could reuse the GRE tunnel id (ID)
Replace the root qdisc of GRETUN with one (prio) that can filter the outgoing traffic by entering the following command as root:
tc qdisc replace dev GRETUN parent root handle HANDLE: prio
Add a filter to pass all outgoing GRE traffic from the machine so that it does not get mirrored
tc filter add dev GRETUN parent HANDLE: \ protocol all prio 1 u32 \ match u32 0 0 flowid HANDLE:1 \ action drop
The GRE tunnel is now unidirectional. This can be tested by generating traffic on one system using ping and viewing it on the other using tcpdump as explained in section Section B.7, "Testing a GRE Tunnel", and then re-doing the test in the other direction. From source (tap) to destination (RUEI Collector) should be working, but from destination (RUEI Collector) to source (tap) should show no traffic at all.
We will now create a virtual tap for one of the local interfaces on the source system by performing the following steps:
Choose an interface (ETH) on the source system whose traffic you want to monitor on the destination RUEI system.
Create a virtual tap for the interface chosen in step 1, using the instructions in Section B.8, "Creating a Virtual Tap".
On the source system, add the created tap interface (TAP) to the bridge (BRTUN) using the instructions in Section B.6.3, "Adding and Removing Bridge Interfaces".
On the destination system, test the incoming GRE tunnel traffic by entering the following as root:
tcpdump -i GRETUN -c 100 -n
Note: as explained in Section B.8.4, "Testing the Tap", the traffic seen the bridge interface on the destination system should now also be the same as the traffic seen on the bridge interface on the source system.
Once have a GRE tunnel set up and tested, the collector can be configured to listen to the traffic on the GRE Tunnel. To enable the RUEI Collector Engine to listen to the GRE Ethernet tunnel:
Using RUEI (Configuration > Security > Collector profiles, note the collector profile that you want to configure. The default network based profile is named System network data collectors. If necessary create a new profile. In the following steps the chosen profile will be referred to as PROFILE.
Make sure the PROFILE Collector you want to configure is governed by the chosen Collector profile.
Log in to the RUEI Reporter system as the $RUEI_USER user.
Enter the following command:
execsql config_set_profile_value PROFILE config ForceInterface add greID
where
greID is the tunnel interface you created earlier.
PROFILE is the profile you choose in step 1 above.
The new configuration should now automatically propagate to the Collector. Note that the collector must be forced to listen to the interface since it is not a physical interface and thus lacks certain internal signals used by the collector to decide if the interface is up or down. The collector would otherwise not use the interface.
Make sure there is not a firewall filtering any packets coming through the interface. It is outside the scope of this document to explain how to perform this task but below is a short list of pitfalls to take into account:
The firewall should be set up to totally ignore the interface, not set up to route everything to a single other interface in the GRE tunnel network. This is because any filtering causes CPU overhead, which can have a negative effect on throughput.
Any generic firewall rules, that is rules covering all interfaces can also apply to the interface currently being configured, must be altered not to cover this interface. As a workaround, add new rules to ignore this interface.
Disable any network throttling that might affect the interface. It is outside the scope of this document to explain how to perform this task.
This section describes how to create a GRE tunnel using the tunnelctl script provided with RUEI. The following topics are covered:
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
bridge - support for bridges
veth - support for virtual ethernet interfaces
The two endpoints are able to reach each other (for example, tested using ping). The relevant ports must have been opened in any firewalls, both on the endpoints as well as on any router in between.
The two endpoints must have an executable copy of the tunnelctl script.
root user access is available on both endpoints.
Perform the following steps to set up the first endpoint:
Note the IP address of the local and remote endpoints.
Create a numeric Identifier (ID) to be used for both endpoints, for example 123. This ID will be used to identify the tunnel on both sides.
Log in as root using ssh and enter the following command:
tunnelctl create gre Local_IP Remote_IP ID
where
Local_IP is the address of the current server.
Remote_IP is the address of the remote server.
ID is the identifier you created in the previous step.
Check that the tunnel has been created:
tunnelctl list
An interface named greID should be listed.
To set up a tunnel both endpoints must be configured. The 'other' endpoint can be a switch or router capable of duplicating streams and sending them out through a GRE Ethernet tunnel, or it may be another Linux server where any duplication/streaming can be set up.
If the other endpoint is a router or switch capable of duplicating streams and sending them out through a GRE Ethernet tunnel, refer to the product documentation for any steps that might be necessary.
If the other endpoint is a Linux server, repeat the steps in Setting Up a Tunnel Endpoint on the second endpoint (noting that you need to reverse the local and remote IP addresses when creating the tunnel).
This section describes how to create a GRE tunnel manually. The following topics are covered:
The following components are required:
Two Oracle Linux version 6 system endpoints, with one endpoint set up as a RUEI Collector
On each system, the following packages must be installed:
iproute2
tcpdump
bridge-utils
On each system, the following kernel modules must be present and loaded:
ip_gre - support for GRE tunneling
The two endpoints are able to reach each other (for example, tested using ping). The relevant ports must have been opened in any firewalls, both on the endpoints as well as on any router in between.
The two endpoints must have an executable copy of the tunnelctl script.
root user access is available on both endpoints.
Perform the following steps to set up the first endpoint:
Note:
Do not bring the interface up until completing this procedure.Note the IP address of the local and remote endpoints.
Create a numeric Identifier (ID) to be used for both endpoints, for example 123. This ID will be used to identify the tunnel on both sides.
Log in as root using ssh and enter the following command to load the GRE modules in the Linux kernel:
modprobe ip_gre
Enter the following command to check that the GRE modules are loaded in the Linux kernel:
lsmod | grep gre
Log in as root using ssh and enter the following command:
ip link add ID type gretap local Local_IP remote Remote_IP
where
Local_IP is the address of the current server.
Remote_IP is the address of the remote server.
ID is the identifier you created in the step 2.
Check that the tunnel has been created:
ip link show
An interface named greID should be listed.
Configure the kernel not to route anything coming from the tunnel interface by performing the steps in Section B.9.1, "Configuring an Interface for Mirrored Traffic", taking care to swap IFACE with the interface name you are currently preparing (for example greID).
To set up a tunnel both endpoints must be configured. The 'other' endpoint can be a switch or router capable of duplicating streams and sending them out through a GRE Ethernet tunnel, or it may be another Linux server where any duplication/streaming can be set up.
If the other endpoint is a router or switch capable of duplicating streams and sending them out through a GRE Ethernet tunnel, refer to the product documentation for any steps that might be necessary.
If the other endpoint is a Linux server, repeat the steps in Setting Up a Tunnel Endpoint Manually on the second endpoint (noting that you need to reverse the local and remote IP addresses when creating the tunnel).
This section describes how to create and set up a linux bridge which will act as a layer 2 hub for mirrored data. You can add virtual taps and GRE tunnels to the bridge to create the required configuration. Setting up multiple bridges is also possible, but such a configuration is beyond the scope of this document. The following topics are covered:
The following components are required:
The following packages must be installed:
iproute2
tcpdump
bridge-utils
The following kernel module must be present and loaded:
bridge
Create a bridge by completing the following steps:
Log in as root using ssh and enter the following command:
brctl addbr BRTUN
where
BRTUN is the name of the bridge.
Enter the following command to check the bridge was created:
brctl show
Enter the following commands to configure the bridge to act as a (dumb) hub instead of a switch:
brctl setfd BRTUN 0brctl setageing BRTUN 0
Enter the following commands to configure the bridge to be silent :
brctl stp BRTUN off
Configure the kernel not to route anything coming from the bridge interface by performing the steps in Section B.9.1, "Configuring an Interface for Mirrored Traffic", taking care to swap IFACE with the interface name you are currently preparing (for example BRTUN>).
Enter the following commands to activate the bridge and set it to accept all traffic :
ip link set BRTUN promisc on arp off up
To add an interface (IFACE) to a bridge, enter the following:
brctl addif BRTUN IFACE
To remove an interface (IFACE) from a bridge, enter the following:
brctl delif BRTUN IFACE
At any time you can see the current configuration of the bridge by entering:
brctl show
Once have a GRE tunnel set up between two endpoints, and an interface for mirrored traffic to ensure that no mirrored traffic is routed on the linux (virtual) machine, an unused GRE Ethernet tunnel can be tested by running ping on one end and tcpdump on the other to see the GRE tunnel traffic. In the steps below the two endpoints are referred to as the source and the destination, where the source signifies the endpoint where ping is running, and the destination is where tcpdump is used to verify the traffic:
Make sure the GRE tunnel interface on either endpoint system is up, by entering the following command as root on both systems:
ip li set GRETUN up
Send ICMP packets through the tunnel, by entering the following command as root on the source system:
ping -I GRETUN 127.1.1.1
The IP address has specifically been chosen so that it does not get inadvertently routed anywhere, as it is a local address. Using ping -I means that the ICMP packets only get sent over the GRE tunnel, restricting the visibility to the destination endpoint.
Check that the GRE encapsulated tunnel traffic was received, by entering the following as root on the destination system, where ETH is the interface the tunnel is routed over (the local endpoint, typically eth0), not the tunnel interface itself.
tcpdump -i ETH -c 100 proto gre
You should see ARP and/or ICMP requests for the above IP address wrapped in GRE packets (GREv0) similar to the following:
… IP server_A > server_B: GREv0, length 46: ARP, Request who-has 127.1.1.1 tell server_A, length 28 … IP server_A > server_B: GREv0, length 102: IP server_A > 127.1.1.1: ICMP echo request, id 62057, seq 1, length 64
This section describes a generic method of creating a ”tap” network interface that will provide mirrored traffic from any other live interface on the Oracle Linux version 6 machine. This method uses linux traffic shaping to mirror incoming and outgoing data from an interface and copy that network traffic to a set of newly created virtual ethernet interfaces.
A set of two virtual ethernet interfaces are connected to each other in such a way that any data flowing into one will flow out of the other, in this sense they act as a virtual NIC cable. These virtual ethernet interfaces are commonly used in virtual networking.
Note that any local interface can be mirrored using this method, including the interface the controlling ssh connection and the interface carrying GRE tunnel traffic. This is possible because GRE traffic will be filtered out of any mirrored traffic by one of the traffic shaping rules in this chapter.
This procedure creates a pair of virtual interfaces, one called ”ETHmirror” and the other called ”ETHtap”. For example if you want to tap interface eth0, you first create a set of virtual interfaces called eth0mirror and eth0tap. The interfaces are named this way to help keep them apart from any other mirroring setups on the system, since this method allows us to mirror more than one local interface into the GRE tunnel. From now on we will reference them as ETH, MIRROR and TAP.
The ETH interface will have its traffic mirrored on the MIRROR interface. All traffic flowing through the MIRROR interface will also be seen on the TAP interface since they are a virtual ethernet pair, so that you can use that TAP interface in any network configuration (directly or in a bridge) that you want.
The following components are required:
The tap is to be created on an Oracle Linux version 6 system
The following packages must be installed:
iproute2
tcpdump
The following kernel module must be present and loaded:
veth - support for virtual ethernet interfaces
A live interface to be mirrored exists, this interface will be referred to from now on as ETH.
Complete the following steps to create the mirror/tap virtual interfaces:
Create a pair of virtual interfaces by entering the following command as root:
ip li ad TAP type veth peer name MIRROR
Activate the interfaces by entering the following command as root :
ip li set dev TAP up promisc on arp off ip li set dev MIRROR up promisc on arp off
Traffic shaping enables the copying of all incoming and outgoing traffic for a given interface (ETH) to the newly created MIRROR virtual interface. How traffic shaping works is not explained in this document, though individual steps will be annotated.
The mirror setup itself is simple, though you do need to add an extra filter to prevent any GRE traffic (packet type GREv0, see Testing a GRE Tunnel) from being mirrored. This must be done to ensure that if you are mirroring the interface the GRE tunnel is transported over, you will not force the GRE tunnel to carry its own traffic (a loop) as that would most certainly cause the network to fail, and the server to fail.
To mirror the incoming traffic:
Add an ingress qdisc to ETH by entering the following command as root:
tc qdisc add dev ETH ingress
Add a filter to pass all incoming GRE traffic to the machine so that it is not mirrored:
tc filter add dev ETH parent ffff: \
protocol all prio 1 u32 \
match ip protocol 47 0xff flowid 1:1 \
action pass
Add a filter to mirror all remaining traffic to our MIRROR interface:
tc filter add dev ETH parent ffff: \ protocol all prio 2 u32 \ match u32 0 0 flowid 1:2 \ action mirred egress mirror dev MIRROR
To mirror all outgoing traffic:
Replace the root qdisc of ETH with one (prio) that can filter the outgoing traffic by entering the following command as root:
tc qdisc replace dev ETH parent root handle 10: prio
Add a filter to pass all outgoing GRE traffic to the machine so that it is not mirrored:
tc filter add dev ETH parent 10: \
protocol all prio 1 u32 \
match ip protocol 47 0xff flowid 10:1 \
action pass
Add a filter to mirror all remaining traffic to our MIRROR interface:
tc filter add dev ETH parent 10: \ protocol all prio 2 u32 \ match u32 0 0 flowid 10:2 \ action mirred egress mirror dev MIRROR
Note:
If you deactivate the MIRROR interface after completing this procedure it will disrupt the ETH network traffic. Leave the MIRROR interface active, since you will only be using the TAP interface in the remaining setup, and that interface can be de-activated without any consequencesAt this point you have two new interfaces, MIRROR and TAP. The MIRROR is used by the traffic shaping rules to mirror the network traffic from ETH to, and TAP is the virtual interface counterpart of MIRROR. Leave MIRROR active from now on, and you can now freely use TAP in our networking setup, as long as you make sure it's data is not being routed by the system. To test whether it works look at the traffic on the TAP interface. That traffic should be the same as the traffic on ETH, minus the GRE traffic.
View the traffic on the TAP interface by entering the following command as root:
tcpdump -i TAP -c 100 -n
Compare the output of step 1 with the output of the following command, which is the traffic on ETH with the GRE traffic filtered out:
tcpdump -i ETH -c 100 -n ! proto gre
Note: To see the same output you should run both commands simultaneously. If you run the previous steps simultaneously you will probably see that the output does not line up, but after finding where they align you should see that they are the same.
This section describes how to ensure that the Linux kernel does not route or filter any packets going through a specific interface. The following topics are covered:
In the following steps IFACE denotes the interface that is being set up to accept any packets without routing them.
Configure the interface to accept all traffic without responding to arp or multicast packets by entering the following command as root:
ip link set IFACE down promisc on arp off multicast off
Note that the above command also brings the interface down if it was not down already, so that you are not inadvertently routing any data. Do not bring the interface up again until all steps are completed.
To make sure the interface will not have an IPv6 address automatically assigned, issue the following commands:
sysctl -w net.ipv6.conf.IFACE.autoconf=0 sysctl -w net.ipv6.conf.IFACE.accept_ra=0
Check if the interface has any IPv4 or IPv6 addresses already:
ip address show IFACE
Remove all addresses listed starting with ”inet” (IPv4) or ”inet6” (IPv6) in the output from the above command (where IP is the address you want to remove)
ip address delete IP dev IFACE
Make sure the interface only respond to ARP requests for its own IP addresses (which it does not receive, so it will never respond):
sysctl -w net.ipv4.conf.IFACE.arp_ignore=1
Turn off reverse path filtering to ensure that the incoming packets are not dropped:
sysctl -w net.ipv4.conf.IFACE.rp_filter=0
Choose an empty routing table number so we can set up (no) routing specifically for the tunnel.
In this example we use table number 200. Ensure that the table is empty using the following command:
ip route show table 200
Note that should you be setting up multiple interfaces on one system using these steps they can all use the same table, as it will remain empty.
Create a routing table rule to have the kernel use the empty table to look up routing information for this interface:
ip rule add iif IFACE table 200
Check that the rule was added by issuing the following command:
ip rule show
The output should look something like this:
0: from all lookup local
32765: from all iif IFACE lookup 200
32766: from all lookup main
32767: from all lookup default
If a firewall is active on the system, make that is it not filtering any packets coming through the interface. It is outside the scope of this document to explain how to do this as there are too many different firewall applications to list here, but below is a short list of pitfalls to take into account:
The firewall should be set up to totally ignore the interface, not set up to route everything to a single other interface in the GRE tunnel network. This is because any filtering causes CPU overhead, which can have a negative effect on throughput.
Any generic firewall rules, i.e. rules covering all interfaces can also apply to the interface currently being configured. These rules must be altered not to cover this interface, or new rules should be added to ignore this interface.
Some systems have network throttling enabled, this must be removed or turned off for the interface being configured, otherwise some packets of the copied/mirrored network may be dropped. How to change the configuration for network throttling fall outside the scope of this document. Though it should be noted that if traffic shaping is used, one should be very careful with respect to the traffic shaping rules introduced in this document (see also Section B.8.3, "Configuring the Mirror", Section B.8.3, "Configuring the Mirror", and also Section B.4, "Configuring a GRE Tunnel Using the tunnelctl Script").
When you are satisfied that the GRE tunnel configuration is working, create a boot script that executes the setup commands described in this appendix. The script should include items for:
GRE Ethernet tunnel creation
Firewall configuration
Network throttling