Background
For users behind the VMware SD-WAN Edge (that is user on the LAN side), one of the options for access Internet is local Internet breakout. When the SD-WAN Edge connected to more than one Internet link, questions like “which link will be picked for local breakout?”, “can the SD-WAN Edge perform traffic load balance?” come up. In this post, will dig out the behavior of VMware SD-WAN local Internet breakout when the SD-WAN Edge is connected with two Internet links.
Lab environment
In order to verify the content of this post is correct or not, tests will be conducted in a lab environment for verification. The lab is a closed environment (dark site), such that the Internet is a simulated Internet.
The following diagram shows the topology for the lab:
The topology consists of a VMware SD-WAN Orchestrator, also called Velocloud Orchestrator (VCO), with IP address 24.17.0.53. There are two VMware SD-WAN Gateway, also called Velocloud Gateway (VCG), they are vcg-40-sfpg01 (24.11.0.54) and vcg-40-sfpg02 (24.11.0.55).
There is a VMware Edge, also called Velocloud Edge (VCE), the name of this VCE is Edge-1. Edge-1 is connected to two Internet links on GE3 and GE4 with IP address 98.1.2.19 and 184.1.2.27 (In test 5.1, this IP is changed to 184.1.2.30) respectively. There are two Internet links for verify different situation with different characteristics of each Internet link. The LAN facing side of Edge-1 is GE2 with IP address 192.168.200.254. There is a client machine named Client100 with IP address 192.168.200.100 to generate traffic to test the local Internet breakout.
There are also two servers namely wordpress05.lab.local (43.254.254.14) and wordpress06.lab.local (24.12.0.14). These two servers are hosting some services, such as WordPress, iperf3 and SSH server.
In the Edge-1, Internet link on GE3 is intentionally configured as 5M (both up and down), while the Internet link on GE4 is intentionally configured as 10M (both up and down). The Internet links are configured to relatively small bandwidth because that will allow saturate the link easier during the test of load balancing or aggregation situation. The follow is the screen capture of the Monitor –> Overview of the Edge-1:
The software version
VCO: 4.0.1
VCG: R401-20201124-GA-53090-64ce7e02d2
VCE: R401-20201110-GA-MGMT-IP-e3fe6a0725
About the GREEN, YELLOW, RED threshold for the WAN link
The VMware SD-WAN is an overlay tunnel technology, this overlay tunnel is sometimes called Dynamic Multipath Optimization (DMPO), the tunnels are being checked continuously for the latency, jitter and packet loss. There are pre-defined threshold to classify the link as “GREEN”, “YELLOW” or “RED” based on different value of latency, jitter and packet loss. The pre-defined threshold can be found in this article: https://kb.vmware.com/s/article/2733094
The follow table is a re-cap of the threshold value for the reader’s reference:
In this post, the Internet link will describe as “GREEN”, “YELLOW” or “RED”, the matrix of different grade of the Internet link quality can be referred to Figure 3 above.
Some housekeeping before going into the tests and conclusions
In usual situation, the SD-WAN Edge gets assigned with at least two SD-WAN Gateways. In this lab test, the SD-WAN Edge, Edge-1, is specifically configured with Partner Gateway, such that only a single SD-WAN Gateway is assigned. This is because when introducing packet loss during the test, if there are multiple Gateways (which result of multiple tunnels), the packet loss will spread randomly to different tunnels so some tunnels get more packet loss while some tunnels get less packet loss. This will make setting the WAN link to the desired color more difficult. As a result, in this post, the Edge-1 is only configured with a single SD-WAN Gateway except in Test 5.1. In all the lab tests expect Test 5.1 in this post, the Edge-1 determines the packet loss, jitter and latency by measuring these three parameters with the SD-WAN Gateway, vcg-40-sfpg02 (24.11.0.55), over the overlay tunnel.
Local Internet breakout behavior
There are a few common questions about local Internet breakout, which will be addressed in this post:
- When use “Auto” link steering, will the VCE select the better link? Such as automatically select the GREEN link over RED link?
- When use “Auto” link steering, and every links are GREEN, which link will be picked?
- Is there any load balancing or link aggregation possible for local Internet breakout with multiple Internet WAN links?
- What happen for link steering “Preferred”, “Available” and “Mandatory” options?
- Is there any source IP persistence in “Auto” link steering?