Test and load balancing of vzb incoming calls on our SIP servers
Christian Lathion, 2009-02-10
Switzernet
Test and load balancing of vzb incoming calls on our SIP servers
Testing all entry points of our network
Incoming call on the same proxy
Incoming call relayed by another proxy
Load balancing on all SIP servers
Incoming pstn calls arrive to our network from one vendor and are relayed by our OpenSER servers to our SIP proxies. The SIP proxies terminate the call to the end-user phone. This document presents tests and details on incoming calls termination. One of the goals was to eliminate the probability that a portion of our sip proxies present problems with incoming calls. It also presents a modification of the OpenSER servers configuration enabling load balancing of incoming calls.
The OpenSER servers have no registration information (on which SIP proxy a user is registered). Thus, INVITE packets are forwarded to one of our SIP proxies, not necessarily the proxy on which the customer is registered. The following servers/IP addresses are involved in the presented sample incoming calls:
An initial test of incoming call was performed on all currently active SIP proxies (fr1, fr2, fr3, fr4, dk1, us1). A modification was made to the OpenSER configuration to route a test number to a defined SIP server without impacting customers’ calls. Instead of the default configuration relaying all incoming calls through the fr4 SIP proxy:
rewritehost("91.121.75.124");
We used (once for each SIP proxy to be tested):
if($oU=~"^\+41215500306$") rewritehost("fr3.youroute.net");
else rewritehost("91.121.75.124");
The test calls did not show any problem. The following capture illustrates the behavior of an incoming call transmitted to fr1. Only signaling is shown, media will be discussed in the following section:
Capture of a test call on each SIP proxy (pcap format): [fr1] [fr2] [fr3] [fr4] [dk1] [us1]
All proxies exhibit the same behavior, no problem was encountered. In these captures, only the vendor <-> entry SIP proxy part of the call is visible. Now that we showed that incoming calls are working from all our SIP servers, we present more details on how they finally terminate to the customer phone. Next sections presents two test calls, once landing on the proxy on which the customer is registered, once on another proxy which has to relay the call.
The first case is when the OpenSER proxy relays the call directly to the “good” SIP proxy (i.e. the one on which the customer is registered). The following graph shows an incoming call relayed to the fr4 proxy, on which the customer is registered:
In this case, the proxy handles signaling and media between the vendor and customer. No other intermediary SIP proxies are involved.
In the more general case, the OpenSER will relay the call to a random SIP proxy, not corresponding to the customer’s registration. In the presented sample call, the INVITE is relayed to fr3, while the customer is registered on fr4. The fr3 proxy has to relay the call, but does not know on which SIP proxy it has to send it. To obtain routing information, the proxy issues a RADIUS request to the billing server. RADIUS requests are primarily used for authorization, authentication and accounting. In our case, they are also used for call routing. The reply from the billing server indicates to which vendor the call must be vendor. In this case, the route is the fr4 SIP proxy, on which the end-user phone is registered:
Cisco-AVPair: h323-ivr-in=PortaBilling_Routing: 41215500306@91.121.75.124;expires=300;credit-time=-1;forward-on-fail=yes;rtpp=3;moh=1;patience=20
Note that a call terminating outside of our network would receive several possible routes, corresponding to each vendor. Apart from this, processing does not change if the call stays in our network: all decisions related to the call (routing, authentication, accounting) are handled by the billing servers through RADIUS. Even if the SIP proxy could terminate the call by itself (if the customer phone is registered on it, or if the proxy had global registration information), it must query the billing server and will not handle routing by itself.
The following two captures, taken on fr3 and fr4, show the obtained call setup. Captures are not available for download, as they contain media sessions. Following the received routing information, the call is relayed to fr4. The media session does not pass through fr4, but remains on fr3:
Capture on the fr3 SIP proxy, receiving the call from the vendor’s side
The fr4 proxy acts as a relay between fr3 and the user phone. We see it does not handle the media (RTP) session:
Capture on the fr4 SIP proxy, on which the user is registered
For a more in-depth study on the media session, we describe the Session Description Protocol (SDP) of the concerned packets. All packets containing a SDP part are not discussed, only the minimal set initiating the media session. The indicated timings correspond to the two previous graphs taken on fr3 or fr4. There is a time shift of approx 300ms between the two captures:
9.940 [fr4]: The initial INVITE issued by the vendor pstn gateway is received on fr3. The request was relayed by the OpenSER proxy. The OpenSER does not handle media and thus the SDP is unchanged. The vendor gateway allocated port 16650 for the media session:
10.283 [fr4]: fr3 transmits the INVITE to fr4. fr3 allocated port 40756 for the media session:
12.051 [fr4]: fr4 relays the INVITE to the end-user SIP phone. SDP is not modified. The media session is still allocated to fr3 on port 40756:
14.647 [fr4]: 200 OK response issued by the customer is received on fr4. The customer is behind NAT, so fr4 receives a private IP address 192.168.1.172. The phone allocated port 16462 (this is the port used behind NAT, not the “public” port) for the media:
14.367 [fr3]: 200 OK is relayed from fr4 to fr3. The private IP address of the customer is replaced by its public IP 62.202.9.105. The port remains unchanged (before the media is initiated, the proxy cannot know which port will be used by the customer outside the NAT, and thus cannot correct it):
All required messages have now been processed. The media session can be established. The following table summarizes the legs composing the call. We recall that the port indicated for the customer does not correspond to the real port outside the NAT (the capture taken on fr3 shows that the used port is 59374):
Leg |
Party 1 |
Party 2 |
Vendor side |
147.188.127.1:16650 |
91.121.101.126:40758 |
Customer side |
91.121.101.126:40758 |
62.202.9.105:16462 |
Filtering the capture with the obtained source and destination ports effectively shows the packets. For example for the Vendor -> SIP proxy leg:
To obtain a better load distribution on our servers, we consider load balancing the incoming calls to all our SIP proxies. The previous experiments gave us an overview of the possible benefits. It would allow load balancing of the media session, providing a better bandwidth distribution between our proxies. The same applies to the CPU load required for media handling (which is low, since our servers do not transcode). Load balancing would not provide a better fault tolerance, since SIP signaling still has to transit between two proxies.
To enable load balancing, a simple modification of our OpenSER servers configuration is required. Our original configuration relays all incoming calls to a single SIP proxy:
rewritehost("91.121.75.124");
We replace it with the following code to distribute incoming calls in round-robin mode:
switch($var(incoming))
{
case 0:
rewritehost("91.121.66.202"); #fr1
$var(incoming)=1;
break;
case 1:
rewritehost("91.121.19.149"); #fr2
$var(incoming)=2;
break;
case 2:
rewritehost("91.121.101.126"); #fr3
$var(incoming)=3;
break;
case 3:
rewritehost("91.121.75.124"); #fr4
$var(incoming)=4;
break;
case 4:
rewritehost("82.103.128.3"); #dk1
$var(incoming)=5;
break;
case 5:
rewritehost("66.234.138.73"); #us1
$var(incoming)=0;
break;
}
xlog("L_INFO","Test LB: $oU $var(incoming)\n");
To see the effect of load balancing, we look at the INVITE packets relayed by the OpenSER server to our SIP proxies. We use the following command to obtain a compact output:
openser4:~# ngrep -pqlt -W byline "^INVITE" port 5060 and src 212.249.15.4 | grep -A1 "^U "
With the default configuration, all INVITE packets are relayed to the same SIP proxy, here 91.212.66.202. Phone numbers are filtered (XXXX) on the presented captures:
U 2009/02/16 12:01:29.435803 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:01:30.325173 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:01:32.310189 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:01:33.525623 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:01:54.291486 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4122550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:02:04.873136 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:02:13.356813 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4122550XXXX@91.121.66.202 SIP/2.0.
--
U 2009/02/16 12:02:15.234977 212.249.15.4:5060 -> 91.121.66.202:5060
INVITE sip:4121550XXXX@91.121.66.202 SIP/2.0.
When enabling the load balancing portion of the configuration, the relayed INVITE packets are dispatched between the SIP proxies:
U 2009/02/16 11:54:44.809505 212.249.15.4:5060 -> 91.121.101.126:5060
INVITE sip:4121550XXXX@91.121.101.126 SIP/2.0.
--
U 2009/02/16 11:54:48.772550 212.249.15.4:5060 -> 91.121.75.124:5060
INVITE sip:4121550XXXX@91.121.75.124 SIP/2.0.
--
U 2009/02/16 11:54:57.087011 212.249.15.4:5060 -> 91.121.19.149:5060
INVITE sip:4121550XXXX@91.121.19.149 SIP/2.0.
--
U 2009/02/16 11:55:05.848852 212.249.15.4:5060 -> 91.121.101.126:5060
INVITE sip:4121550XXXX@91.121.101.126 SIP/2.0.
--
U 2009/02/16 11:55:12.223585 212.249.15.4:5060 -> 91.121.75.124:5060
INVITE sip:4122550XXXX@91.121.75.124 SIP/2.0.
--
U 2009/02/16 11:55:18.649577 212.249.15.4:5060 -> 82.103.128.3:5060
INVITE sip:4121550XXXX@82.103.128.3 SIP/2.0.
--
U 2009/02/16 11:55:24.393722 212.249.15.4:5060 -> 82.103.128.3:5060
INVITE sip:4122550XXXX@82.103.128.3 SIP/2.0.
--
U 2009/02/16 11:55:36.991301 212.249.15.4:5060 -> 66.234.138.73:5060
INVITE sip:4121550XXXX@66.234.138.73 SIP/2.0.
Looking at the packets does not give a clear view of the load balancing, since the round-robin order is not visible (e.g. 82.103.128.3 receives two consecutive incoming calls). This is because each OpenSER thread uses its own internal version of the round-robin counter. In the logs, we see that the load balancing works as expected by examining the counter of each individual thread:
Feb 10 17:53:38 openser4 SER[31337]: Test LB: +41215502246 1
Feb 10 17:53:43 openser4 SER[31335]: Test LB: +41215502161 5
Feb 10 17:54:11 openser4 SER[31337]: Test LB: +41215504768 2
Feb 10 17:54:17 openser4 SER[31336]: Test LB: +41225500802 5
Feb 10 17:54:18 openser4 SER[31338]: Test LB: +41225502775 1
Feb 10 17:54:22 openser4 SER[31338]: Test LB: +41225500891 2
Feb 10 17:54:32 openser4 SER[31336]: Test LB: +41215501474 0
Feb 10 17:54:34 openser4 SER[31335]: Test LB: +41225500711 0
Feb 10 17:54:54 openser4 SER[31337]: Test LB: +41215503183 3
Feb 10 17:55:24 openser4 SER[31337]: Test LB: +41225500322 4
Feb 10 17:55:37 openser4 SER[31337]: Test LB: +41215501043 5
Feb 10 17:56:43 openser4 SER[31338]: Test LB: +41215503222 3
Feb 10 17:56:45 openser4 SER[31335]: Test LB: +41225501845 1
Feb 10 17:57:36 openser4 SER[31337]: Test LB: +41215504668 0
Feb 10 17:57:52 openser4 SER[31336]: Test LB: +41215503222 1
Feb 10 17:58:14 openser4 SER[31338]: Test LB: +41215500447 4
Feb 10 17:58:46 openser4 SER[31337]: Test LB: +41215503268 1
Feb 10 17:59:04 openser4 SER[31336]: Test LB: +41215502871 2
Lastly, looking at the network utilization (packets/sec) of the SIP proxy which initially received incoming calls shows a decrease after load balancing is enabled. The effect is slight, since our proxies mainly handle outgoing calls. The configuration for load balancing took place on 2009-02-10 at 17:30:
* * *