Astrad Version 13
André Guimarães, 2012-04-27
Switzernet
This Astrad
version requires version 7 of DBA.
The major changes between this version and Astrad v12 are: optimization of puppet installation, changes in DEBIAN repositories and correction of CDRs of some failed calls.
Changes and new functionalities
Asterisk Configuration files
Major changes in each of the configuration files are described next:
/etc/asterisk/extensions.conf
Changes in the layout.
Message “the next” was removed from the code that played it in follow me. When calling the support line for instance as we have several offline phones in the list it would play several consecutive times this message.
Added RouteNum field to custom CDR file (described bellow).
/etc/asterisk/sip.conf
Changed the user agent field to include the Astrad version.
/etc/asterisk/cdr_custom.conf
The order of the dates in this log was changed to: start, answer and end date.
Added duration of the call, which includes ringing time and speaking time.
Added a new field (RouteNum) which indicates which of the routes was used for the call. This will enable us to see if we have problems with certain destinations in vendors. If a call doesn’t exit by a vendor but is successful for another it might indicate a problem.
cat /var/log/asterisk/cdr-custom/Master.csv |grep -vE "1_0" |grep ANSWERED
zcat /var/log/asterisk/cdr-csv-old/* | grep -vE "1_0" |grep ANSWERED
/etc/logrotate.d/asterisk
Deleted inexistent files. When logrotate tried to read these files it would get an error.
Asterisk
Asterisk 1.8.7 sends the port 0 in the From and Contact fields of the SIP Notify message due to a bug and so only a few phones (the one which disregard both fields) answer to Asterisk. Asterisk keeps resending the Notify message until it reaches the maximum retry timeout (30 seconds). This problem is generating unnecessarily CPU and bandwidth usage. It keeps however the NAT open and so this isn’t a problem from the customer’s point of view. From the release notes any version superior to Asterisk 1.8.10.0 resolves this issue. [2]
To solve the problem a new version of asterisk was compiled. Astrad v13 uses now Asterisk v1.8.12.0. With this version instead of:
NOTIFY sip:412155XXX71@XXX.218.18.25:5060 SIP/2.0
Via: SIP/2.0/UDP 91.121.142.9:0;branch=z9hG4bK651daf78;rport
Max-Forwards: 70
From: "asterisk" <sip:asterisk@91.121.142.9:0>;tag=as4e217dab
To: <sip:412155XXX71@XXX.218.18.25:5060>
Contact: <sip:asterisk@91.121.142.9:0>
Call-ID: 57554bd36b2dc889207b5f033d93c846@91.121.142.9:0
CSeq: 102 NOTIFY
User-Agent: Astrad013
Date: Mon, 30 Apr 2012 12:19:45 GMT
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH
Supported: replaces, timer
Subscription-State: terminated
Event: keep-alive
Content-Length: 0
Asterisk now sends:
NOTIFY sip:412155XXX26@XXX.168.1.58:5070 SIP/2.0
Via: SIP/2.0/UDP
91.121.122.64:5060;branch=z9hG4bK7d97e9b8;rport
Max-Forwards: 70
From: "asterisk" <sip:asterisk@91.121.122.64>;tag=as60950f80
To: <sip:412155XXX26@XXX.168.1.58:5070>
Contact: <sip:asterisk@91.121.122.64:5060>
Call-ID: 2365819735992daa04ca7fc454190e21@91.121.122.64:5060
CSeq: 102 NOTIFY
User-Agent: Astrad013
Date: Mon, 30 Apr 2012 12:23:07 GMT
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH
Supported: replaces, timer
Subscription-State: terminated
Event: keep-alive
Content-Length: 0
This way each phone replies to the correct ip and port and there aren’t any retransmissions of Notify’s.
Asterisk sounds
Missing sounds were added as well as translations in Spanish and Portuguese.
Database
111201-update.sql
[diff]
The permissions for monitoring users in this file were corrected. Fields user and host were swapped. Also now both monitor servers have select permissions for the database.
Billing
/etc/astrad/script/ast-rad-acc.pl
A problem was detected with this version CDRs when the call failed. In some failure cases a CDR with maximum duration would be created. Upon further inspection of the failed billing Radius’ packet it was detected that the field used to defined the start date was Thu, 01 Jan 1970 00:00:00 GMT, the linux 0 seconds date. Also in these cases the account could not be attributed to any user due to a missing values and it would be inserted only in CDR_Vendors. Both problems are now corrected.
Another problem was later detected. The call duration was including some of the ringing time. The script that sends the billing Radius packets was using different dates in the “Stop answer” and the “Stop originate” packets. On the first, shown on the right bellow, it was using the time the call was being answered while on the other one it was using the time of the route dial. Apparently the “Stop originate” has priority over the “Stop answer” when calculating the call duration in Porta-SIP.
Now both messages send the same time and the duration checks with asterisk CDRs (ringing time differs sometimes 2 seconds). In the image bellow the asterisk CDR is shown in black as well as the CDR in Billing for the same call.
In both the talking time is 266 seconds (4:26) and the setup time is 15 seconds (281-266).
Puppet
[diff]
Puppet asterisk’s module was split in two: one installation module and one configuration module. In the configuration modules are all configuration files and Astrad scripts. The installation file manages the verification and installation of packages and verifies the permissions and existence of other needed files (like sound files). The verifications of these files takes much time (2~5 minutes) and causes an increase of CPU. Thus the installation module should be active only when installing a server. The configuration module takes less than 15 seconds and can be left active. However this module should also be inactive if there aren’t any changes as it causes small CPU peaks upon verification.
The astrad modules can now be called from a single astrad module that checks the variable Astrad version and installs/updates it. This behavior can be controlled by the use of variables permitting a replication across all servers or just in one for one or multiple versions.
The Debian repository was changed to “http://archive.debian.org/debian/ lenny” as now Lenny is in end of life and was removed from all public repositories.
Performance
When empty (without customers) there is a slight improvement. The pictures bellow shows the server before and after the migration. Only the control phone is registered. There are less peaks and the average is slightly lower.
Comparing two servers with customers the average is slightly lower. The server on the left has Astrad v11 and an average of 170 users. The one on the right has Astrad v13 and an average of 155 users. The lower CPU might be due to this customer difference.
The database size is decreased by 3. This is due to the removal of the field Deny and the merging of sippeers and sipusers into sipdevices. To compare the database size the following query was used:
mysql> SELECT table_schema "Data Base Name",
sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB"
FROM information_schema.TABLES
GROUP BY table_schema ;
|
Data taken from an Astrad v11 +--------------------+----------------------+ | Data Base Name | Data Base Size in MB | +--------------------+----------------------+ | asterisk | 17.48745728 | | information_schema | 0.00781250 | | mysql | 0.62737942 | +--------------------+----------------------+ |
Data taken from an Astrad v13 +--------------------+----------------------+ | Data Base Name | Data Base Size in MB | +--------------------+----------------------+ | asterisk | 5.51494598 | | information_schema | 0.00781250 | | mysql | 0.62831783 | +--------------------+----------------------+ |
The table bellow shows the average length of one row and the total data for the registration tables. sipdevices is only slightly larger than sippeers due to the additional field context. This means that each astrad only replicates now 13% of the previous amount of registration data. Also the total data used for registration has decreased 75%. The remaining context is mainly in multiple_ua.
|
mysql> SHOW TABLE STATUS LIKE 'sip%'; |
|
|
Data taken from an Astrad v11 +-----------+-------+----------------+-------------+ | Name | Rows | Avg_row_length | Data_length | +-----------+-------+----------------+-------------+ | sipconfig | 5 | 25 | 128 | | sippeers | 21602 | 76 | 1644540 | | sippeers2 | 21759 | 96 | 2105384 | | sipusers | 21559 | 541 | 11671424 | +-----------+-------+----------------+-------------+ |
Data taken from an Astrad v13 +-------------+-------+----------------+-------------+ | Name | Rows | Avg_row_length | Data_length | +-------------+-------+----------------+-------------+ | sipconfig | 5 | 25 | 128 | | sipdevices | 21602 | 80 | 1728448 | | sipdevices2 | 21604 | 100 | 2169492 | +-------------+-------+----------------+-------------+ |
The database should still be further
optimized by first removing all invalid accounts (
Problems
This section describes this version’s major problems.
The registration curve in the graph of one of the servers installed with astrad v13 was very unstable. This problem was solved by restarting asterisk after the installation in puppet and it is described in depth in this document: [1]
The replies to Notify’s sent by asterisk to keep the NAT, weren’t receive by Asterisk. Asterisk was sending in the fields “From” and “Contact” the port 0. Due to this the logs were getting filled and there were a lot of retransmissions. The problem was solved by migrating to a more recent version. Described in [Asterisk]
The CDR generated by this version was billing also ringing time. The problem was solved by correct the field h323-connect-time to have the same value in both the message “Sop originate” and “Stop answer”. Also when a call was not answered due to an error, the CDR had the maximum value for duration. This problem was also solved. These problems are described in detail in [Billing].
References
Master Mysql Astrad DBA V007
http://switzernet.com/3/public/120308-dba-v7
Master MySQL-Astrad versions (DBA)
http://switzernet.com/3/public/110317-db3-versions/
Astrad Versioning
http://switzernet.com/3/public/110126-astrad-versions/
List of functionalities to add to Astrad
http://switzernet.com/3/public/110523-astrad-wish-list/
Bug reports
[1] http://www.switzernet.com/3/public/120424-astrad-v13-reg-graph/
[2] http://downloads.asterisk.org/pub/telephony/asterisk/releases/ChangeLog-1.8.12.0-rc2