High load average with low CCU

Status
Not open for further replies.

hoangsonk49

New Member
Sep 1, 2019
3
1
3
34
Hi everyone,
I'm using FreeSWITCH-1.6.20~64bit on Linux. We tested successfully on dev enviroment with 3000 CCU without any problem, but got issue on product server at 500 CCU.
- The scenario is very simple:
+ sipp send INVITE at 3000 CCU,
+ freeswitch play audio
+ after 10s, sipp send BYE to terminate the call

- On the dev server (16 core/20G RAM), the load avareage is about 0.2 at 3000 CCU (by SIPP)
1567474109940.png
1567474177251.png
- On the product server (16 core/32G RAM), the load average is getting higher very fast
1567474396596.png
And it cause some problem with freeswitch:
- The session is queued (it should be around 3000 CCU), the problem still happends at 500 CCU
1567474467518.png
- And there 's 5 seconds delayed between the time of receiving INVITE and processing. We do not see the problem with begining calls, but as time goes by, the delayed time is getting longer.
1567474841624.png

I think that we have missed something in the configuration of Freeswitch or OS but don't know exavtly what it is. We have already checked:
- ulimit
- max-session
- max call rate
- Ignore cdr-csv or xml

Please advise if you have any idea about this. I hereby attach the lua script running on freeswitch. Thank you very much.
 

Attachments

  • 1567474095063.png
    1567474095063.png
    7.6 KB · Views: 19
  • 1567475418548.png
    1567475418548.png
    21 KB · Views: 16
  • 1567475428480.png
    1567475428480.png
    21 KB · Views: 16
  • final.lua
    939 bytes · Views: 6

DigitalDaz

Administrator
Staff member
Sep 29, 2016
3,038
556
113
Your calls per second is way to high, no way you are getting close to that for any sustained period of time. Without lots of changes.
 

hoangsonk49

New Member
Sep 1, 2019
3
1
3
34
Hi DigitalDaz, the duration of each call is 10 seconds, so the call rate is about 300 calls per second (3000 CCU), and i think freeswitch is able to handle that. In the other hand, Freeswitch still working on dev enviroment at the same rate, but now on product server, it cannot handle even 100 CPS
 

DigitalDaz

Administrator
Staff member
Sep 29, 2016
3,038
556
113
Yep, I see that, have you tweaked php settings etc? I have never seen a freeswitch box do a sustained 100CPS and I have tried. Keep us posted on what you do get though as I'm very interested.
 

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,383
364
83
On the production server you are starting to go into swap even though you have more memory than the dev box. Do you have any idea why? Like DigitalDaz, I am also very interested.
 

KonradSC

Active Member
Mar 10, 2017
166
98
28
I think your testing method is not right. You are worried about CPU, however you need to be looking at your SIP retransmissions. Those are very high. From my understanding each SIP Profile runs as a single mod_sofia thread.

Slowly turn up your Calls Per Second rate in SIPP and look for retransmissions. Once you start seeing retransmissions you have become mod_sofia thread bound.

If you have devices registering to the same SIP Profile that will affect things too.

I typically think of it like "SIP Packets/Second/Profile".

This is the reason people use OpenSIPS/Kamalio for SIP and FreeSWITCH for RTP processing.
 
  • Like
Reactions: hoangsonk49

hoangsonk49

New Member
Sep 1, 2019
3
1
3
34
Thank you all , and very special thanks to KonradSC. As your suggestion, I have tested with differrent call rate level and it is running well at 2000 CCU. The problem is the high Calls Per Second in the previous testing, it caused the load average getting high and could not running at 2000 CCU, but now with the lower rate, it can running well . Thanks again.
 
  • Like
Reactions: KonradSC

netpro25

New Member
Apr 7, 2019
2
0
1
40
So I'm reviving this old thread. Have you guys ever tried the following sofia settings? I dug these up online. I wonder if this is a fix to allow a single sofia profile to handle more calls by enabling muli-threading.

Code:
<param name="max-reg-threads" value="8"/>
<param name="inbound-reg-in-new-thread" value="true"/>
 

ricofranco

New Member
Jun 8, 2018
10
0
1
45
So I'm reviving this old thread. Have you guys ever tried the following sofia settings? I dug these up online. I wonder if this is a fix to allow a single sofia profile to handle more calls by enabling muli-threading.

Code:
<param name="max-reg-threads" value="8"/>
<param name="inbound-reg-in-new-thread" value="true"/>
Hello did you test this parameter in witch freeswitch/fusion versions ???? Did you got good results ?
 
Status
Not open for further replies.