500 Internal Server Error. from Yealink Phones on BLF Notify

Status
Not open for further replies.

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
Hello & Happy New Year!

Just in case this is affecting others. Today many of my Yealink phones, T46Ss and T27Gs have started returning a 500 Internal Server Error when they receive a notification message for a BLF.

When I find out what is happening, I will update this post
 
  • Like
Reactions: JamesBorne

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
Update: The issue is also affecting T23Gs and T21Ps that we have out there. Rebooting the phone solves the problem every time but it doesn't help me work out the cause...

Wondering if this could be a date/time struct issue in the phone related to the new year? Having said that we didn't see any issues in previous years but then we have had firmware updates.

Adrian.
 

screwloose

Member
Feb 5, 2017
49
9
8
40
HI did you ever sort this out? I have been seeing issues lately like this too on a t41s. Their are some notes about out of order sequence numbers on the notify packet that are causing it. I haven't found a fix yet.
 

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
HI did you ever sort this out? I have been seeing issues lately like this too on a t41s. Their are some notes about out of order sequence numbers on the notify packet that are causing it. I haven't found a fix yet.
Hi there, no I never worked out the cause. It started happening on New Years Day, so I assumed it was something to do with the way the phones handled date/time in the SIP stack. Rebooting the phones fixed it.

The notes you have found sound interesting, have you performed a packet capture so you can see for yourself if the CSeq are really out of sequence?
 

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
That sounds like fairly normal behavior.
There are a few exceptions, but generally, the CSeq header increases by 1 for each new request. CSeq helps an endpoint to identify out of sequence messages but also allows the endpoint to differentiate between re-transmissions where the CSeq will be the same and new requests where it will be different. A response to a request will have the same CSeq number as the CSeq in the request.
 

Claudio

New Member
Jan 3, 2018
18
0
1
41
So i have this issue across all T46G, S, T48S,T41 yealinks and have been fighting this issue for some time now well before new years. I have attached the screenshot of sngrep showing the multiple errors. No one has been able to explain why the BLF's will stop working over time. We need to get a solution to this problem. I have tried firmware from the latest beta back to the 66.82.0.10. Any suggestions would be greatly appreciated.


Claudio
 

Attachments

  • sip 500 park blf.png
    sip 500 park blf.png
    361.8 KB · Views: 20

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
@Claudio You are seeing exactly the behavior that I encountered at New Year, not just NOTIFYs from parking lots but all other extensions. However I have not seen a repeat of the problem since New Year. If I do encounter the problem again I will investigate further.

If you post a pcap trace, I (and I'm sure others) will look at it.
 

Claudio

New Member
Jan 3, 2018
18
0
1
41
So here is a screenshot of the SIP 500 error. If more info is needed i can provide that too.
 

Attachments

  • Screen Shot 2019-03-12 at 10.22.13 PM.png
    Screen Shot 2019-03-12 at 10.22.13 PM.png
    242.6 KB · Views: 15

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
From the screenshot I notice that the phone firmware is at version 66.83.0.10, the latest is 66.84.0.15. It maybe worth updating to see if this has any impact on the issue. All my phones are currently running the latest firmware.
 

Claudio

New Member
Jan 3, 2018
18
0
1
41
Yeah we were on the latest. I was told that Park BLF's were broken so I started downgrading until it was fixed. It improved briefly. I suspect the server may have been the issue. I just finished migrating to a newly built server on the master branch and updated Freeswitch. Lets hope that fixes it. I will report back. Thanks for looking


Claudio
 

agree

Member
Aug 26, 2018
135
24
18
Hey we're at the new year a year later @Adrian Fretwell I just noticed this issue on my server as well, all phones are responding to the NOTIFY with a 500 internal server error. id anyone found the cause?
 
  • Like
Reactions: JamesBorne

agree

Member
Aug 26, 2018
135
24
18
I just checked my packet captures, the last pcap file of 2019 it was still working fine the first pcap file of 2020 I already see the 500 error
 

agree

Member
Aug 26, 2018
135
24
18
I just spoke with a Freeswitch expert he said it's a known issue in Freeswitch. The CSeq in Freeswitch is based on the number of seconds elapsed from the beginning of the year. Since it is a new year, the CSeq from last year was higher than today, and the phone doesn't like it. rebooting the phones should fix it.
 
  • Like
Reactions: MTR

Adrian Fretwell

Well-Known Member
Aug 13, 2017
1,386
364
83
Hi @agree I have just done a packet trace, and yes I'm seeing the 500 internal error from many phones, mostly Yealinks but some Polycoms too. Looks like we need to do a mass phone reboot!
 

JamesBorne

Active Member
Jan 24, 2019
294
56
28
Australia
I've just heard from three separate customers/internal networks that their Yealink BLF aren't changing either!
I'll reboot my phones and see if that does it too. (Maybe this might need to be pinned for a month or so to prevent new topics being created @DigitalDaz?)
 

JamesBorne

Active Member
Jan 24, 2019
294
56
28
Australia
Useful script. I think it's also in the maintenance scripts on install?
*edit: Forgot to mention, click "Show All" if you have multiple tenants/domains.
I raise you a "2 click" reboot solution (if you exclude logging in and going to Status > Registrations) :)

1577986603515.png
 
Last edited:
  • Like
Reactions: Adrian Fretwell

MTR

Member
Oct 25, 2017
181
9
18
45
I just spoke with a Freeswitch expert he said it's a known issue in Freeswitch. The CSeq in Freeswitch is based on the number of seconds elapsed from the beginning of the year. Since it is a new year, the CSeq from last year was higher than today, and the phone doesn't like it. rebooting the phones should fix it.
thank you for that info
 
  • Like
Reactions: JamesBorne
Status
Not open for further replies.