Umberto Mautone added a comment - 05/Apr/12 1:55 PM This happens quite frequently under heavy load. Currently running on a box with roughly 300 CPS.

Jeff Lenk added a comment - 06/Apr/12 10:37 PM you have custom module here so It is very difficult to know whether the problem is really related to this call or something unrelated.

Umberto Mautone added a comment - 09/Apr/12 1:01 PM I had considered that so I replaced this:           switch_core_session_execute_application(session, "respond", "503 No Circuit Available"); ... with this:           switch_channel_hangup(chan, SWITCH_CAUSE_NORMAL_CIRCUIT_CONGESTION); ... and the issue disappeared.

Anthony Minessale II added a comment - 10/Apr/12 2:53 PM what happens if you only send "503" ? I'm trying to reproduce this but 600cps on sipp at a dp that uses this respond app wont trigger it for me.

Umberto Mautone added a comment - 10/Apr/12 7:43 PM Same thing happens. I had "respond" in a few places in my app and it would die randomly on any of them. It was never predictable and I wasn't able to reproduce it manually. It only seems to happen in production. The dump always showed it dying in Sofia and always at the same spot: #0 0x00002aaaab97af5f in su_block_add (home=0x2aaaf440c430, sub=0x2aaaf4c7e750, size=21, zero=do_malloc) a t su_alloc.c:338 338 while (b->sub_nodes[h].sua_data) { Replacing "respond" with switch_channel_hangup() made the problem disappear except that it has no way of sending responses SIP responses such as 402 (payment required).

Anthony Minessale II added a comment - 10/Apr/12 8:12 PM We do have a cause to sip code translation that takes place. Maybe we can add some new Fs causes for hangup app that translates to some of those sip codes. I wish I could pinpoint this.....

Umberto Mautone added a comment - 10/Apr/12 9:15 PM - edited That would be a good solution given its stability. The occurrence is so random. I haven't been able to discern a pattern or set of circumstances that lead to the crash. Turning on full logging would fill up available storage quickly @ 300cps+.

Pawel Pierscionek added a comment - 10/May/12 5:42 AM - edited same here. Am calling respond from dialplan. <action application="respond" data="403 Invalid Callerd ID number format"/> ore was generated by `/usr/local/freeswitch/bin/freeswitch -nonat -nc -hp -nocal -nort -nosql'. Program terminated with signal 11, Segmentation fault. #0 0x00002ac823fe9ca0 in pthread_mutex_lock () from /lib64/libpthread.so.0 (gdb) bt #0 0x00002ac823fe9ca0 in pthread_mutex_lock () from /lib64/libpthread.so.0 #1 0x00002aaaab93d6cf in su_alloc (home=0x2aab2c9f3c40, size=33) at su_alloc.c:880 #2 0x00002aaaab93ea0c in su_strdup (home=0x2aab2c9f3c40, s=0x2aaae5553d24 "Invalid Callerd ID number format") at su_strdup.c:55 #3 0x00002aaaab848118 in sofia_receive_message (session=<value optimized out>, msg=0x57936c50) at mod_sofia.c:2470 #4 0x00002ac8235e8c25 in switch_core_session_perform_receive_message (session=0x2aaae5690628, message=0x21,     file=<value optimized out>, func=<value optimized out>, line=<value optimized out>) at src/switch_core_session.c:698 #5 0x00002aaac8a6d19b in respond_function (session=0x2aaae5690628, data=0x2aaae5553d20 "403 Invalid Callerd ID number format")     at /usr/src/freeswitch/src/mod/applications/mod_dptools/mod_dptools.c:1218 #6 0x00002ac8235ea029 in switch_core_session_exec (session=0x2aaae5690628, application_interface=0x2dbabc8,     arg=0x2aaae5553d20 "403 Invalid Callerd ID number format") at src/switch_core_session.c:2275 #7 0x00002ac8235ea611 in switch_core_session_execute_application_get_flags (session=0x2aaae5690628,     app=0x2aaae5553d18 "respond", arg=<value optimized out>, flags=<value optimized out>) at src/switch_core_session.c:2160 #8 0x00002ac8235ed243 in switch_core_session_run (session=0x2aaae5690628) at src/switch_core_state_machine.c:213 #9 0x00002ac8235e7fd0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaae5690628)     at src/switch_core_session.c:1365 #10 0x00002ac823fe773d in start_thread () from /lib64/libpthread.so.0 #11 0x00002ac825105d1d in clone () from /lib64/libc.so.6

Anthony Minessale II added a comment - 10/May/12 10:56 AM you can't tack on a "me too" unless you reproduce with GIT HEAD and the line numbers show that you aren't. There is a change to sofia i did to try and deal with this.

Umberto Mautone added a comment - 10/May/12 11:17 AM I haven't had recurrence since updating a few versions ago.We can close this ticket unless Pawel can show it still happening with GIT HEAD.

Ken Rice added a comment - 21/May/12 10:08 AM Closing due to reporters request... This can be reopened if the other report contributor sees a need to do so

Pawel Pierscionek added a comment - 02/Jun/12 2:02 PM Still happening :(( It takes several days of 30CPS traffic for the bug to show up. FreeSWITCH Version 1.2.0 (git-25287f3 2012-05-28 19-21-06 +0000) Linux fs1 2.6.18-238.12.1.el5.028stab091.1 #1 SMP Wed Jun 1 13:20:25 MSD 2011 x86_64 x86_64 x86_64 GNU/Linux glibc-2.5-49.el5_5.4 ---------------------------------------- warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff6e920000 Core was generated by `/usr/local/freeswitch/bin/freeswitch -nonat -nc -nort -nocal -nosql'. Program terminated with signal 6, Aborted. #0 0x00002ba7d27d5265 in raise () from /lib64/libc.so.6 (gdb) bt #0 0x00002ba7d27d5265 in raise () from /lib64/libc.so.6 #1 0x00002ba7d27d6d10 in abort () from /lib64/libc.so.6 #2 0x00002ba7d280f84b in __libc_message () from /lib64/libc.so.6 #3 0x00002ba7d281730f in _int_free () from /lib64/libc.so.6 #4 0x00002ba7d281776b in free () from /lib64/libc.so.6 #5 0x00002aaaab6e690a in sub_alloc (home=0x2aaabff71980, sub=0x2aaabff72440, size=33, zero=do_malloc) at su_alloc.c:468 #6 0x00002aaaab6e6b00 in su_alloc (home=0x2aaabff71980, size=33) at su_alloc.c:880 #7 0x00002aaaab6e94fc in su_strdup (home=0x2aaabff71980, s=0x2aaabf36bd74 "Invalid Callerd ID number format") at su_strdup.c:55 #8 0x00002aaaab5f39e0 in sofia_receive_message (session=<value optimized out>, msg=0x4bbe3b80) at mod_sofia.c:2478 #9 0x00002ba7d0d58d9b in switch_core_session_perform_receive_message (session=0x2aaabe8705d8, message=0x2fae, file=<value optimized out>, func=<value optimized out>, line=<value optimized out>) at src/switch_core_session.c:721 #10 0x00002aaab081d40b in respond_function (session=0x2aaabe8705d8, data=0x2aaabf36bd70 "403 Invalid Callerd ID number format") at /usr/src/freeswitch/src/mod/applications/mod_dptools/mod_dptools.c:1242 #11 0x00002ba7d0d5a169 in switch_core_session_exec (session=0x2aaabe8705d8, application_interface=0x585bbd0, arg=0x2aaabf36bd70 "403 Invalid Callerd ID number format") at src/switch_core_session.c:2321 #12 0x00002ba7d0d5a751 in switch_core_session_execute_application_get_flags (session=0x2aaabe8705d8, app=0x2aaabf36bd68 "respond", arg=<value optimized out>, flags=<value optimized out>) at src/switch_core_session.c:2206 #13 0x00002ba7d0d5d6f3 in switch_core_session_run (session=0x2aaabe8705d8) at src/switch_core_state_machine.c:217 #14 0x00002ba7d0d5b050 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaabe8705d8) at src/switch_core_session.c:1411 #15 0x00002ba7d175a73d in start_thread () from /lib64/libpthread.so.0 #16 0x00002ba7d2878d1d in clone () from /lib64/libc.so.6 (gdb)

Anthony Minessale II added a comment - 02/Jun/12 7:42 PM do you have some sort of specific setup like a custom app or test case?

Pawel Pierscionek added a comment - 04/Jun/12 1:25 AM hmm, Just two identical SUN servers with CentOS 5, and FS running in OpenVZ container. Same thing happens to both instances on both servers. A simple dialplan with a bit of lua for setting vars and memcache for longest matching prefix decision . Am calling respond app quite a lot when caller id does not match some regexps. I have like 5-10 respond calls a second. No test case yet. I'll setup some SIPP scenario and try to recreate the thing.

Umberto Mautone added a comment - 11/Jun/12 12:25 PM Just happened again to me as well. Commit => 2983003f95bc7d2f3949ef4146b0c809a70d5fdd #0 0x0000003cb4808e45 in __pthread_mutex_lock_full () from /lib64/libpthread.so.0 #1 0x00007f4fcc84d99f in su_alloc (home=0x7f4ca3245380, size=17) at su_alloc.c:880 #2 0x00007f4fcc845d4b in su_strdup (home=0x7f4ca3245380, s=0x7f4f793b20e4 "Payment Required") at su_strdup.c:55 #3 0x00007f4fcc752417 in sofia_receive_message (session=<value optimized out>, msg=<value optimized out>) at mod_sofia.c:2480 #4 0x00007f4fce4089d0 in switch_core_session_perform_receive_message (session=0x7f4cabf05be8, message=0x7f4f793b1df0,     file=<value optimized out>, func=<value optimized out>, line=<value optimized out>) at src/switch_core_session.c:721 #5 0x00007f4fb39b4ef7 in respond_function (session=<value optimized out>, data=<value optimized out>)     at /root/freeswitch/freeswitch/src/mod/applications/mod_dptools/mod_dptools.c:1242 #6 0x00007f4fce40ac3a in switch_core_session_exec (session=0x7f4cabf05be8, application_interface=0x7cd8d0,     arg=0x7f4f793b20e0 "402 Payment Required") at src/switch_core_session.c:2321 #7 0x00007f4fce40b172 in switch_core_session_execute_application_get_flags (session=0x7f4cabf05be8, app=0x7f4fb01d3ab9 "respond",     arg=0x7f4f793b20e0 "402 Payment Required", flags=<value optimized out>) at src/switch_core_session.c:2206 #8 0x00007f4fb01d0dcd in postreport (session=0x7f4cabf05be8, result=4)     at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1407 #9 endcall (session=0x7f4cabf05be8, result=4) at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1451 #10 0x00007f4fb01d2576 in mod_routing_app_function (session=0x7f4cabf05be8, data=<value optimized out>)     at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1332 #11 0x00007f4fce40ac3a in switch_core_session_exec (session=0x7f4cabf05be8, application_interface=0x87c488, arg=0x7f4fce5400ca "")     at src/switch_core_session.c:2321 #12 0x00007f4fce40b172 in switch_core_session_execute_application_get_flags (session=0x7f4cabf05be8, app=0x7f4ca94ebbb0 "routing",     arg=0x7f4fce5400ca "", flags=<value optimized out>) at src/switch_core_session.c:2206 #13 0x00007f4fce40d9b4 in switch_core_standard_on_execute (session=0x7f4cabf05be8) at src/switch_core_state_machine.c:217 #14 switch_core_session_run (session=0x7f4cabf05be8) at src/switch_core_state_machine.c:440 #15 0x00007f4fce40a53e in switch_core_session_thread (thread=<value optimized out>, obj=0x7f4cabf05be8)     at src/switch_core_session.c:1411 #16 0x0000003cb48077f1 in start_thread () from /lib64/libpthread.so.0 #17 0x0000003cb40e5ccd in clone () from /lib64/libc.so.6

Anthony Minessale II added a comment - 11/Jun/12 1:05 PM do you have this core where I can ssh and poke at it myself?

Git added a comment - 11/Jun/12 1:21 PM Repository: freeswitch Branch: refs/heads/master Commit: 56325e8 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=56325e8 Updated By: anthm@freeswitch.org Comment: FS-4079 possible fix FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Umberto Mautone added a comment - 13/Jun/12 12:50 PM Updated to current GIT to test the fix. Will advise in a few days as the occurrence isn't predictable. You have access to the server, per my email. I'm not sure if that core is still available, though, or if it would still be useful after the code was updated.

Git added a comment - 19/Jun/12 11:02 AM Repository: freeswitch Branch: refs/heads/ FS-3432 Commit: 56325e8 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=56325e8 Updated By: d@d-man.org Comment: FS-4079 possible fix FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Anthony Minessale II added a comment - 27/Jun/12 11:39 AM Please try latest: commit b0f53cd62593db9376cea1de24a96840479b08c7 Author: Anthony Minessale < anthm@freeswitch.org > Date: Wed Jun 27 09:32:54 2012 -0500      FS-4353 --resolve this may also fix FS-4079

Peter Olsson added a comment - 11/Jul/12 10:13 AM Umberto - have you tried if this is resolved in latest git? /Peter

Umberto Mautone added a comment - 18/Jul/12 10:39 AM - edited It didn't fix the issue. Two crashes at "respond" an hour apart from each other yesterday using git 9ca929218fafa1fe4271693c16f3dbe05c884730 ==== Core was generated by `/usr/local/freeswitch/bin/freeswitch -nc -nosql -nonat'. Program terminated with signal 11, Segmentation fault. #0 nua_server_request_is_pending (session=0x2aaad0abc578, msg=0x45ba8d20) at /root/freeswitch/freeswitch/libs/sofia-sip/libsofia-sip-ua/nua/nua_server.h:161 161 return sr && sr->sr_response.msg; (gdb) bt #0 nua_server_request_is_pending (session=0x2aaad0abc578, msg=0x45ba8d20) at /root/freeswitch/freeswitch/libs/sofia-sip/libsofia-sip-ua/nua/nua_server.h:161 #1 sofia_receive_message (session=0x2aaad0abc578, msg=0x45ba8d20) at mod_sofia.c:2419 #2 0x00002b316916497b in switch_core_session_perform_receive_message (session=0x2aaad0abc578, message=0x0, file=<value optimized out>, func=<value optimized out>, line=<value optimized out>)     at src/switch_core_session.c:722 #3 0x00002aaab822c483 in respond_function (session=0x2aaad0abc578, data=0x45ba9140 "402 Payment Required") at /root/freeswitch/freeswitch/src/mod/applications/mod_dptools/mod_dptools.c:1244 #4 0x00002b3169165d49 in switch_core_session_exec (session=0x2aaad0abc578, application_interface=0x7d354f0, arg=0x45ba9140 "402 Payment Required") at src/switch_core_session.c:2332 #5 0x00002b31691663a1 in switch_core_session_execute_application_get_flags (session=0x2aaad0abc578, app=0x2aaabba309b9 "respond", arg=<value optimized out>, flags=<value optimized out>)     at src/switch_core_session.c:2207 #6 0x00002aaabba2de5e in postreport (session=0x2aaad0abc578, result=<value optimized out>) at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1424 #7 endcall (session=0x2aaad0abc578, result=<value optimized out>) at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1468 #8 0x00002aaabba2ef93 in mod_routing_app_function (session=0x2aaad0abc578, data=<value optimized out>) at /root/freeswitch/freeswitch/src/mod/applications/mod_routing/mod_routing.c:1349 #9 0x00002b3169165d49 in switch_core_session_exec (session=0x2aaad0abc578, application_interface=0x2aaab010f4b8, arg=0x2b3169293aaf "") at src/switch_core_session.c:2332 #10 0x00002b31691663a1 in switch_core_session_execute_application_get_flags (session=0x2aaad0abc578, app=0x2aaad082ae10 "routing", arg=<value optimized out>, flags=<value optimized out>)     at src/switch_core_session.c:2207 #11 0x00002b3169169343 in switch_core_session_run (session=0x2aaad0abc578) at src/switch_core_state_machine.c:217 #12 0x00002b3169166ca0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaad0abc578) at src/switch_core_session.c:1412 #13 0x00000038afa0677d in start_thread () from /lib64/libpthread.so.0 #14 0x00000038af2d325d in clone () from /lib64/libc.so.6

Git added a comment - 18/Jul/12 9:48 PM Repository: freeswitch Branch: refs/heads/master Commit: 5867d04 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=5867d04 Updated By: anthm@freeswitch.org Comment: FS-4079 please try this FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Peter Olsson added a comment - 28/Jul/12 7:50 AM Umberto, Did you try this? /Peter

Umberto Mautone added a comment - 28/Jul/12 10:21 PM Yes but it's not a predictable bug. It can sometimes run for weeks before crashing and then may crash a few times in a row. I'd like to run this for a few more weeks before closing it.

Matthew Williams added a comment - 04/Aug/12 1:57 PM I'm having a similar problem with the respond app on 1.2.0-rc2; git at commit e523076274 on Fri, 03 Aug 2012 19:45:09 Crashed twice at seemingly random. Using the respond app. I have this one in a VZ container also Crash one #0 0x00002aaaab50b3b7 in su_block_add (home=0x2aaab955a330, sub=0x2aaab933d100, size=8, zero=do_malloc) at su_alloc.c:339 339 h = (size_t)((uintptr_t)p % b->sub_n); (gdb) bt #0 0x00002aaaab50b3b7 in su_block_add (home=0x2aaab955a330, sub=0x2aaab933d100, size=8, zero=do_malloc) at su_alloc.c:339 #1 sub_alloc (home=0x2aaab955a330, sub=0x2aaab933d100, size=8, zero=do_malloc) at su_alloc.c:525 #2 0x00002aaaab50c2a0 in su_alloc (home=0x2aaab955a330, size=8) at su_alloc.c:885 #3 0x00002aaaab505f9c in su_strdup (home=0x2aaab955a330, s=0x2aaacd2c21e8 "Decline") at su_strdup.c:55 #4 0x00002aaaab40b40e in sofia_on_hangup (session=0x2aaacd2d9a58) at mod_sofia.c:543 #5 0x00002b8127921be3 in switch_core_session_hangup_state (session=0x2aaacd2d9a58, force=<value optimized out>) at src/switch_core_state_machine.c:625 #6 0x00002b8127923612 in switch_core_session_run (session=0x2aaacd2d9a58) at src/switch_core_state_machine.c:415 #7 0x00002b81279212c0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaacd2d9a58) at src/switch_core_session.c:1412 #8 0x00002b812854977d in start_thread () from /lib64/libpthread.so.0 #9 0x00002b812966625d in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab50b3b7 in su_block_add (home=0x2aaab955a330, sub=0x2aaab933d100, size=8, zero=do_malloc) at su_alloc.c:339         h = 0         probe = 18446744073709551615 #1 sub_alloc (home=0x2aaab955a330, sub=0x2aaab933d100, size=8, zero=do_malloc) at su_alloc.c:525         sua = <value optimized out>         data = 0x2aaac8f4b4d0         preload = 0x0         __PRETTY_FUNCTION__ = "sub_alloc" #2 0x00002aaaab50c2a0 in su_alloc (home=0x2aaab955a330, size=8) at su_alloc.c:885         data = <value optimized out> #3 0x00002aaaab505f9c in su_strdup (home=0x2aaab955a330, s=0x2aaacd2c21e8 "Decline") at su_strdup.c:55         n = <value optimized out>         retval = 0x0 #4 0x00002aaaab40b40e in sofia_on_hangup (session=0x2aaacd2d9a58) at mod_sofia.c:543         reason = "Q.850;cause=21;text=\"CALL_REJECTED\"", '\000' <repeats 92 times>         bye_headers = 0x0         val = 0x0         max_forwards = 0x2aaacd2c2288 "69"         a_session = <value optimized out>         tech_pvt = 0x2aaacd06c938         channel = 0x2aaacd2df618         cause = SWITCH_CAUSE_CALL_REJECTED         sip_cause = 603         ps_cause = <value optimized out>         use_my_cause = <value optimized out>         gateway_name = <value optimized out>         gateway_ptr = <value optimized out>         __func__ = "sofia_on_hangup"         __PRETTY_FUNCTION__ = "sofia_on_hangup" #5 0x00002b8127921be3 in switch_core_session_hangup_state (session=0x2aaacd2d9a58, force=<value optimized out>) at src/switch_core_state_machine.c:625         cause = SWITCH_CAUSE_CALL_REJECTED         cause_q850 = SWITCH_CAUSE_CALL_REJECTED         proceed = <value optimized out>         global_proceed = <value optimized out>         state = CS_HANGUP         endpoint_interface = <value optimized out>         driver_state_handler = 0x2aaaab7936e0         application_state_handler = <value optimized out>         hook_var = <value optimized out>         use_session = <value optimized out>         __func__ = "switch_core_session_hangup_state"         __PRETTY_FUNCTION__ = "switch_core_session_hangup_state" #6 0x00002b8127923612 in switch_core_session_run (session=0x2aaacd2d9a58) at src/switch_core_state_machine.c:415         global_proceed = 1         proceed = <value optimized out>         ptr = 0x0         rstatus = <value optimized out>         state = CS_HANGUP         endstate = CS_NEW         endpoint_interface = <value optimized out>         driver_state_handler = 0x2aaaab7936e0         application_state_handler = <value optimized out>         new_loops = 59999         __PRETTY_FUNCTION__ = "switch_core_session_run"         __func__ = "switch_core_session_run" #7 0x00002b81279212c0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaacd2d9a58) at src/switch_core_session.c:1412         session = 0x2aaacd2d9a58         event = <value optimized out>         event_str = 0x0         val = <value optimized out>         __func__ = "switch_core_session_thread"         __PRETTY_FUNCTION__ = "switch_core_session_thread" #8 0x00002b812854977d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00002b812966625d in clone () from /lib64/libc.so.6 No symbol table info available. Crash Two Program terminated with signal 11, Segmentation fault. #0 0x00002ac3ce64dd10 in pthread_mutex_lock () from /lib64/libpthread.so.0 (gdb) bt #0 0x00002ac3ce64dd10 in pthread_mutex_lock () from /lib64/libpthread.so.0 #1 0x00002aaaab50c28f in MEMLOCK (home=0x2aaab3175340, size=8) at su_alloc.c:213 #2 su_alloc (home=0x2aaab3175340, size=8) at su_alloc.c:885 #3 0x00002aaaab505f9c in su_strdup (home=0x2aaab3175340, s=0x2aaabe5c3cd0 "Decline") at su_strdup.c:55 #4 0x00002aaaab40b40e in sofia_on_hangup (session=0x2aaabd93b3b8) at mod_sofia.c:543 #5 0x00002ac3cda23be3 in switch_core_session_hangup_state (session=0x2aaabd93b3b8, force=<value optimized out>) at src/switch_core_state_machine.c:625 #6 0x00002ac3cda25612 in switch_core_session_run (session=0x2aaabd93b3b8) at src/switch_core_state_machine.c:415 #7 0x00002ac3cda232c0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaabd93b3b8) at src/switch_core_session.c:1412 #8 0x00002ac3ce64b77d in start_thread () from /lib64/libpthread.so.0 #9 0x00002ac3cf76825d in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002ac3ce64dd10 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00002aaaab50c28f in MEMLOCK (home=0x2aaab3175340, size=8) at su_alloc.c:213 No locals. #2 su_alloc (home=0x2aaab3175340, size=8) at su_alloc.c:885         data = <value optimized out> #3 0x00002aaaab505f9c in su_strdup (home=0x2aaab3175340, s=0x2aaabe5c3cd0 "Decline") at su_strdup.c:55         n = <value optimized out>         retval = 0x0 #4 0x00002aaaab40b40e in sofia_on_hangup (session=0x2aaabd93b3b8) at mod_sofia.c:543         reason = "Q.850;cause=21;text=\"CALL_REJECTED\"", '\000' <repeats 92 times>         bye_headers = 0x0         val = 0x0         max_forwards = 0x2aaabe5c3d70 "66"         a_session = <value optimized out>         tech_pvt = 0x2aaabeaa8c48         channel = 0x2aaabd940f78         cause = SWITCH_CAUSE_CALL_REJECTED         sip_cause = 603         ps_cause = <value optimized out>         use_my_cause = <value optimized out>         gateway_name = <value optimized out>         gateway_ptr = <value optimized out>         __func__ = "sofia_on_hangup"         __PRETTY_FUNCTION__ = "sofia_on_hangup" #5 0x00002ac3cda23be3 in switch_core_session_hangup_state (session=0x2aaabd93b3b8, force=<value optimized out>) at src/switch_core_state_machine.c:625         cause = SWITCH_CAUSE_CALL_REJECTED         cause_q850 = SWITCH_CAUSE_CALL_REJECTED         proceed = <value optimized out>         global_proceed = <value optimized out>         state = CS_HANGUP         endpoint_interface = <value optimized out>         driver_state_handler = 0x2aaaab7936e0         application_state_handler = <value optimized out>         hook_var = <value optimized out>         use_session = <value optimized out>         __func__ = "switch_core_session_hangup_state"         __PRETTY_FUNCTION__ = "switch_core_session_hangup_state" #6 0x00002ac3cda25612 in switch_core_session_run (session=0x2aaabd93b3b8) at src/switch_core_state_machine.c:415         global_proceed = 1         proceed = <value optimized out>         ptr = 0x0         rstatus = <value optimized out>         state = CS_HANGUP         endstate = CS_NEW         endpoint_interface = <value optimized out>         driver_state_handler = 0x2aaaab7936e0         application_state_handler = <value optimized out>         new_loops = 59999         __PRETTY_FUNCTION__ = "switch_core_session_run"         __func__ = "switch_core_session_run" #7 0x00002ac3cda232c0 in switch_core_session_thread (thread=<value optimized out>, obj=0x2aaabd93b3b8) at src/switch_core_session.c:1412         session = 0x2aaabd93b3b8         event = <value optimized out>         event_str = 0x0         val = <value optimized out>         __func__ = "switch_core_session_thread"         __PRETTY_FUNCTION__ = "switch_core_session_thread" #8 0x00002ac3ce64b77d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00002ac3cf76825d in clone () from /lib64/libc.so.6 No symbol table info available. (gdb)

Matthew Williams added a comment - 04/Aug/12 2:47 PM I've found some commonality in both of my crashes. According to the sip traces a Cancel is set < 100ms after the initial invite. I bet there is a race between cancel and respond here.

Matthew Williams added a comment - 04/Aug/12 5:17 PM I now have 4 examples of it happening today all with the same type of call.

William King added a comment - 04/Aug/12 9:43 PM I was able to reproduce Umberto Mautone's crash in nua_server_request_is_pending once with sipp and a static dialplan. Attempting additional reproductions.

William King added a comment - 04/Aug/12 9:47 PM Ok, I was able to reproduce it again with sipp. Tony, I have this setup on a test box. The last lines I see when the console log level is set to err is: 2012-08-04 22:45:37.385540 [CONSOLE] sofia.c:1381 MSG Thread 2 Started 2012-08-04 22:45:37.385540 [CONSOLE] sofia.c:1381 MSG Thread 1 Started Segmentation fault (core dumped) and that crashed after ~8000 calls.

William King added a comment - 04/Aug/12 11:12 PM Ok, after an update(I checked out older code to reproduce the crash that Umberto reported) I have now reproduced the crash Matthew Williams reported using the same technique.

William King added a comment - 05/Aug/12 1:33 AM The current problem is a numerical exception specifically a divide by zero. On su_alloc.c:339 here: h = (size_t)((uintptr_t)p % b->sub_n); b->sub_n is 0. Tracing b back to su_alloc.c:885 the problem is that MEMLOCK(home) is not fetching the requested space. This is the source of both of the crashes reported recently by Matthew Williams. Now why MEMLOCK isn't able to reserve the space, or why sofia doesn't check first to make sure that MEMLOCK actually returned the space I'm not sure.

William King added a comment - 05/Aug/12 1:50 AM In mod_sofia.c:543 phrase = su_strdup(nua_handle_home(tech_pvt->nh), tech_pvt->respond_phrase); The su_home_t returned by nua_handle_home(tech_pvt->nh) should be checked for something like: su_home_t *home = nua_handle_home(tech_pvt->nh); if ( home->suh_blocks->sub_n == 0 ) {    /* Allocate more blocks for the su_home_t */ } Still working on a patch with little luck yet.

Matthew Williams added a comment - 05/Aug/12 2:39 PM I added a sleep 150 before calling the respond app and this box has been up for a day now.

Git added a comment - 06/Aug/12 4:11 PM Repository: freeswitch Branch: refs/heads/master Commit: c3de963 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=c3de963 Updated By: anthm@freeswitch.org Comment: FS-4079 please test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 07/Aug/12 12:06 PM Okay, after removing the respond app completely from my dial plan it's still crashing fairly consistently. Below is with the respond app removed from the dialplan. The BT after that is the latest with the respond fix. #0 0x00002aaaab50737b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 231 base->sup_vtable->su_port_decref(self, 0, who); (gdb) bt #0 0x00002aaaab50737b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 #1 su_msg_destroy (rmsg=<value optimized out>) at su_root.c:1191 #2 0x00002aaaab4343f7 in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1520 #3 0x00002aaaab42e950 in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aab505bbd36 "Trying", nua=0xdf64650, profile=0xdf1c8a0,     nh=0x2aab506a4bd0, sofia_private=0x2aaae3f98050, sip=0x0, de=0x2aab286597c0, tags=0x2aab505bb880) at sofia.c:971 #4 0x00002aaaab4343ef in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1517 #5 0x00002aaaab434528 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1566 #6 0x00002add462ad77d in start_thread () from /lib64/libpthread.so.0 #7 0x00002add473ca25d in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab50737b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 No locals. #1 su_msg_destroy (rmsg=<value optimized out>) at su_root.c:1191         msg = 0x2aaae38911b0         __PRETTY_FUNCTION__ = "su_msg_destroy" #2 0x00002aaaab4343f7 in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1520         de = 0x2aab28235a60         nh = 0x2aab506a4bd0         nua = 0xdf64650         profile = 0xdf1c8a0 #3 0x00002aaaab42e950 in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aab505bbd36 "Trying", nua=0xdf64650, profile=0xdf1c8a0,     nh=0x2aab506a4bd0, sofia_private=0x2aaae3f98050, sip=0x0, de=0x2aab286597c0, tags=0x2aab505bb880) at sofia.c:971         qde = 0x0         tech_pvt = <value optimized out>         auth_res = <value optimized out>         session = <value optimized out>         channel = <value optimized out>         gateway = <value optimized out>         locked = <value optimized out>         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #4 0x00002aaaab4343ef in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1517         de = 0x2aab286597c0         nh = 0x2aab506a4bd0         nua = 0xdf64650         profile = 0xdf1c8a0 #5 0x00002aaaab434528 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1566         de = 0x0         pop = 0x2aab286597c0         q = 0x2aaaab85aad8         my_id = <value optimized out>         __func__ = "sofia_msg_thread_run" #6 0x00002add462ad77d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00002add473ca25d in clone () from /lib64/libc.so.6 No symbol table info available. (gdb) Crash from the latest Program terminated with signal 11, Segmentation fault. #0 0x00002aaaab50749b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 231 base->sup_vtable->su_port_decref(self, 0, who); (gdb) (gdb) (gdb) bt #0 0x00002aaaab50749b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 #1 su_msg_destroy (rmsg=<value optimized out>) at su_root.c:1191 #2 0x00002aaaab434517 in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1529 #3 0x00002aaaab42e9e0 in our_sofia_event_callback (event=nua_i_cancel, status=200, phrase=0x2aaaac74d850 "OK", nua=0x2aaab400c270, profile=0x125eb740,     nh=0x2aaaac05e320, sofia_private=0x2aab0c644c90, sip=0x2aaaac167438, de=0x2aab0c1ba7d0, tags=0x2aaaac74d840) at sofia.c:975 #4 0x00002aaaab43450f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1526 #5 0x00002aaaab434648 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1575 #6 0x00002ab99f4e577d in start_thread () from /lib64/libpthread.so.0 #7 0x00002ab9a06023ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab50749b in su_port_decref (rmsg=<value optimized out>) at su_port.h:231 No locals. #1 su_msg_destroy (rmsg=<value optimized out>) at su_root.c:1191         msg = 0x2aab0c024990         __PRETTY_FUNCTION__ = "su_msg_destroy" #2 0x00002aaaab434517 in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1529         de = 0x2aab0c084d60         nh = 0x2aaaac05e320         nua = 0x2aaab400c270         profile = 0x125eb740 #3 0x00002aaaab42e9e0 in our_sofia_event_callback (event=nua_i_cancel, status=200, phrase=0x2aaaac74d850 "OK", nua=0x2aaab400c270, profile=0x125eb740,     nh=0x2aaaac05e320, sofia_private=0x2aab0c644c90, sip=0x2aaaac167438, de=0x2aab0c1ba7d0, tags=0x2aaaac74d840) at sofia.c:975         qde = 0x0         tech_pvt = <value optimized out>         auth_res = <value optimized out>         session = <value optimized out>         channel = <value optimized out>         gateway = <value optimized out>         locked = <value optimized out>         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #4 0x00002aaaab43450f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1526         de = 0x2aab0c1ba7d0         nh = 0x2aaaac05e320         nua = 0x2aaab400c270         profile = 0x125eb740 #5 0x00002aaaab434648 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1575         de = 0x0         pop = 0x2aab0c1ba7d0         q = 0x2aaaab85aad8         my_id = <value optimized out>         __func__ = "sofia_msg_thread_run" #6 0x00002ab99f4e577d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00002ab9a06023ed in clone () from /lib64/libc.so.6 No symbol table info available. (gdb)

Git added a comment - 07/Aug/12 5:22 PM Repository: freeswitch Branch: refs/heads/master Commit: d806014 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=d806014 Updated By: anthm@freeswitch.org Comment: FS-4079 try this FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 13/Aug/12 11:04 AM Okay, I've been running this for 4 days now. We're hitting a seg right here consistently. Program terminated with signal 11, Segmentation fault. #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x6da4dc0, profile=0x2aaaac02ccc0, nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40,     sip=0x2aab4c4445c8, de=0x2aab305241b0, tags=0x2aab4c989c20) at sofia.c:8027 8027 sofia_glue_get_addr(de->data->e_msg, network_ip, sizeof(network_ip), &network_port); (gdb) bt #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x6da4dc0, profile=0x2aaaac02ccc0, nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40,     sip=0x2aab4c4445c8, de=0x2aab305241b0, tags=0x2aab4c989c20) at sofia.c:8027 #1 0x00002aaaab430fdf in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aab4c989c30 "Trying", nua=0x6da4dc0, profile=0x2aaaac02ccc0,     nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40, sip=0x2aab4c4445c8, de=0x2aab305241b0, tags=0x2aab4c989c20) at sofia.c:1192 #2 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532 #3 0x00002aaaab42eb0e in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aaaffe26467 "Trying", nua=0x6da4dc0, profile=0x2aaaac02ccc0,     nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40, sip=0x0, de=0x2aab30c47880, tags=0x2aaaffe26020) at sofia.c:980 #4 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532 #5 0x00002aaaab434798 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1581 #6 0x00002b4ad3cee77d in start_thread () from /lib64/libpthread.so.0 #7 0x00002b4ad4e0b3ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x6da4dc0, profile=0x2aaaac02ccc0, nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40,     sip=0x2aab4c4445c8, de=0x2aab305241b0, tags=0x2aab4c989c20) at sofia.c:8027         session = 0x0         key = '\000' <repeats 127 times>         un = <value optimized out>         rpid = <value optimized out>         passerted = <value optimized out>         ppreferred = <value optimized out>         privacy = <value optimized out>         call_info = <value optimized out>         tech_pvt = <value optimized out>         channel = <value optimized out>         channel_name = <value optimized out>         displayname = <value optimized out>         destination_number = <value optimized out>         from_user = <value optimized out>         from_host = <value optimized out>         referred_by_user = <value optimized out>         context = <value optimized out>         dialplan = <value optimized out>         network_ip = '\000' <repeats 79 times>         proxied_client_ip = '\000' <repeats 79 times>         v_event = 0x0         sess_count = 1343         sess_max = 3000         is_auth = <value optimized out>         calling_myself = <value optimized out>         network_port = 0         is_nat = <value optimized out>         aniii = <value optimized out>         acl_token = '\000' <repeats 511 times>         transport = <value optimized out>         gw_name = <value optimized out>         gw_param_name = <value optimized out>         call_info_str = <value optimized out>         bnh = <value optimized out>         sip_acl_authed_by = '\000' <repeats 511 times>         sip_acl_token = '\000' <repeats 511 times>         __func__ = "sofia_handle_sip_i_invite"         __PRETTY_FUNCTION__ = "sofia_handle_sip_i_invite" #1 0x00002aaaab430fdf in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aab4c989c30 "Trying", nua=0x6da4dc0, profile=0x2aaaac02ccc0,     nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40, sip=0x2aab4c4445c8, de=0x2aab305241b0, tags=0x2aab4c989c20) at sofia.c:1192         tech_pvt = 0x0         auth_res = <value optimized out>         session = 0x0 ---Type <return> to continue, or q <return> to quit---         channel = 0x0         gateway = 0x0         locked = 0         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #2 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532         de = 0x2aab305241b0         nh = 0x2aab4cafb150         nua = 0x6da4dc0         profile = 0x2aaaac02ccc0 #3 0x00002aaaab42eb0e in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aaaffe26467 "Trying", nua=0x6da4dc0, profile=0x2aaaac02ccc0,     nh=0x2aab4cafb150, sofia_private=0x2aab31b99e40, sip=0x0, de=0x2aab30c47880, tags=0x2aaaffe26020) at sofia.c:980         qde = 0x0         tech_pvt = <value optimized out>         auth_res = <value optimized out>         session = <value optimized out>         channel = <value optimized out>         gateway = <value optimized out>         locked = <value optimized out>         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #4 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532         de = 0x2aab30c47880         nh = 0x2aab4cafb150         nua = 0x6da4dc0         profile = 0x2aaaac02ccc0 #5 0x00002aaaab434798 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1581         de = 0x0         pop = 0x2aab30c47880         q = 0x2aaaab85cad8         my_id = <value optimized out>         __func__ = "sofia_msg_thread_run" #6 0x00002b4ad3cee77d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00002b4ad4e0b3ed in clone () from /lib64/libc.so.6 No symbol table info available.

Git added a comment - 13/Aug/12 1:47 PM Repository: freeswitch Branch: refs/heads/master Commit: 4ef9b3f http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=4ef9b3f Updated By: anthm@freeswitch.org Comment: FS-4079 new patch, please test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 14/Aug/12 1:10 AM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 9311e78 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=9311e78 Updated By: SwK Comment: FS-4079 try this FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 14/Aug/12 1:11 AM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 608ad58 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=608ad58 Updated By: SwK Comment: FS-4079 new patch, please test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 14/Aug/12 2:04 PM Just pushed this one out into production. Will let you know. I can give you access to the box with the previous crashes too if need be.

Matthew Williams added a comment - 14/Aug/12 2:13 PM Latest Program terminated with signal 11, Segmentation fault. #0 0x00002aaaab416b5c in sofia_handle_sip_i_invite (nua=0x1abee40, profile=0x1a9e470, nh=0x2aaaf811e3f0, sofia_private=0x2aaadc9a5820, sip=0x2aaaf8325688,     de=0x1, tags=0x2aaaf85337c0) at sofia.c:8605 8605 if (mp->mp_payload && mp->mp_payload->pl_data && mp->mp_content_type && mp->mp_content_type->c_type) { (gdb) (gdb) bt #0 0x00002aaaab416b5c in sofia_handle_sip_i_invite (nua=0x1abee40, profile=0x1a9e470, nh=0x2aaaf811e3f0, sofia_private=0x2aaadc9a5820, sip=0x2aaaf8325688,     de=0x1, tags=0x2aaaf85337c0) at sofia.c:8605 #1 0x00002aaaab430fe1 in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aaaf85337d0 "Trying", nua=0x1abee40, profile=0x1a9e470,     nh=0x2aaaf811e3f0, sofia_private=0x2aaadc9a5820, sip=0x2aaaf8325688, de=0x2aaadc48c2a0, tags=0x2aaaf85337c0) at sofia.c:1194 #2 0x00002aaaab43461f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1534 #3 0x00002aaaab434ba1 in sofia_event_callback (event=<value optimized out>, status=<value optimized out>, phrase=<value optimized out>, nua=0x1abee40,     profile=0x1a9e470, nh=0x2aaaf811e3f0, sofia_private=<value optimized out>, sip=0x2aaaf8325688, tags=0x2aaaf85337c0) at sofia.c:1711 #4 0x00002aaaab4ba577 in nua_application_event (dummy=<value optimized out>, sumsg=0x41ffec58, ee=<value optimized out>) at nua_stack.c:393 #5 0x00002aaaab503a54 in su_base_port_execute_msgs (queue=0x2aaaf83a4c50) at su_base_port.c:280 #6 0x00002aaaab503f81 in su_base_port_step (self=0x1a7f890, tout=1000) at su_base_port.c:454 #7 0x00002aaaab427040 in sofia_profile_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:2416 #8 0x00002af59686377d in start_thread () from /lib64/libpthread.so.0 #9 0x00002af5979803ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab416b5c in sofia_handle_sip_i_invite (nua=0x1abee40, profile=0x1a9e470, nh=0x2aaaf811e3f0, sofia_private=0x2aaadc9a5820, sip=0x2aaaf8325688,     de=0x1, tags=0x2aaaf85337c0) at sofia.c:8605         mp = 0x2aaaf837b4b0         session = 0x2aaadc519d38         key = '\000' <repeats 127 times>         un = <value optimized out>         rpid = 0x0         passerted = <value optimized out>         ppreferred = <value optimized out>         privacy = <value optimized out>         call_info = <value optimized out>         tech_pvt = <value optimized out>         channel = 0x2aaadc51f8f8         channel_name = 0x2aaadc3a3ce8 " 18009672346@65.98.149.31 :5060"         displayname = 0x2aaaf802e4ea "18009672346"         destination_number = 0x2aaaf802e3cb "13232902979"         from_user = 0x2aaaf802e4ea "18009672346"         from_host = 0x2aaaf802e4f6 "65.98.149.31"         referred_by_user = <value optimized out>         context = <value optimized out>         dialplan = <value optimized out>         network_ip = "64.135.116.166\000\000\060RT\253\252*\000\000pRT\253\252*\000\000\260RT\253\252*\000\000\360RT\253\252*\000\000(ST\253\252*\000\000hST\253\252*\000\000\240ST\253\252*\000\000\237\225\223\227\365*\000"         proxied_client_ip = "xOT\253\252*\000\000\260OT\253\252*\000\000\360OT\253\252*\000\000(PT\253\252*\000\000XPT\253\252*\000\000\220PT\253\252*\000\000\320PT\253\252*\000\000\bQT\253\252*\000\000PQT\253\252*\000\000\210QT\253\252*\000"         v_event = 0x0         sess_count = 1177         sess_max = <value optimized out>         is_auth = 0         calling_myself = -1420492197         network_port = 5060         is_nat = 0x0         aniii = 0x0         acl_token = '\000' <repeats 511 times>         transport = SOFIA_TRANSPORT_UDP         gw_name = <value optimized out>         gw_param_name = <value optimized out>         call_info_str = <value optimized out>         bnh = <value optimized out>         sip_acl_authed_by = '\000' <repeats 511 times>         sip_acl_token = '\000' <repeats 511 times>         __func__ = "sofia_handle_sip_i_invite"         __PRETTY_FUNCTION__ = "sofia_handle_sip_i_invite" #1 0x00002aaaab430fe1 in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aaaf85337d0 "Trying", nua=0x1abee40, profile=0x1a9e470,     nh=0x2aaaf811e3f0, sofia_private=0x2aaadc9a5820, sip=0x2aaaf8325688, de=0x2aaadc48c2a0, tags=0x2aaaf85337c0) at sofia.c:1194 ---Type <return> to continue, or q <return> to quit---         tech_pvt = 0x0         auth_res = <value optimized out>         session = 0x0         channel = 0x0         gateway = 0x0         locked = 0         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #2 0x00002aaaab43461f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1534         de = 0x2aaadc48c2a0         nh = 0x2aaaf811e3f0         nua = 0x1abee40         profile = 0x1a9e470 #3 0x00002aaaab434ba1 in sofia_event_callback (event=<value optimized out>, status=<value optimized out>, phrase=<value optimized out>, nua=0x1abee40,     profile=0x1a9e470, nh=0x2aaaf811e3f0, sofia_private=<value optimized out>, sip=0x2aaaf8325688, tags=0x2aaaf85337c0) at sofia.c:1711         de = 0x0         critical = <value optimized out>         __func__ = "sofia_event_callback" #4 0x00002aaaab4ba577 in nua_application_event (dummy=<value optimized out>, sumsg=0x41ffec58, ee=<value optimized out>) at nua_stack.c:393         frame = {{nf_next = 0x0, nf_saved = {0x0}}}         nua = 0x1abee40         e = 0x2aaaf85337a0         nh = 0x2aaaf811e3f0         __func__ = "nua_application_event" #5 0x00002aaaab503a54 in su_base_port_execute_msgs (queue=0x2aaaf83a4c50) at su_base_port.c:280         root = <value optimized out>         f = 0x2aaaf8325688         msg = 0x0         n = 4318 #6 0x00002aaaab503f81 in su_base_port_step (self=0x1a7f890, tout=1000) at su_base_port.c:454         now = {tv_sec = 3553960061, tv_usec = 601716}         __PRETTY_FUNCTION__ = "su_base_port_step" #7 0x00002aaaab427040 in sofia_profile_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:2416         profile = 0x1a9e470         node = 0x0         s_event = 0x0         sanity = <value optimized out>         worker_thread = 0x1a9fc60         st = SWITCH_STATUS_SUCCESS         __func__ = "sofia_profile_thread_run"         __PRETTY_FUNCTION__ = "sofia_profile_thread_run" #8 0x00002af59686377d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #9 0x00002af5979803ed in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit---

Git added a comment - 14/Aug/12 5:05 PM Repository: freeswitch Branch: refs/heads/master Commit: 0d05762 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=0d05762 Updated By: anthm@freeswitch.org Comment: FS-4079 revert 4ef9b3fe5622e417e8cd2916491e0058492a3bb7 FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 14/Aug/12 10:58 PM Here is one from the previous build or the revert that is slightly different than the ones before. #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x2aaabbb9b808, de=0x2aaac7917fd0, tags=0x2aaabb9624d0) at sofia.c:8027 8027 sofia_glue_get_addr(de->data->e_msg, network_ip, sizeof(network_ip), &network_port); (gdb) bt #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x2aaabbb9b808, de=0x2aaac7917fd0, tags=0x2aaabb9624d0) at sofia.c:8027 #1 0x00002aaaab430fdf in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aaabb9624e0 "Trying", nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x2aaabbb9b808,     de=0x2aaac7917fd0, tags=0x2aaabb9624d0) at sofia.c:1192 #2 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532 #3 0x00002aaaab42eb0e in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aaabbc16ec9 "Trying", nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x0, de=0x2aaac702fec0,     tags=0x2aaabbc16a20) at sofia.c:980 #4 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532 #5 0x00002aaaab434798 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1581 #6 0x00002ad7825e177d in start_thread () from /lib64/libpthread.so.0 #7 0x00002ad7836fe3ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00002aaaab4153f8 in sofia_handle_sip_i_invite (nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x2aaabbb9b808, de=0x2aaac7917fd0, tags=0x2aaabb9624d0) at sofia.c:8027         session = 0x0         key = '\000' <repeats 127 times>         un = <value optimized out>         rpid = <value optimized out>         passerted = <value optimized out>         ppreferred = <value optimized out>         privacy = <value optimized out>         call_info = <value optimized out>         tech_pvt = <value optimized out>         channel = <value optimized out>         channel_name = <value optimized out>         displayname = <value optimized out>         destination_number = <value optimized out>         from_user = <value optimized out>         from_host = <value optimized out>         referred_by_user = <value optimized out>         context = <value optimized out>         dialplan = <value optimized out>         network_ip = '\000' <repeats 79 times>         proxied_client_ip = '\000' <repeats 79 times>         v_event = 0x0         sess_count = 742         sess_max = 3000         is_auth = <value optimized out>         calling_myself = <value optimized out>         network_port = 0         is_nat = <value optimized out>         aniii = <value optimized out>         acl_token = '\000' <repeats 511 times>         transport = <value optimized out>         gw_name = <value optimized out>         gw_param_name = <value optimized out>         call_info_str = <value optimized out>         bnh = <value optimized out>         sip_acl_authed_by = '\000' <repeats 511 times>         sip_acl_token = '\000' <repeats 511 times>         __func__ = "sofia_handle_sip_i_invite"         __PRETTY_FUNCTION__ = "sofia_handle_sip_i_invite" #1 0x00002aaaab430fdf in our_sofia_event_callback (event=nua_i_invite, status=100, phrase=0x2aaabb9624e0 "Trying", nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x2aaabbb9b808,     de=0x2aaac7917fd0, tags=0x2aaabb9624d0) at sofia.c:1192         tech_pvt = 0x0         auth_res = <value optimized out>         session = 0x0         channel = 0x0         gateway = 0x0         locked = 0         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #2 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532         de = 0x2aaac7917fd0         nh = 0x2aaabbdb3360         nua = 0x4946510         profile = 0x4921430 #3 0x00002aaaab42eb0e in our_sofia_event_callback (event=nua_i_state, status=100, phrase=0x2aaabbc16ec9 "Trying", nua=0x4946510, profile=0x4921430, nh=0x2aaabbdb3360, sofia_private=0x2aaac71227e0, sip=0x0, de=0x2aaac702fec0,     tags=0x2aaabbc16a20) at sofia.c:980         qde = 0x0         tech_pvt = <value optimized out>         auth_res = <value optimized out>         session = <value optimized out>         channel = <value optimized out>         gateway = <value optimized out>         locked = <value optimized out>         check_destroy = <value optimized out>         __func__ = "our_sofia_event_callback"         __PRETTY_FUNCTION__ = "our_sofia_event_callback" #4 0x00002aaaab43465f in sofia_process_dispatch_event (dep=<value optimized out>) at sofia.c:1532         de = 0x2aaac702fec0         nh = 0x2aaabbdb3360 ---Type <return> to continue, or q <return> to quit---         nua = 0x4946510         profile = 0x4921430 #5 0x00002aaaab434798 in sofia_msg_thread_run (thread=<value optimized out>, obj=<value optimized out>) at sofia.c:1581         de = 0x0         pop = 0x2aaac702fec0         q = 0x2aaaab85cad8         my_id = <value optimized out>         __func__ = "sofia_msg_thread_run" #6 0x00002ad7825e177d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #7 0x00002ad7836fe3ed in clone () from /lib64/libc.so.6

Git added a comment - 15/Aug/12 10:51 PM Repository: freeswitch Branch: refs/heads/master Commit: bb69310 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=bb69310 Updated By: anthm@freeswitch.org Comment: FS-4079 FS-4540 please update to this version FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 17/Aug/12 12:19 PM I've upgraded to this version. No crashes as of yet but freeswitch is now hanging calls. It will not shut down and it intermittently responds on the cli. I'll have an instance running in this state that you can have a look at after a bit here.

Git added a comment - 17/Aug/12 1:23 PM Repository: freeswitch Branch: refs/heads/master Commit: b81e780 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b81e780 Updated By: anthm@freeswitch.org Comment: FS-4079 FS-4540 new patch FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 19/Aug/12 1:45 PM Looks like there is a memory leak here. It seems to be eating up all my ram then the box starts swapping and FS doesn't respond on the cli.

Git added a comment - 21/Aug/12 12:14 AM Repository: freeswitch Branch: refs/heads/master Commit: b6f64dc http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b6f64dc Updated By: anthm@freeswitch.org Comment: FS-4079 FS-4540 please update and test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Matthew Williams added a comment - 21/Aug/12 6:06 PM Here is a seg after upgrading to the following. I'm not sure if it has anything to do with the problems were having but I'll post it anyway. FreeSWITCH Version 1.3.0+git~20120821T170910Z~d0dd441380 (1.3.0; git at commit d0dd441380 on Tue, 21 Aug 2012 17:09:10 Z) (gdb) bt #0 allocator_free (pool=0x2aaac44aefe8) at memory/unix/apr_pools.c:332 #1 apr_pool_destroy (pool=0x2aaac44aefe8) at memory/unix/apr_pools.c:794 #2 0x00002aabfcf4b915 in pool_thread (thread=<value optimized out>, obj=<value optimized out>) at src/switch_core_memory.c:532 #3 0x00002aabfdb8577d in start_thread () from /lib64/libpthread.so.0 #4 0x00002aabfeca23ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 allocator_free (pool=0x2aaac44aefe8) at memory/unix/apr_pools.c:332         max_index = 3299424592         current_free_index = 3288334456 #1 apr_pool_destroy (pool=0x2aaac44aefe8) at memory/unix/apr_pools.c:794         active = 0x3a726566785f6c62         allocator = 0x2aaac48f6ef0 #2 0x00002aabfcf4b915 in pool_thread (thread=<value optimized out>, obj=<value optimized out>) at src/switch_core_memory.c:532         pop = 0x2aaac44aefe8         x = 150 #3 0x00002aabfdb8577d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00002aabfeca23ed in clone () from /lib64/libc.so.6 No symbol table info available.

Anthony Minessale II added a comment - 21/Aug/12 7:48 PM update to HEAD again please

Matthew Williams added a comment - 22/Aug/12 9:33 AM Looks like it's crashing in the same spot (1.3.0; git at commit 00f586ca5a on Tue, 21 Aug 2012 21:19:25 Z) #0 allocator_free (pool=0x2aaacc897738) at memory/unix/apr_pools.c:332 332 index = node->index; (gdb) bt #0 allocator_free (pool=0x2aaacc897738) at memory/unix/apr_pools.c:332 #1 apr_pool_destroy (pool=0x2aaacc897738) at memory/unix/apr_pools.c:794 #2 0x00002ba08bc04915 in pool_thread (thread=<value optimized out>, obj=<value optimized out>) at src/switch_core_memory.c:532 #3 0x00002ba08c83e77d in start_thread () from /lib64/libpthread.so.0 #4 0x00002ba08d95b3ed in clone () from /lib64/libc.so.6 (gdb) bt full #0 allocator_free (pool=0x2aaacc897738) at memory/unix/apr_pools.c:332         max_index = 3451247008         current_free_index = 3422552184 #1 apr_pool_destroy (pool=0x2aaacc897738) at memory/unix/apr_pools.c:794         active = 0x20         allocator = 0x2aaacca25470 #2 0x00002ba08bc04915 in pool_thread (thread=<value optimized out>, obj=<value optimized out>) at src/switch_core_memory.c:532         pop = 0x2aaacc897738         x = 158 #3 0x00002ba08c83e77d in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x00002ba08d95b3ed in clone () from /lib64/libc.so.6 No symbol table info available.

Anthony Minessale II added a comment - 22/Aug/12 4:17 PM update again

Git added a comment - 23/Aug/12 11:09 AM Repository: freeswitch Branch: refs/heads/master Commit: 5213c76 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=5213c76 Updated By: anthm@freeswitch.org Comment: FS-4079 FS-4540 minor refactoring to fix some minor side effects of recent changes FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 23/Aug/12 2:55 PM Repository: freeswitch Branch: refs/heads/master Commit: 04924f1 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=04924f1 Updated By: anthm@freeswitch.org Comment: FS-4079 FS-4540 regression from 5213c764bbaf779a0b4c4f9feb35f13b5097c6c4 please update ASAP if you are running this one FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Anthony Minessale II added a comment - 31/Aug/12 5:39 PM please update again

Anthony Minessale II added a comment - 01/Sep/12 11:46 AM and again

Matthew Williams added a comment - 08/Sep/12 4:24 PM Between FreeSWITCH Version 1.3.0+git~20120824T225353Z~60cd56d1fc (1.3.0; git at commit 60cd56d1fc on Fri, 24 Aug 2012 22:53:53 Z) and FreeSWITCH Version 1.3.0+git~20120906T165403Z~cbefc9dae1 (1.3.0; git at commit cbefc9dae1 on Thu, 06 Sep 2012 16:54:03 Z) There is a memory leak somewhere in there. It ate up 8 gig in a 24 hour period. Nothing changed on the config side.

Umberto Mautone added a comment - 08/Sep/12 4:33 PM I have this happening to me as well since updating. Mine eats up about 26 gig in 12 hours.

Git added a comment - 12/Sep/12 12:20 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: b0537b2 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b0537b2 Updated By: SwK Comment: FS-4079 revert 4ef9b3fe5622e417e8cd2916491e0058492a3bb7 FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 12/Sep/12 12:21 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: a7fadff http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=a7fadff Updated By: SwK Comment: FS-4079 FS-4540 please update to this version FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 12/Sep/12 12:22 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: d9aac5a http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=d9aac5a Updated By: SwK Comment: FS-4079 FS-4540 new patch FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 12/Sep/12 12:22 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: a235b70 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=a235b70 Updated By: SwK Comment: FS-4079 FS-4540 please update and test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 12/Sep/12 12:23 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 42a233d http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=42a233d Updated By: SwK Comment: FS-4079 FS-4540 minor refactoring to fix some minor side effects of recent changes FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 12/Sep/12 12:23 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 9a6b914 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=9a6b914 Updated By: SwK Comment: FS-4079 FS-4540 regression from 5213c764bbaf779a0b4c4f9feb35f13b5097c6c4 please update ASAP if you are running this one FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:41 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: d806014 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=d806014 Updated By: SwK Comment: FS-4079 try this FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:44 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 4ef9b3f http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=4ef9b3f Updated By: SwK Comment: FS-4079 new patch, please test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:45 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 0d05762 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=0d05762 Updated By: SwK Comment: FS-4079 revert 4ef9b3fe5622e417e8cd2916491e0058492a3bb7 FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:46 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: bb69310 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=bb69310 Updated By: SwK Comment: FS-4079 FS-4540 please update to this version FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:46 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: b81e780 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b81e780 Updated By: SwK Comment: FS-4079 FS-4540 new patch FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:47 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: b6f64dc http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b6f64dc Updated By: SwK Comment: FS-4079 FS-4540 please update and test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:48 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 5213c76 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=5213c76 Updated By: SwK Comment: FS-4079 FS-4540 minor refactoring to fix some minor side effects of recent changes FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 20/Sep/12 4:48 PM Repository: freeswitch Branch: refs/heads/v1.2.stable Commit: 04924f1 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=04924f1 Updated By: SwK Comment: FS-4079 FS-4540 regression from 5213c764bbaf779a0b4c4f9feb35f13b5097c6c4 please update ASAP if you are running this one FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Peter Olsson added a comment - 02/Oct/12 11:00 AM Is this resolved yet?

Chris Fogel added a comment - 02/Oct/12 11:10 AM I've been running git 15ef892 2012-09-28 03:48:18Z for 4 days and it appears to have resolved the issue. I recommend upgrading to this version, but I will keep you updated if I see anything.

Peter Olsson added a comment - 02/Oct/12 11:15 AM Thanks for the feedback. Lets keep this open for the rest of the week, to make sure everything works fine. /Peter

Peter Olsson added a comment - 15/Oct/12 12:56 AM It seems this is resolved now.

Git added a comment - 19/Oct/12 8:41 AM Repository: freeswitch Branch: refs/heads/video-media-bug Commit: bb69310 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=bb69310 Updated By: dujinfang@gmail.com Comment: FS-4079 FS-4540 please update to this version FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 19/Oct/12 8:42 AM Repository: freeswitch Branch: refs/heads/video-media-bug Commit: b81e780 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b81e780 Updated By: dujinfang@gmail.com Comment: FS-4079 FS-4540 new patch FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 19/Oct/12 8:43 AM Repository: freeswitch Branch: refs/heads/video-media-bug Commit: b6f64dc http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=b6f64dc Updated By: dujinfang@gmail.com Comment: FS-4079 FS-4540 please update and test FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 19/Oct/12 8:44 AM Repository: freeswitch Branch: refs/heads/video-media-bug Commit: 5213c76 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=5213c76 Updated By: dujinfang@gmail.com Comment: FS-4079 FS-4540 minor refactoring to fix some minor side effects of recent changes FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Git added a comment - 19/Oct/12 8:44 AM Repository: freeswitch Branch: refs/heads/video-media-bug Commit: 04924f1 http://fisheye.freeswitch.org/changelog/freeswitch.git/?cs=04924f1 Updated By: dujinfang@gmail.com Comment: FS-4079 FS-4540 regression from 5213c764bbaf779a0b4c4f9feb35f13b5097c6c4 please update ASAP if you are running this one FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com

Auto Admin added a comment - 01/Nov/12 9:17 PM Due to a long period of inactivity (13 or more days), this issue is due to be automatically close within 24 hours. If this issue is not actually resolved, please reopen it and add appropriate comments to help developers fix the issue. Thanks,   Jira Admin

Auto Admin added a comment - 02/Nov/12 9:17 PM Since there has been no change to the status of this issue, it is being closed for inactivity Thanks,   Jira Admin FreeSWITCH Support Contracts and Consulting Services available! Contact us: Email: consulting@freeswitch.org Web: http://www.freeswitch.org Phone: +1-918-420-9266 Tollfree: +1-877-742-2583 Fax: +1-918-420-9267 iNum: +883 5100 1420 9266 Come To ClueCon in August to learn more about FreeSWITCH and Internet Telephony! http://www.cluecon.com