FreeSWITCH

improve messaging performance of mod_erlang_event

Details

  • Type: Improvement Improvement
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: mod_erlang_event
  • Labels:
    None
  • Environment:
    Mac 10.6.7 64bit
  • Platform:
    OS/X/gcc
  • FreeSWITCH GIT Revision:
    FreeSWITCH Version 1.0.head (git-765908f 2011-05-22 19-10-52 -0500)
  • Reproduced with GIT HEAD?:
    N/A
  • lsb_release:
    N/A

Description

As ref to an email in the list, I found that mod_erlang_event can send messages as slow as 1/100ms = 10/s. It is because that there's an ei_accept_tmo which blocks 100ms in each loop when is cannot get a message from erlang side even there's many messages to be send in FS queue.

Looks mod_event_socket use a multi-thread solution but not in mod_erlang_event, however, this is a simple hack which can be 100 times faster than the old approach.

Sorry I'm attaching this patch in a hurry based on FreeSWITCH Version 1.0.head (git-765908f 2011-05-22 19-10-52 -0500)
Let me know if if you have problem to merge and I can update to git head

Thanks.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Andrew Thompson added a comment -
Will investigate.
Show
Andrew Thompson added a comment - Will investigate.
Hide
Permalink
Seven Du added a comment -
I did a light performance test yesterday on CentOS 64bit. Without this patch FS event queue soon get overload and I can see many Lost Events error. But with this patch it works better.

I do not has exact numbers but probably the original can't handle 10cps and the patch can work w/o problem with 15cps.

Thanks.
Show
Seven Du added a comment - I did a light performance test yesterday on CentOS 64bit. Without this patch FS event queue soon get overload and I can see many Lost Events error. But with this patch it works better. I do not has exact numbers but probably the original can't handle 10cps and the patch can work w/o problem with 15cps. Thanks.
Hide
Permalink
Seven Du added a comment -
I have a new patch for this.

However, as I found somewhere causes core dumps in the mod, so I think we'd better to do this later.
Show
Seven Du added a comment - I have a new patch for this. However, as I found somewhere causes core dumps in the mod, so I think we'd better to do this later.
Hide
Permalink
Seven Du added a comment -
updated.

I also added some if (!session_element) return NULL; though that might not likely happen, but I did see session_element was NULL before we fixed the listener been freed issue.

sorry for the // style comments.
Show
Seven Du added a comment - updated. I also added some if (!session_element) return NULL; though that might not likely happen, but I did see session_element was NULL before we fixed the listener been freed issue. sorry for the // style comments.
Hide
Permalink
Ken Rice added a comment -
any update on this?
Show
Ken Rice added a comment - any update on this?
Hide
Permalink
Seven Du added a comment -
sorry, I got no email alert on the comment update.

I saw this update

http://fisheye.freeswitch.org/changelog/freeswitch.git?cs=ce186707ff2a4b7ee2a5a3a4280bb27e734cff37
http://jira.freeswitch.org/browse/FS-3577

not sure is their actually a different between _tmo(1) and 10, but I

think this patch should be better if _tmo(1) does better than 10. And I used this patch for a while without problem.

I thought this patch should be ok to apply but at the same time I found another problem

http://jira.freeswitch.org/browse/FS-3432

and because we are not sure if this patch can cause that problem so apply delayed. And Andrew said he was scared about the the _tmo(1).

also there's a trick to speed this up without any patch, that is to send back an atom 'ok' from erlang side on any event received so _tmo(100) will return immediately without blocking so much.

Show
Seven Du added a comment - sorry, I got no email alert on the comment update. I saw this update http://fisheye.freeswitch.org/changelog/freeswitch.git?cs=ce186707ff2a4b7ee2a5a3a4280bb27e734cff37 http://jira.freeswitch.org/browse/FS-3577 not sure is their actually a different between _tmo(1) and 10, but I think this patch should be better if _tmo(1) does better than 10. And I used this patch for a while without problem. I thought this patch should be ok to apply but at the same time I found another problem http://jira.freeswitch.org/browse/FS-3432 and because we are not sure if this patch can cause that problem so apply delayed. And Andrew said he was scared about the the _tmo(1). also there's a trick to speed this up without any patch, that is to send back an atom 'ok' from erlang side on any event received so _tmo(100) will return immediately without blocking so much.
Hide
Permalink
Darren A Schreiber added a comment -
I think the proper solution to this is to create two threads, one for sending and one for receiving, modeled after mod_event_socket, no? If so, I will patch this to do so...
Show
Darren A Schreiber added a comment - I think the proper solution to this is to create two threads, one for sending and one for receiving, modeled after mod_event_socket, no? If so, I will patch this to do so...
Hide
Permalink
Tamas Cseke added a comment -
could you please also try FS-3432? (FS-3432 git branch)
Generally it fixes memory corruptions but I also tried to used session hash wrlocking where it is really a must
Show
Tamas Cseke added a comment - could you please also try FS-3432 ? ( FS-3432 git branch) Generally it fixes memory corruptions but I also tried to used session hash wrlocking where it is really a must
Hide
Permalink
Bing Li added a comment -
"also there's a trick to speed this up without any patch, that is to send back an atom 'ok' from erlang side on any event received so _tmo(100) will return immediately without blocking so much."

Steve, did you mean send 'ok' in freeswitch.erl? Can you give an example? Thanks.
Show
Bing Li added a comment - "also there's a trick to speed this up without any patch, that is to send back an atom 'ok' from erlang side on any event received so _tmo(100) will return immediately without blocking so much." Steve, did you mean send 'ok' in freeswitch.erl? Can you give an example? Thanks.
Hide
Permalink
Ken Rice added a comment -
Darren, please review
Show
Ken Rice added a comment - Darren, please review

People

  • Assignee:
    Darren A Schreiber
    Reporter:
    Seven Du
Vote (2)
Watch (3)

Dates

  • Created:
    Updated: