[three]Bean
Fedmsg Reliability -- No More Dropped Messages?
Feb 28, 2013 | categories: fedmsg, zeromq, fedora View CommentsQuick Timeline:
- February, 2012 - I started thinking about fedmsg.
- March, 2012 - Development gets under way.
- July, 2012 - Stuff gets working in staging.
- August-September, 2012 - The first messages hit production. We notice that some messages are getting dropped. We chalk it up to programming/configuration oversights. We are correct about that in some cases.
- October-November, 2012 - Some messages are still getting dropped. Theory develops that it is due to zeromq2's lack of TCP_KEEPALIVE. It takes time to package zeromq3 and test this, integrate it with Twisted bindings, and deploy. More message sources come online during this period. Other stuff like externally consuming messages [1] [2], datanommer [3] [4], and identica [5] happen too.
- January, 2013 - I enabled the TCP_KEEPALIVE bits in production at FUDCon Lawrence.
I haven't noticed any dropped messages since then: a marked improvement. However, our volume of messages is so much higher now that it is difficult to eyeball. If you notice dropped messages, please let me know in #fedora-apps on freenode.