We’re up and running again!
Tuesday, January 29th, 2008What a day! It turns out the co.mments server had an obscure bug, very simple to fix but also hard to identify. I noticed that bug before, but it happend so infrequently that I couldn’t determine when or why it happens. Today, it started showing up much more frequently on many WordPress blogs.
The bug causes the server to enter an infinite loop, eat up memory and stop processing requests. Tracking a conversation on one of the many blogs that exhibit this behavior caused part of the server to crash, and instead of seeing an error message you would get no response. It happened on too many blogs at once, that all that load caused the entire server to crash repeatedly.
To add insult to injury, the load on the server was so high it would take several minutes to restart, and then a few more minutes to come crashing down. Unfortunately, my attempts to keep it running while fixing the bug only worsened the situation, and the entire site was down for most of the day.
Thankfully, once I found it, the fix was easy. The bug is squashed, co.mments is up and running, speedy and responsive, and has already caught up with all of the day’s comments. No data lost.
We’re back to normal. Thank you for your patience.
Subscribe





