Related Entries

India PyCon 2009
Quick wallpaper changer
Load testing with Grinder
Adding namespace to XML
Opera RSS to OPML

« HTML Email isn't rich
» Can you spare a dime?

Lack of timeout hurts

My scripts bite dust. Laziness is a bad thing.

As pretty much everyone used to the net knows by now, the MS SQL Server wreaked havoc yesterday. Indirectly, it also completely screwed up my blogroll and news aggregator Python scripts.

I used urllib module for fetching data from URLs. While the worm was having a swell time, most of the servers were very very slow to respond. This module doesn't implement timeouts. Since I run these scripts from Cron, all these jobs kept running forever.

Lesson learned: When you think of a possible bug, fix it at the earliest. I was lazy and kept postponing this and it hit finally :-( My good hosts at Cube-Soft were kind enough to take down these cron jobs for me and notified me.

At the moment, I’ve decided to keep running these from command line instead of Cron.

I’ve not done much research on the options. But I’m fairly certain, I’ll have to use TimeoutSocket.py. Then get some enlightenment from Effbot’s excellent writeup about asyncore module. Perhaps even re-write these scripts to execute in multiple threads.

Posted: January 26, 2003 02:17 PM
python

  1. TimeoutSocket.py does not work for urllib.

    The good news is: Python 2.3 (currently only in the second alpha release but solid enough to use for our production web monitoring application) has socket timeouts (and for the Win32 users, SSL is compiled in).

    Posted by: Craig Allen on April 23, 2003 12:47 PM
//-->