Simple Django/python utility to check all the import statements in your project

As I mentioned in my previous posting, I was stuck with Flup exceptions and I am trying to implement what Maxx had suggested. But one interesting thing I did, just before MAXX suggested, to check all the import statements in my entire project codebase. Ofcourse this is not a Django level thing, but at the python level. I started writing one and made that a Django command extension, run as python manage.py imports_checker. It helped me in identifying many faulty imports which could be coined as one of the culprits for the "flup's dreaded exception". The code is mentioned below:

class Command(NoArgsCommand):
    option_list = NoArgsCommand.option_list
    help = "Scans through the entire project directory and collects all the stale/obsolete import statements"
    requires_model_validation = True

def import_statement_extractor(self,directory_path,python_file):
    python_file = '%s/%s' % (directory_path,python_file)
    file_content = open(python_file).readlines()
    for line in file_content:
        line = line.strip()
        if line.startswith('from') or line.startswith('import'):
            try:
                exec(line)
#                    print '%s:==>:%s:Pass' % (python_file,line)
            except ImportError:
                print '%s:XXX:%s:Fail' % (python_file,line)

def directory_py_files(self,parent_directory):
   import os
   directory_generator = os.walk(parent_directory)
   directory_info = directory_generator.next()
   for file in directory_info[2]:
       if file.endswith('py'):
          self.import_statement_extractor(directory_info[0],file)
   for directory in directory_info[1]:
       if not directory.startswith('.'):
          print '\n'
          self.directory_py_files('%s/%s' % (parent_directory,directory))
   
def handle_noargs(self, **options):
    from django.conf import settings
    self.directory_py_files(settings.ROOT_PATH)

This is a generic command, it does not check the settings.INSTALLED_APPS setting for cleaning up imports but it does a decent job of identifying all stale or obsolete imports.

Django doesn't suck ok! But.....

I am a big fan of Django and working on it is really exciting(still after 2 years. I was from java world before). But these days its driving me nuts in one context. Its with my deployment stack. I am using lighttpd + flup + django combination to run my website. It was running fine till recently and then the problem started...

I am facing the dreaded "Unhandled Exception" raised by Flup. The sad part is its raised at the webserver (lighttpd+flup as I call fcgi level) level and not at the application level(Django). So no 500 email is raised about where the problem is.

Our entire team struggled hard to cleanup the codebase, incase of any ambigous imports and someones of that sort, just to be eliminate the chances of raising errors due to the ambiguous imports. And we cleaned up many things in the code. Still the same exception.

I checked the lighttpd logs. All I see is "Interface Error/Connection already closed." Its only occurring when my applicaiton is running in FCGI mode. So the problem is with how flup is actually dealing with my code(application).

To be frank I am really frustrated with Flup's error handling. It doesn't tell you anything. Worst of all, it shows the same "Unhandled Exception" to the Users. At one point I started posting into recently popular programming questions site stackoverflow.com to get some technical help and made a posting. I received a few answers which were actually good, but made my opinion about Flup far more worser than I could ever imagine. So I finally decided to replace flup in the whole website configuration stack. But I stumbled upon django's fastcgi.py(which is responsible running django in fcgi and btw thats how I run my app). It had flup dependancies. What the heck? Django is having hardcoded dependancies with the package whose author itself is not using it? Which means I cant bypass flup till I use Django? How do I get pass this? I dont want to leave Django no matter what, at the same time I cant have my users see this "Unhandled Exception" page just because Django depends on Flup. Now tell me what should I do?