Technology Learning and Unlearning

Appropriate placement for my testcases belonging to a non-app in Django

2009-03-04T05:20:00.000-08:00

I have built my website in Django. And like any other django project I have got apps inside the project root directory and some special folders like(extensions a.k.a custom django command extensions). With an app, we dont have any problem with having testcases. "tests.py" inside the app directory will be the solution. But for a special folder(which is not an app or which doesn't have a models.py) where do I place the testcases. I tried placing the tests.py inside the extensions directory and it said the directory is not a model and unable to run the tests. HOw do I solve this? How can I have a proper placement of testcases related to non-apps?

Hosting an Email server and receiving all mails pertaining to @mysite.com

2009-02-23T05:56:00.000-08:00

I have question regarding emails sent/received by any website out there.
Assuming its mysite.com. Everyuser will be
getting an email address say something like j...@mysite.com. Any
person using the same website or any other website or any one on the
planet can send an email to j...@mysite.com and for jack there is a
Inbox feature built into his dashboard, where he will be seeing all
these emails(forget about jack sending emails to anyone, now). So to
achieve this, I have changed MX records of my site(in godaddy) to:
Priority:10
Host:@
Goes To:www.mysite.com.

And the MX records lookup says:
;; ANSWER SECTION:
www.mysite.com. 3600 IN CNAME mysite.com.
mysite.com. 604800 IN MX 10 www.mysite.com.

And I have configured postfix on my production machine(www.mysite.com
server) to receive all mails. Even then its not receiving any emails
(all getting bounced). May I know whats happening? So MX records is the only one which will be responsible for any screwups? I have tried many sites like stackoverflow but
didn't receive any help. Please help me.

Simple Django/python utility to check all the import statements in your project

2009-02-11T23:06:00.000-08:00

As I mentioned in my previous posting, I was stuck with Flup exceptions and I am trying to implement what Maxx had suggested. But one interesting thing I did, just before MAXX suggested, to check all the import statements in my entire project codebase. Ofcourse this is not a Django level thing, but at the python level. I started writing one and made that a Django command extension, run as python manage.py imports_checker. It helped me in identifying many faulty imports which could be coined as one of the culprits for the "flup's dreaded exception". The code is mentioned below:


class Command(NoArgsCommand):
    option_list = NoArgsCommand.option_list
    help = "Scans through the entire project directory and collects all the stale/obsolete import statements"
    requires_model_validation = True

def import_statement_extractor(self,directory_path,python_file):
    python_file = '%s/%s' % (directory_path,python_file)
    file_content = open(python_file).readlines()
    for line in file_content:
        line = line.strip()
        if line.startswith('from') or line.startswith('import'):
            try:
                exec(line)
#                    print '%s:==>:%s:Pass' % (python_file,line)
            except ImportError:
                print '%s:XXX:%s:Fail' % (python_file,line)

def directory_py_files(self,parent_directory):
   import os
   directory_generator = os.walk(parent_directory)
   directory_info = directory_generator.next()
   for file in directory_info[2]:
       if file.endswith('py'):
          self.import_statement_extractor(directory_info[0],file)
   for directory in directory_info[1]:
       if not directory.startswith('.'):
          print '\n'
          self.directory_py_files('%s/%s' % (parent_directory,directory))
   
def handle_noargs(self, **options):
    from django.conf import settings
    self.directory_py_files(settings.ROOT_PATH)

This is a generic command, it does not check the settings.INSTALLED_APPS setting for cleaning up imports but it does a decent job of identifying all stale or obsolete imports.

Django doesn't suck ok! But.....

2009-02-09T09:29:00.000-08:00

I am a big fan of Django and working on it is really exciting(still after 2 years. I was from java world before). But these days its driving me nuts in one context. Its with my deployment stack. I am using lighttpd + flup + django combination to run my website. It was running fine till recently and then the problem started...

I am facing the dreaded "Unhandled Exception" raised by Flup. The sad part is its raised at the webserver (lighttpd+flup as I call fcgi level) level and not at the application level(Django). So no 500 email is raised about where the problem is.

Our entire team struggled hard to cleanup the codebase, incase of any ambigous imports and someones of that sort, just to be eliminate the chances of raising errors due to the ambiguous imports. And we cleaned up many things in the code. Still the same exception.

I checked the lighttpd logs. All I see is "Interface Error/Connection already closed." Its only occurring when my applicaiton is running in FCGI mode. So the problem is with how flup is actually dealing with my code(application).

To be frank I am really frustrated with Flup's error handling. It doesn't tell you anything. Worst of all, it shows the same "Unhandled Exception" to the Users. At one point I started posting into recently popular programming questions site stackoverflow.com to get some technical help and made a posting. I received a few answers which were actually good, but made my opinion about Flup far more worser than I could ever imagine. So I finally decided to replace flup in the whole website configuration stack. But I stumbled upon django's fastcgi.py(which is responsible running django in fcgi and btw thats how I run my app). It had flup dependancies. What the heck? Django is having hardcoded dependancies with the package whose author itself is not using it? Which means I cant bypass flup till I use Django? How do I get pass this? I dont want to leave Django no matter what, at the same time I cant have my users see this "Unhandled Exception" page just because Django depends on Flup. Now tell me what should I do?

Django Models Tweaks.

2009-02-02T11:50:00.000-08:00

I was struggling with joins while using DJango models. After some searching i got some valuable inputs so I have crystallized it according to my understanding.

Let us take an example of a Model - UserDetails which contains other including age and address .
My intention is to query only age and address - then
user_details = UserDetails.objects.all().values('age','address') should return a dictionary of results with keys 'age' and 'address'.
This is easier to compute and I feel that it is a better programming practice to query only the data which we would be processing.

Coming to the issue of joining two Models and displaying a common result.
The first model is the UserDetails Model ( user_details table ) and the UserFoodHabits Model ( user_food_habits table )

user_details = UserDetails.objects.all().extra(tables=['user_food_habits'],where=['user_details.food_habits_id = user_food_habits.id'],select={'habit':user_food_habits.habit, 'habit_type':user_food_habits.type})

Please note that the where clause is written following the raw SQL syntax.

The resulting query set would contain all the attributes of the UserDetails object and additional keys for 'habit' and 'habit_type'. The following piece of code illustrates that.

for user in user_details:
print 'Name %s, Age %s , Habit %s, Type %s ' % (user.name, user.age, user.habit, user.habit_type)

DJango models do not however officialy support group bys and aggregate functions. So it cannot be used with the extra parameter.
But there is a workaround for it :
http://www.evanreiser.com/2009/01/howto-group-by-in-django.html
Apparently DJango 1.1 would have better support for aggregation and stuff !

A small step.

2008-12-20T21:27:00.000-08:00

My learning/unlearning of open source technology started about 2 years ago with my college project. Although it was very superficial , it was a start alright.
Now i have undertaken the challenge of experting open source techs like Linux, Python, Django for good. Before this my technology concentration was limited to Windows, J2EE technologies like Hibernate, Spring , Tapestry etc.
My new Dell Inspiron 1525 would be home to all my tech innovations hereafter.

Being a newbie, I am going to record all my experiences with Linux, Python, Django etc. So that I keep a record of all the problems and solutions i face and resolve. Of course the larger idea is to document my learning curve !!!