Yesterday Donovan Preston released new versions of both eventlet and Spawning. What are those, you ask? Eventlet is a networking library written using coroutines instead of normal subroutes, which makes writing networked non-blocking IO applications much much simpler. Spawning is a WSGI server, written using eventlet, which supports all of the things you'd expect of a good WSGI server: multiple processes, multiple threads, etc.
Considering that I sit next to Donovan at work all day, I've overheard him extolling the numerous advantages to using a server such as Spawning--the most obvious of which is completely graceful code reloading. Donovan has given a presentation explaining how all of this works, the slides of which probably explain it better than I could. When he told me that he'd added the ability to easily run Django apps with Spawning, I decided to check it out.
First, I installed spawning:
sudo easy_install Spawning
And away setuptools went and installed all of the prerequisites and the package itself. (I have had problems with this in the past, but grabbing greenlet, eventlet, simplejson, PasteDeploy, and Spawning and installing them individually does the trick).
The next thing to do is go to the directory which holds your settings.py, or at least make setup.py available on the Python path. I tend to find it easier to just go to the directory. Then type the following:
spawn --factory=spawning.django_factory.config_factory settings --port 9090 -s 4 -t 100
This starts up a Spawning server with 4 processes and 100 threads. I chose those numbers almost completely arbitrarily. (Well, that's not entirely true, my Apache setup previously had 4 processes and 100 requests per child. I know that requests per child doesn't map at all to threads, but that's where I got the number.) The next thing to do is visit your site, but instead of visiting the normal port 80 or 8000, visit port 9090. If you're running it on your own box, that should be http://127.0.0.1:9090/.
For me, it worked like a charm. It felt like my server was responding faster than ever, but at that point it was just a feeling. To get some quantitative analysis, I ran apachebench with 20 concurrent requests for a total of 10000 requests. On my Apache + mod_wsgi setup, I got 235.65 requests per second. That was really good, I thought! However, with the Spawning setup, I got 347.20 requests per second. I would need to test this much more in-depth if I were a statistician, but it's good enough for me as it did confirm my qualitative analysis.
If you're viewing this on my website directly, then you've already used Spawning, as I've switched over this blog to use the new server. Let me know what you think! Has my site slowed to a crawl? Is it going faster than ever (because I know everyone remembers the speed at which eflorenzano.com loads)?
In all, it was an extremely easy upgrade. I would recommend that everyone who has an interest in these types of things at least try it--especially if you're looking into other pure-python WSGI servers like CherryPy.
UPDATE: If you were having troubles reaching the site before, it's because I was having problems with my database due to another app on the same server, not due to anything that Spawning was doing wrong.
Journey to Intermediary Models
It's been around an 8 month journey in adding intermediary model support to Django, starting with a ticket opened during a sprint by Jacob Kaplan-Moss. Earlier that month I had been wrestling with several pretty nasty models, each with two foreign keys and many extra properties on that relation. I kept thinking that Django makes everything else so easy, but that in at least this one aspect, it doesn't make things easy enough. When Jacob posted his API idea, I was hooked.
I quickly posted some thoughts and asked for a bit of clarification, and volunteered to write the patch. Jacob responded saying he was glad to help me out. It was only then that I actually delved into the code to see how it could be done. It was only then that I realized that I had absolutely no knowledge or familiarity with that portion of the codebase--a portion that is nontrivial, to say the least. But that's the good thing about committing to something: you feel pressure to follow through.
Despite my incredible Naïvité, the patch got to a rudimentary usable state very quickly, and then started to flounder. It was then that Russell Keith-Magee came into the picture, continually prodding me to add more tests, and thinking up many different test cases that I would have never come up with on my own. I'm 100% certain that if Russ had not lent his expertise and guidance on this patch, it would have gotten lost and forgotten for a long time until someone more capable came along to take a look at it.
A few hours ago, Russ committed the patch to trunk. It's interesting to see the reactions that some people have, but on any project like this you'll always be scrutinized. In any case, check out the two new bits of documentation, and see if intermediary models are right for your project. After having this great experience with working on a patch for Django, I'll definitely be looking to help out in other places as well. My advice for anyone looking to get involved with the project is to, well, get involved! Jump in over your head. It's more fun that way.
PyMag
For around as long as I have been working on intermediary model support, I've been supporting another Django project of mine: django-threadedcomments. One of the things that I noticed a few months into maintaining the project is that outside of a few people who were actively using it, not many people really knew about its existence. So when Doug Hellmann (of the famously excellent PyMOTW series) contacted me about writing an article for PyMag, it was immediately apparent what I would love to write about. Over 4000 words later, between finals and school projects and moving across state boundaries, the article was written.
To be honest, I had almost completely forgotten about having written it, aside from one short e-mail conversation with the technical editor. It turns out that the July 2008 edition of PyMag has arrived and my article is listed under "featured articles". How cool is that!? I'm quite proud of the article, and really hope that it helps some people out with their Django websites. If you aren't a PyMag subscriber, then what are you waiting for?
This post seems to be doing a lot of self-promotion--sorry for that. But these two things really made my day, and to me, blogging is about sharing those awesome days with others.
It's always been a goal of mine to post screencasts here on my blog, but for whatever reason I never ended up getting around to it. Today, that all changes as I have created two new screencasts. Of course, this space is already very well-covered by both Michael Trier and Brian Rosner, so hopefully this adds something new to the conversation.
Setting up a Django Development Environment
In this screencast I show how I typically set up my Django development environment. It goes through installing Django by checking out the latest development version and linking it to the correct places on your system. It also talks about how to install reusable applications. Finally, it covers how to update all of those projects and keep a toolbox of snippets for your personal use.
The simple pylink command that I use in the screencast is this:
#!/bin/bash
ln -s `pwd`/$1 `python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/$1
UPDATE: Joshua Uziel has sent me a much more robust version of this script, which handles the edge cases much better. I highly recommend using this version instead of my one-liner.
#!/bin/bash
SITE_PACKAGES="$HOME/prog/python/site-packages"
if [ ! $SITE_PACKAGES ]
then
SITE_PACKAGES=`python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`
fi
BASE=`basename $1`
DIR=`dirname $1`
cd $DIR
ln -sfnv `pwd`/$BASE $SITE_PACKAGES/$BASE
UPDATE2: Zachary Voase has updated this new version, and it seems to be even more improved. I'm loving this! He has also written a "pyunlink" script, which can be found here.
Please let me know in the comments if you have any other tips and tricks for setting up a development environment for Django.
Using Django-Pagination
Django-pagination is an application that I wrote and released a while ago, which I use all the time, but that hasn't really seen much attention. In this screencast, I show how to take an existing project with too much data on one page, and use django-pagination to quickly and easily paginate the items on the page. There is a bit more documentation for the project that's available in the project directory if you do a subversion checkout, and docstrings throughout the source code, if you're interested in how it works.
Keep in Mind
These are my very first screencasts, ever. I'm not entirely sure what I'm doing yet, and the only way I can improve is by your feedback. If you have any advice and/or criticisms of these screencasts, please don't keep your mouth shut--speak up, and let me know in the comments. Hopefully someone finds these useful, and thanks for watching!
Japan: What a Trip
There comes a time towards the end of most vacations where you've had a good time, but you're ready to head back home and sleep in your own bed and see your friends. No offense to any of my friends, but that time never came for me in Japan. I completely fell in love with Tokyo, and didn't want to come home.
I spent the vast majority of my time in Tokyo itself, but there was so much to do that there was never a dull moment. Whether it was being completely and utterly lost in the red light district of Shinjuku, playing pachinko in Ueno, or people watching in Shibuya, I was always up to something. Staying in hostels was probably the best choice that I could have made, because I met so many interesting and fun people! Who else would I have done karaoke with, had it not been for the people I met at the hostel?
Spending a night in a capsule hotel was really interesting, too. There's way more space in there than you would think! You can sit up, or just watch crazy people pretending to time travel on Japanese TV (the best parts are the advertisements, though). And Japanese people are stylish! Some of the styles are very over-the-top, and some are very conservative (never before have I seen so many suits and ties), but almost everyone is trying their hardest to go for style.
Good thing the German guys that I met had met Kazuya, or how would I have gotten in to that crazy hole-in-the-wall bar with only Japanese people and no tourists?
I could really go on endlessly about the trip, but needless to say, I'll have the memories that will last forever. If anyone is thinking of going to Japan, then stop thinking, just do it!
San Francisco
As of the 4th of July, I'm a resident of the state of California. I've got an awesome apartment on Bush street between Powell and Stockton--near Union Square where there are a lot of tourists. It's interesting because whenever I walk outside, I hear a whole variety of different languages and accents. It's really neat, although I suspect after a year its novelty will have worn off and then some.
It's also the first time that I've lived without any roommates. To me, this is refreshing and fun. It's nice to get together with friends whenever I want, but also to be able to relax and avoid all possible forms of human interaction when I feel so inclined. This is helped by the fact that I'm connected to the fastest internet connection that I've ever owned.
Also starting work at Mochi Media has been really good. There's a lot of new stuff to learn, though, so I'm not able to crank out as much code yet as I would like. Their primary web development environment is Pylons, which has a decidedly different worldview than Django. I definitely think that there are good things and bad things from both sides, and hopefully the good things can eventually be shared. I'll definitely revisit this once I've got more experience on the Pylons side.
Cheers to travelling around the globe, living independently, and learning new things! Because in a nutshell, that's what I've been doing.
Up next: A series of screencasts.
All Content
