Thursday 31 May 2012

Adding a Package to PyPi For the 1st Time.

I’ve used PyPi to download but up until today I’d never actually uploaded a package to it. Today, however, is an auspicious day at c&c because finally after five years worth of internal iterations the core ingredients of our internal Django framework for building client sites are being released as [open source packages] under a 3 clause BSD license. Whilst we’re about half completed on all of our packages there are a small handful that are ready to go live to world (django-ccpages, django-ccnews and django-ccthumbs) as experimental releases.

 # this is sucky pip install http://designcc.aws.amazon.com/releases/django-ccpackage-0.0.1.tgz # this is not sucky pip install django-ccpackage 

Up until this point we’ve just been installing our packages off S3 at urls that are unknown to the python community. The former example isn’t exactly friendly, but the latter one is. To turn the latter example into a reality I had to package the code, register with PyPI and finally upload it. Here’s how I did it.

Packaging

To make your code deployable to others from PyPi it has to be packaged up and you may find (as I did) that this can be a little tricksy. I managed to fumble my way through this by looking at other peoples setup.py files on github and cooking up my own various recipes. Maybe it’s just because I’m new to packaging up python code for distribution, but I think there is room to improve on this front. Sucky is sucky, but in short once you’ve got your setup.py authored along with the other recommended files (MANIFEST.in, README, LICENSE) then it’s just case of running the following command

 python setup.py sdist 

This will generate the a distributable package and I’d recommend that you test the resulting file by installing it locally. It took me many, many attempts to get the package to correctly bundle the django templates, css, images and javascript.

Registering

I was very pleasantly surprised to see that you can register with PyPi from the command line using python setup.py register. Because it was the first time I’d ran this command I was prompted to setup an account or login, I choose to setup an account and within a few seconds of typing and a click of a confirmation email I was ready to login.

Running python setup.py register for the second time I was able to login wherein my package was registered with PyPI. I was also prompted to optionally store my authentication information on my machine to speed up subsequent interactions with the python setup.py register command.

Because I’d successfully logged in and my package was registered with PyPI and visible on PyPi I thought I was ready to be able to type –

 pip install django-ccthumbs Downloading/unpacking django-ccthumbs No distributions at all found for django-ccthumbs 

That command didn’t work because hadn’t uploaded it — schoolboy error #1.

Uploading

After five mins of head scratching, one cup of tea and writing this blog post I discovered that my package wasn’t available because I hadn’t uploaded it.

“Aha” I thought to myself. I thought I’d cracked it –

 python setup.py upload running upload error: no dist file created in earlier command 

I thought I was going mad because I could see the dist file, it was right there in all of it’s tar and gzipped glory. I’d like to say that I didn’t run that command fifty more times, each time hoping for a magical intervention that would result in a successful uploading, but I did and none of the 51 attempts worked.

So then, for no apparent reason I tried this –

 python setup.py sdist upload ... running upload submitting dist/django-ccthumbs-0.0.2.tar.gz Server response (200): OK 

I was beaming happiness so I thought I’d go for the kill

 pip install django-ccthumbs 

It worked, django-ccthumbs installed itself as did the dependancy of PIL. I have to say I did a little victory dance because as small and as insignificant as that release was, it was my first and it was special.

I did however find something wrong after I uploaded it so I had to release another version that fixed the error — this was schoolboy error #2.

Next Time

I’ll be sure to use the test PyPi site and add the -r flag to my interactions with the python setup.py register command so I can test things remotely without cluttering up the live version and having to do arbitrary version bumps to fix schoolboy errors.

Instead of using a README.md file, I’ll use a README.rst file. This is because the .rst format can be used by both PyPi and Github. Getting better at writing in restructured text is a good thing because all of these projects will require documentation.

Bumps and minor niggles aside, “yay me” I’ve now put my first packages out there in the wild. Now it’s where the hard work starts of maintaining the packages, managing their admin and making sure the documentation is up to scratch.

Be sure to checkout designcc’s github account for all of our freshly new opensourced django packages.