Sometimes installing the specific packages and then discovering you require another set of packages for one application you're developing(or trying to develop, atleast) can all be a bit overwhelming, especially for a Python newbie. I recently discovered Virtualenv, Virtualenvwrapper and pip which have made my life much, much easier. Yes, I know these have been the Python developer's tools for years but I learnt python hardly a month back, so yeah, these tools are brand new to me :P

Virtualenv

Virtualenv creates a virtual environment for your python project. It allows for your project to have it's own set of packages which are completely abstracted from your natively installed packages. What I am trying to say here is, you can have Django 1.5 installed on your system, but you can install Django 1.6 or the dev version that'll be accessible only by your project.

To get started, let's install virtualenv:

nitin@jane-saucy:~$ sudo apt-get install python-virtualenv

Creating a project:

The name of our project is going to be coolpyapp

To create a virtual environment for your project - virtualenv project-name,

nitin@jane-saucy:~$ virtualenv coolpyapp
New python executable in coolpyapp/bin/python
Installing setuptools, pip...done.

You'll notice a new directory called coolpyapp created with a bunch of sub-directories in it. We'll get to it in a minute.

Using the virtual environment:

In order to use it, we'll cd into it and then source the bin/activate file everytime we want to start our virtual environment.

nitin@jane-saucy:~$ cd coolpyapp/
nitin@jane-saucy:~/coolpyapp$ source bin/activate
(coolpyapp)nitin@jane-saucy:~/coolpyapp$

Notice the (project-name) before the prompt? That indicates that you're working inside a virtual environment for the project.

To get out of the virtual environment use the deactivate command to get out of it.

(coolpyapp)nitin@jane-saucy:~/coolpyapp$ deactivate
nitin@jane-saucy:~/coolpyapp$

<hr />

Virtenvwrapper

Virtualenvwrapper is a wrapper for virtualenv, only thing, it makes much easier to use.

To install,

nitin@jane-saucy:~/coolpyapp$ sudo apt-get install virtualenvwrapper

Note: Installing virtualenvwrapper automatically installs virtualenv, since it's a dependancy

Let's add a few things to ~/.bash_profile or ~/.bashrc:

export WORKON_HOME="$HOME/.virtualenvs"
source /usr/local/bin/virtualenvwrapper.sh

The first line adds a new environment variable and the second line sources the virtualenvwrapper script. These two settings get applied everytime a new shell is fired up. Why did we do that? We'll get to that in a minute.

Moving on!

Creating a project:

To create a new virtual environment for a project - mkvirtualenv project-name

nitin@jane-saucy:~$ mkvirtualenv coolpyapp
New python executable in coolpyapp/bin/python
Installing setuptools, pip...done.
(coolpyapp)nitin@jane-saucy:~$ ls
2buntu-django-blog-nitins-fork  hhn          py_django_tut_setup_hist_dump.txt
2buntu_hist_dump.txt            hn           temp
Desktop                         hnews        Templates
Documents                       Music        Ubuntu One
Downloads                       pgadmin.log  Videos
examples.desktop                Pictures     VirtualBox VMs
exp-django                      Public

Notice a couple of things:

1. Creating a project, automatically activated the virtual environment without you having to source the activate file.
2. A new directory with the project name was not created.

Now, peek into the ~/.virtualenvs directory, you'll notice that a coolpyapp directory was created there instead. In this, you'll find all the directories and pip and what not, things that would've been in the folder if you created a virtual environment using virtualenv.

nitin@jane-saucy:~$ ls -laF ~/.virtualenvs/coolpyapp/
total 24
drwxr-xr-x 6 nitin nitin 4096 Feb 15 02:52 ./
drwxr-xr-x 6 nitin nitin 4096 Feb 15 02:52 ../
drwxr-xr-x 2 nitin nitin 4096 Feb 15 02:52 bin/
drwxr-xr-x 2 nitin nitin 4096 Feb 15 02:52 include/
drwxr-xr-x 3 nitin nitin 4096 Feb 15 02:52 lib/
drwxr-xr-x 2 nitin nitin 4096 Feb 15 02:52 local/

So now you understand why we added the environment variable called WORKON_HOME to our .bash_profile (or .bashrc). The virtualenvwrapper script is sourced everytime you fire up a shell, and it looks into the WORKON_HOME environment variable to find the directory where all the configurations of all the virtual environments created are stored and is ready to serve you.

Using the virtual environment:

I am betting you're curious why in the world the environment variable is called WORKON_HOME. In order to use a virtual environment you use the command workon project-name with virtualenvwrapper. Curiousity satisfied?

Simply typing workon without being followed by a project name will list all your projects (virtual environments). You can then switch to the environment of your choice and all it's configurations in it's full g(l)ory will be loaded. To get out of the environment, we use the same deactivate command as in virtualenv.

nitin@jane-saucy:~$ workon
coolpyapp
django_learn
nitin@jane-saucy:~$ workon coolpyapp
(coolpyapp)nitin@jane-saucy:~$ echo "Blah"
Blah
(coolpyapp)nitin@jane-saucy:~$ deactivate
nitin@jane-saucy:~$

You can also set project directories with virtualenvwrapper which when you switch to a virtual environment with the workon command will put you in the project directory automatically. To set the project directory we navigate to the project directory and from inside the project directory, use the setvirtualenvproject command.

Walkthrough (Setting the project directory):

1. Creating the project directory and cd-ing into it.

   nitin@jane-saucy:~$ workon coolpyapp
   (coolpyapp)nitin@jane-saucy:~$ mkdir coolpyapp
   (coolpyapp)nitin@jane-saucy:~$ cd coolpyapp/
   

2. Setting the project directory.

   (coolpyapp)nitin@jane-saucy:~/coolpyapp$ setvirtualenvproject 
   Setting project for coolpyapp to /home/nitin/coolpyapp
   (coolpyapp)nitin@jane-saucy:~/coolpyapp$ deactivate
   nitin@jane-saucy:~$
   

3. Activating the virtual environment now will directly put you in the project directory no matter where you are on the filesystem.

   nitin@jane-saucy:~/coolpyapp$ cd /var/www
   nitin@jane-saucy:/var/www$ workon coolpyapp
   (coolpyapp)nitin@jane-saucy:~/coolpyapp$
   

<hr />

Pip

Pip is a Python Package Manager. Most of the time it's used to download a list of packages from a requirements.txt file which in turn are downloaded from PyPI. You could also use it to download individual packages from PyPI. We'll be doing that in our examples.

When you create a virtual environment for a project, pip is installed by default in it to give you a head start.

nitin@jane-saucy:~$ ls ~/.virtualenvs/coolpyapp/bin/
activate          easy_install      pip2            preactivate    python2.7
activate.csh      easy_install-2.7  pip2.7          predeactivate
activate.fish     get_env_details   postactivate    python
activate_this.py  pip               postdeactivate  python2

So now, you can get right down to installing packages specific for your virtual environment. In our example, I am going to install Django for our coolpyapp project.

(coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip install Django
Downloading/unpacking Django
  Downloading Django-1.6.2-py2.py3-none-any.whl (6.7MB): 6.7MB downloaded
Installing collected packages: Django
Successfully installed Django
Cleaning up...
(coolpyapp)nitin@jane-saucy:~/coolpyapp$ django-admin.py --version
1.6.2

Notice you don't need to add sudo to install packages? That's the advantage of virtual environments, you are encouraged to break things :P

Pip Quickie - Frequently Used Commands:

• To install a specific version of a package - pip install package-name==version-number. Example:

  $ pip install Django==1.5.5
  

• To list all the installed packages - pip list

  (coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip list
  argparse (1.2.1)
  Django (1.6.2)
  pip (1.5.2)
  setuptools (2.1)
  wsgiref (0.1.2)
  

• To show information about a specific installed package - pip show package-name

  (coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip show pip
  ---
  Name: pip
  Version: 1.5.2
  Location: /home/nitin/.virtualenvs/coolpyapp/lib/python2.7/site-packages
  Requires:
  

• To generate output suitable for a requirements.txt file based on currently installed packages - pip freeze

  (coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip freeze
  Django==1.6.2
  argparse==1.2.1
  wsgiref==0.1.2
  

• To uninstall a certain package - pip uninstall package-name

  (coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip uninstall Django
  Uninstalling Django:
  ...
  ....
  .... Output skipped to save space
  ....
  ....
  Proceed (y/n)? y
    Successfully uninstalled Django
  (coolpyapp)nitin@jane-saucy:~/coolpyapp$ pip list
  argparse (1.2.1)
  pip (1.5.2)
  setuptools (2.1)
  wsgiref (0.1.2)
  

And that's that. Hope you found this post interesting enough to get to the bottom line here. Sorry about dragging this post too long.

Cheers!

Cross posted from my blog