Interesting Talk: "Side Effects are a Public API"

I've just watched this very interesting talk by Christopher Armstrong

• Side Effects are a Public API

Interesting Talk: "Beyond PEP 8, Best practices for beautiful intelligible code"

I've just watched this great talk by Raymond Hettinger

• Beyond PEP 8, Best practices for beautiful intelligible code

Kata: Password validation in Python

Yesterday I did the Password validation kata in Python.

I used TDD to write the code.

These are the tests using Pytest:

and this is the resulting code:



If you want to follow the process step by step, have a look at the commits.

You can find all the code in GitHub.

Interesting Talk: "Django's architecture - the good, the bad, and the ugly"

I've just watched this interesting talk about Django's architecture by Andrew Godwin:

• Django's architecture - the good, the bad, and the ugly

Andrew Godwin is a Django core developer and the creator of the great South, "a Django application to provide schema and data migrations".
In this talk, he gives a good overview of Django's architecture and his opinion about which are the good, bad and ugly parts of Django.

Interesting Talk: "Django: Web Development for Perfectionists with Deadlines"

I've just watched this interesting talk about Django by Jacob Kaplan-Moss:

• Django: Web Development for Perfectionists with Deadlines

No doubt, this talk is out of date now. Django has evolved a lot since 2006, but I think it's good to understand the needs that originated the development of Django in the first place.

Embedding Python in C++: Hello World

This afternoon I've been "procrastiworking" (working on something that isn't mandatory to postpone doing something mandatory) a bit.
As a result I end up watching this talk by Michael Fötsch:

• Supercharging C++ Code with Embedded Python

I just couldn't get to the end of the talk.
When I saw the "Hello world" example, I had to stop the video and try it myself.
This is the code snippet of the experiment:

#include <Python.h>
int main(int argc, char* argv[])
{
  Py_Initialize();
  PyRun_SimpleString(
    "print 'Hello world!'"
  );
  Py_Finalize();
  return 0;
}

This is how I compiled it:

$ g++ -o helloWorld helloWorld.cpp -I/usr/include/python2.6/ -lpython2.6

And this is the result of running it:

$ ./helloWorld
Hello world!

Nice!!
This is just the tip of the iceberg. To learn more about embedding Python in C++ watch Michael's talk.
He also has more advanced material in his blog:

• Embedding Python – Tutorial – Part 1 

There's also more information in Python's Documentation:

• Embedding Python in Another Application 

and in this Jun Du's tutorial:

• Embedding Python in C/C++: Part I

Now you also have material to procrastiwork for a while.
Have fun!

How to install an alternative version of Pyhton and use it in a virtual environment

I've decided to learn a bit more of Python. My friend @remosu recommended me to use virtual environments so that I can practice with different versions of Python.

First I installed an alternative version of Python on my Ubuntu following the first two steps in this great post by Eli Bendersky:

• Installing Python 2.7 on Ubuntu 

I installed first some required packages:

$ sudo apt-get install libreadline-dev
$ sudo apt-get install libsqlite3-dev
$ sudo apt-get install libbz2-dev
$ sudo apt-get install libssl-dev

Then I downloaded Python from http://www.python.org/, configured and built it:

$ ./configure
$ make -j

Then I stopped following Eli's post because I wanted to keep the version that was already installed on my computer instead of replacing it with a new version.

I started googling how to it and after a while I found this discussion in Stack Exchange Unix & Linux:

• Using different versions of Python 

In there I found out that the "trick to easier installation of multiple interpreters from source" was using (thaks to vperic's answer):

$ sudo make altinstall

After doing that I had two Python versions living together in the same Ubuntu. Then I only had to use virtualenvwrapper to create my virtual environment.
First I installed virtualenv and virtualenvwrapper using pip:

$ pip install virtualenv
$ pip install virtualenvwrapper

And executed virtualenvwrapper.sh:

$ export WORKON_HOME=~/Envs
$ mkdir -p $WORKON_HOME
$ source /usr/local/bin/virtualenvwrapper.sh
virtualenvwrapper.user_scripts creating /home/myuser/Envs/initialize
virtualenvwrapper.user_scripts creating /home/myuser/Envs/premkvirtualenv
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postmkvirtualenv
virtualenvwrapper.user_scripts creating /home/myuser/Envs/prermvirtualenv
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postrmvirtualenv
virtualenvwrapper.user_scripts creating /home/myuser/Envs/predeactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postdeactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/preactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/get_env_details
virtualenvwrapper.user_scripts creating /home/myuser/Envs/premkproject
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postmkproject
virtualenvwrapper.user_scripts creating /home/myuser/Envs/prermproject
virtualenvwrapper.user_scripts creating /home/myuser/Envs/postrmproject

Finally, I created a virtual environment with python 2.7 called learning_env:

$ mkvirtualenv --python python2.7 learning_env
Running virtualenv with interpreter /usr/local/bin/python2.7
New python executable in learning_env/bin/python2.7
Also creating executable in learning_env/bin/python
Installing setuptools............................done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /home/myuser/Envs/learning_env/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/learning_env/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/learning_env/bin/preactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/learning_env/bin/postactivate
virtualenvwrapper.user_scripts creating /home/myuser/Envs/learning_env/bin/get_env_details
(learning_env)~$

To check that the virtual environment really had the right version of Python I did:

$ workon learning_env
(learning_env)bscuser@trikitrok:~$ python
Python 2.7.3 (default, Nov 21 2012, 01:38:50) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Then I got out of the virtual environment and checked that I still had the same version of Python on my Ubuntu:

$ workon learning_env
(learning_env)~$ deactivate
$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Now I have my own learning environment to play with Python 2.7.

Interesting Talk: "Fake It Til You Make It: Unit Testing Patterns With Mocks and Fakes"

I've just watched this interesting talk by Brian K. Jones:

• Fake It Til You Make It: Unit Testing Patterns With Mocks and Fakes

The author talks about unit testing and mocks in Python projects.
Also have a look at his github repositories.

Interesting Talk: "Stop Mocking, Start Testing"

I've just watched this interesting talk by two Google engineers Augie Fackler and Nathaniel Manista:

• Stop Mocking, Start Testing

The speakers explain their experiences in testing a big project written in Python and how their testing style has evolved with time.

Interesting Talk: "Don't call us, we'll call you: callback patterns and idioms"

I've just watched this interesting talk by Alex Martelli:

• Don't call us, we'll call you: callback patterns and idioms

Youl find the talk slides here.

Last Pet Project: backmongo

@remosu and I have been working during the last months in another pet project: backmongo.

backmongo is a simple REST interface for MongoDB written in Python that can be used straight away from a Backbone application.

We plan to use it in our next backbone pet projects, so we'll add new features as we need them.

Last Pet Project: Event-Driven Molecular Dynamics

During the last week @remosu and I have been pair-programming a pet project: edmd.

It is a Python implementation of a simple Event-Driven Molecular Dynamics algorithm described in the Event-Driven Molecular Dynamics chapter of the book Computational Granular Dynamics: Models and Algorithms by Thorsten Pöschel and Thomas Schwager.

We plan to revisit edmd in the future to optimize the event-updating phase by using a cell list.

Streamlines with Matplotlib

This is a translation into English of an old post in Spanish.

As an assignment for the Numerical Methods for Dissipative Systems course I had to solve the 2D Laplace equation in a rectangle with some given boundary conditions.
I wrote a program to solve it in C, but to do the streamline graphics I used MatPlotLib.
This is the Python script that generated the streamlines:

# -*- coding: iso-8859-15
import sys
from numpy import * 
from pylab import load  
import matplotlib.pyplot as plt
if len(sys.argv) >= 1:
    psi = load(sys.argv[1])
    cs = plt.contour(psi,20)
    plt.clabel(cs)
    plt.xlabel("y/h")
    plt.ylabel("x/h")
    plt.show()
else:
    print "This program needs the path to the file that contains the psi matrix"

To generate the streamlines we should execute the script passing to it the path of the file where the solution matrix is. In this case the entries of the matrix must be separated by white spaces. Reading the help of the load function, you can learn how to modify it to use different delimiters.

I run the script from iPython:

$ ipython
Python 2.5.2 (r252:60911, Dic 20 2009, 23:16:55) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.4 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: run ~/grafico.py 'pathToFileWithSolutionMatrix'

and this was the result:

To be able to run it, besides python, you'll need to install the following modules: numpy, pylab and matPlotLib (you can find them all in the Ubuntu repositories).

Python idioms: Building strings

I'll start a series of posts about Python idioms.
What is a programming idiom? According to Wikipedia's Programming Idiom article, a programming idiom is defined as "the use of an unusual or notable feature that is built in to a programming language".

Python is very rich in useful idioms. Python programmers who take advantage of them are known as pythonistas. On that sense, I highly recommend you to read the famous article Code like a Pythonista: Idiomatic Python by David Goodger.

Once defined the concept of programming idiom and introduced the concept pythonista, let's talk about one of favorite idioms: building strings from substrings.

Imagine that you have a list of strings,

bands = ['Machine Head','Metallica','Opeth','Veil of Maya']

and you want to concatenate each item in the list to form a unique comma separared string. If you come from the C, Java syntax world, you would write something like this:

output = ''
for band in bands[:-1]:
    output += band + ', '
output += bands[-1] 

If you print the content of the output variable, you will get a list of the bands:

print output
>>> Machine Head, Metallica, Opeth, Veil of Maya

This is a very inefficient way to concatenate strings in Python because in each iteration of the for loop, a temporal string is generated before the string addition and thrown away after.

The pythonic way is faster and more elegant:

>>> print ', '.join(bands)
Machine Head, Metallica, Opeth, Veil of Maya

I coded a small example to compare the performance of both techniques:

from functools import wraps
import time

def timed(f):
  @wraps(f)
  def wrapper(*args, **kwds):
    start = time.clock()
    result = f(*args, **kwds)
    elapsed = time.clock() - start
    #print "%s took %d time to finish" % (f.__name__, elapsed)
    print "%.5gs" % (elapsed)
    return result
  return wrapper

bands = ['Machine Head','Metallica','Opeth','Veil of Maya']*100000

@timed
def func1():
  output = ''
  for band in bands[:-1]:
    output += band + ', '
  output += bands[-1]
  return output

@timed 
def func2():
   output = ', '.join(bands)
   return output

func1()
func2()

The result is:

$ python test.py

0.07s

0.01s

If we increase one order of magnitude the bands list:

$ python test.py

0.62s

0.1s

the difference in performance becomes more relevant.

PS: In my toy example, I used the timed decorator that can be found in this stackoverflow thread.

Configuring Eclipse for TDD in Python: The Nosetests way

In my daily work, Python is my main (and really beloved) programming language. For coding in Python, I usually work with vim in remote machines and Eclipse IDE in my desktop one. In the next sections I'll explain how to convert Eclipse in the killer TDD Python-IDE, but you can have a look at the official Python Wiki for more options: Python IDE's and Python Editors.

1. Installing Eclipse

In the Eclipse downloads section, get the Eclipse Classic package that corresponds to your platform (I'll focus on Linux in this post, but it should be easy to follow the same steps in Windows and Mac OS platforms as well).

Installing Eclipse is as easy as extracting the downloaded package if you have an existing Java Virtual Machine installed in your system. If not, check this FAQ.

2. PyDev

PyDev is a Python IDE for Eclipse with a lot of goodies such code completion, syntax highlighting, debugger, unit testing support and refactoring options. You can find a very nice installation manual in the offical project page: PyDev Manual (Installing). For lazy readers, it should be as easy as opening Eclipse, go to Help > Install New Software, adding a new software location with the Add button and fill in the location field with the url http://pydev.org/updates. Once added, check PyDev option and follow the instructions.

3. Nosetests

python-nose or nosetests is an unit test-based framework that is capable of discover and execute your tests without having to register them first. Nose makes my life much easier writing test functions to check for exceptions and discovering and executing my test packages. However, I strongly suggest you to read this article to understand which are the main motivations to use a unit test framework and why choosing nose.

Nose is available via setuptools:

$ easy_install nose

Now try to import nose in python:

$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nose
>>>

If nose installation is fine, you shouldn't see any error messages after importing the nose package in the Python console.

If you don't have setuptools installed in your system, there are a few more options to install nose package, check the official installation guide.

4. Configuring Eclipse for continuous-testing

First of all, create a new PyDev project: File > New > Project, navigate to PyDev and choose "PyDev Project". Click "Next", choose a project name (in our case "tdd-example"), select Grammar Version as 2.6 or 2.7 (nose supports Python 3 syntax, but many plugins not) and select the option "Add project directory to the PYTHONPATH?". Finally, click "Finish". You should see now in the PyDev Package Explorer view your new tdd-example project.

Now, go to Run >  External Tools > External Tools Configurations. In Program option, create a new one. Name could be "NoseTestTool", working directory will be "${project_loc}" and in location the path to nosetests binary, in my case "/usr/local/bin/nosetests".

Right click in the project tdd-example > Properties and navigate to Builders. Click in Import and choose the previous created NoseTestTool. Make sure is the first option selecting it and clicking the Up button. Now, click in Edit. In the Main tab, add in Arguments field "--processes=4" where processes is the number of simultaneous threads that will be used to execute your tests. In my case, I use 4 cores, so I set processes to 4.
In the Environment tab, create a new variable with name "PYTHONPATH" and value "${project_loc}".
In Build Options tab, check the options "Allocate Console (necessary for input)", "Launch in background", "After a Clean", "During manual builds" and "During auto builds".

To test that everything is configured fine, create a new Python file in your project, test_all.py and include the following code:

def test1():
    assert True
    
def test2():
    assert False 

At the time that you save the file, nose may execute your tests automagically and an ouput like this should be shown:

.F

======================================================================

FAIL: test_all.test2

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python2.6/dist-packages/nose-1.1.2-py2.6.egg/nose/case.py", line 197, in runTest

    self.test(*self.arg)

  File "/dev/workspace/python/tdd-example/test_all.py", line 6, in test2

    assert False

AssertionError

----------------------------------------------------------------------

Ran 2 tests in 0.002s

FAILED (failures=1)

Nose searches for files and functions that start with "test*". In our case, two functions will be tested, test1 and test2, and one of them will fail (assert False) as seen in the previous output.

I hope this howto will motivate people working with continuous TDD in Python :)

Miniprueba con SWIG

Hoy hice una pequeña prueba con SWIG.
El ejemplo que aparece en SWIG: donde C y Python se dan la mano (está muuuy bien explicado) hizo que me picará la curiosidad.

SWIG es una herramienta que permite conectar programas escritos en C/C++ con diferentes lenguajes de alto nivel (lenguajes que soporta).
En este caso, lo probé con Python y C.

Este es el header de mi código en C:

#ifndef __DATOS_PERSONALES_H__
#define  __DATOS_PERSONALES_H__

#include 
#include 

/* Tipos */
typedef struct {
  char nombre[150];
  int  edad;
}DatosPersona;

/* Prototipos */
extern int edad_get(DatosPersona a);
extern char * nombre_get(DatosPersona a);
extern void nombre_set(DatosPersona *a, char * nombre);
extern void edad_set(DatosPersona *a, int edad);
extern void muestra(DatosPersona a);

#endif /* __DATOS_PERSONALES_H__ */

Y este es el código en si:

#include "datosPersonales.h"

int edad_get(DatosPersona a)
{
    return a.edad;
}

char * nombre_get(DatosPersona a)
{
   return a.nombre;
}

void nombre_set(DatosPersona *a, char * nombre)
{
   strcpy(a->nombre, nombre);
}

void edad_set(DatosPersona *a, int edad)
{
    a->edad = edad;
}

void muestra(DatosPersona a)
{
    printf("Me llamo %s y tengo %d años.\n", a.nombre, a.edad);
}

Ahora viene lo interesante. Hay que definir una interfaz en un fichero con extensión .i, en la que se dan a SWIG instrucciones para construir el envoltorio para usar el código.
Hay muchas posibilidades, (la documentación es muy extensa), y después de pasarme unas horillas buscando llegué a lo siguiente:

%module datosPersonales
%{
#include "datosPersonales.h"
%}

typedef struct {
  char nombre[150];
  int  edad;
}DatosPersona;

%extend DatosPersona
{
    DatosPersona(char *nombre, int edad)
    {
        DatosPersona *datos = (DatosPersona *) malloc(sizeof(DatosPersona));
        strcpy ( datos->nombre, nombre );
        
        datos->edad = edad;
        return datos;
    }
    
    ~DatosPersona() 
    { 
        free(self); 
    }
    
    int getEdad()
    { 
        edad_get( *self ); 
    }
    
    char * getNombre()
    { 
        return nombre_get( *self ); 
    }
    
    void setNombre(char *nombre)
    {
        nombre_set(self, nombre);
    }
    
    void setEdad(int edad)
    {
        edad_set(self, edad);
    }
    
    void muestraDatos()
    {
        muestra(*self);
    } 
};

En esta interfaz se extiende el tipo de datos de C con una serie de funciones miembro que nos permitirán usar orientación a objetos en Python.

Para compilar hice lo siguiente:

$ swig -python datosPersonales.i
$ ls
datosPersonales.c  datosPersonales.i   datosPersonales_wrap.c  
datosPersonales.h  datosPersonales.py
$ gcc -c datosPersonales.c
$ ls
datosPersonales.c  datosPersonales.i  datosPersonales.py
datosPersonales.h  datosPersonales.o  datosPersonales_wrap.c
$ gcc -c datosPersonales_wrap.c -I /usr/include/python2.5/
$ ls
datosPersonales.c  datosPersonales.i  datosPersonales.py datosPersonales_wrap.o 
datosPersonales.h  datosPersonales.o  datosPersonales_wrap.c
$ gcc -shared datosPersonales.o datosPersonales_wrap.o -o _datosPersonales.so
$ ls
datosPersonales.c  datosPersonales.i  datosPersonales.py   datosPersonales_wrap.c  
datosPersonales.h  datosPersonales.o  _datosPersonales.so  datosPersonales_wrap.o

Ahora ya está todo listo para poder usar el módulo datosPersonales.py desde Python.

import datosPersonales as dp

datos = dp.DatosPersona("Lolo", 34)

print datos.getEdad()
print datos.getNombre()

datos.muestraDatos()

datos.setNombre("Javi")
datos.setEdad(31)
datos.muestraDatos()

del datos

Y esto es lo que resulta al ejecutar el script:

$ python runMe.py 
34
Lolo
Me llamo Lolo y tengo 34 años.
Me llamo Javi y tengo 31 años.

Este ejemplo no deja de ser una tontería, pero las posibilidades de SWIG son grandísimas (sólo hay que ver algunos de los artículos online que aparecen en Internet).
Más adelante, si tengo algo de tiempo, me gustaría correr alguna simulación escrita en C desde Python y usar MatPlotLib para hacer los gráficos.

Streamlines con MatPlotLib

Durante las vacaciones de navidad tuve que hacer un programa para resolver la ecuación de Laplace en 2D en un rectangulo con unas determinadas condiciones de contorno.
Hice el programa en C, pero los gráficos de las streamlines los hice con MatPlotLib.
Este es el script de Python que genera las streamlines:

# -*- coding: iso-8859-15
# grafico.py
import sys
from numpy import * 
from pylab import load  
import matplotlib.pyplot as plt
if len(sys.argv) >= 1:
psi = load(sys.argv[1])
cs = plt.contour(psi,20)
plt.clabel(cs)
plt.xlabel("y/h")
plt.ylabel("x/h")
plt.show()
else:
print "Este programa necesita la ruta del fichero que contiene la matriz psi"

Para ejecutarlo basta con llamar al programa pasándole como parámetro la ruta del fichero donde está la matriz. En este caso las entradas de la matriz están separadas por un espacio. Leyendo la ayuda de load se puede ver cómo usar otros delimitadores.

Lo ejecuté desde iPython:

$ ipython
Python 2.5.2 (r252:60911, Dic 20 2009, 23:16:55) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.4 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: run ~/grafico.py 'rutaFicheroMatriz'

y este es el resultado:

Para poder usarlo, aparte de Python, hay que instalarse los modulos: numpy, pylab y matplotlib (todo está en los repositorios)

Intermediate and Advanced Software Carpentry in Python

Intermediate and Advanced Software Carpentry in Python
Un curso bastante bueno de Python que toca temas más avanzados que los que aparecen en los cursos que ya había posteado, ver Python en Garajeando.

Libros gratuitos Python

Link con una lista de libros gratuitos de Python.

Software Carpentry

Encontré este sitio: Software Carpentry en el que explican muchas cosas muy útiles para desarrollar software.