Lately, I have been working on a Python library for converting PMML represented machine learning models into Python scikit-learn classifiers that can be readily used for predictions. When setting up Travis-CI for automated code testing, I learned a few tricks and tips that are specific to scientific Python packages, such as numpy and scipy. If you are setting up Travis-CI for tests that are dependent on these Python packages and having build errors, hopefully this post will give you some ideas that you can test out.

pip or conda

I will write about installing Python packages in Travis-CI environment with pip and conda.

Package Dependencies

My unittest code requires Python's machine learning library, scikit-learn. Scikit-learn package depends on scipy. Scipy depends on numpy, which requires Fortran packages LAPACK and BLAS.

While I've been happy with using pip for installing Python packages, I realized that it isn't efficient in installing bulk packages with complicated dependency relationship. Pip would attempt to install the last package on the dependency chain, and abort the build:

When I specified the order of packages to install by writing it an individual command line for each one of them, I ran into another issue, which is related to non-Python packages.

non-Python Packages

Scipy and numpy are written heavily in C to speed up the mathematical computations. Numpy's required packages include Fortran packages LAPACK and BLAS.

Pip is a Python package installer, and it isn't great at installing non-Python packages. I noticed numpy and scipy will fail to install correctly because Travis could not find LAPACK and BLAS. Even after I tried installing them manually, the error would persist.

conda to the Rescue

Conda is great for installing scientific packages. After reading many entries on Stack Overflow, I decided to try installing packages with Conda, and the environment was able to install the packages correctly! However, I ran into ImportError.

Python Unittest Packages

Conda ignores any packages installed in the virtual environment, and therefore nose that was pre-installed in Travis had to be installed again. In fact, in virtual environments, ignoring pre-installed packages is an advantage, because it allows the user to paint on a completely blank canvas, rather than a canvas with some streaks drawn on. So I added nose to the packages to install, and voila, my build was successful, and so were my tests!

Hope this post was helpful for you if you are about to build a scientific project on Travis. Thanks for reading!