PyPy: Faster Than an Unladen Swallow (Part 1)

Much of Python’s success as a language—including it’s widespread adoption, extensive software ecosystem, and ease of use—is due to the robust design of the language’s official interpreter which actually implements the functions of the language.. This implementation is called “CPython” because it is written in C. Python’s syntax borrows heavily from C and some Python modules are also written in that low-level compiled language, but Python user’s aren’t necessarily bound to it.

The allure of leveraging other languages, libraries, frameworks, and development tools is one key reason why Python developers have built and used alternative Python implementations. Jython, for example is brings Python 2.7 to the Java virtual machine (VM). For developers who love to use Microsoft’s .NET Framework, IronPython enables them to utilize both .NET and Python libraries while enabling other .NET languages to use Python code.

Different Runtimes for Different Needs

Developers also choose to try other Python implementations as a way of increasing the performance of existing or new Python code. While careful optimizations can increase execution speed and slash memory usage on CPython, it’s often easier (and safer) in practice to run unmodified code on another Python implementation. After this initial performance boost, the usual optimization techniques can still be applied (avoiding unnecessary function calls, eliminating redundant copies of the same data, etc . . .) in a way that targets that particular implementation.

One such Python runtime that created a lot of buzz was Pyston, a just in time (JIT) compiling runtime for Python which was sponsored by Dropbox and discussed in our post from last January. Just over two weeks after our article went up however, Dropbox (the current employer of Python creator and BDFL Guido van Rossum) halted its support of Pyston, leaving just one other major performance-oriented Python runtime: PyPy.

Before we talk about PyPy though, we should also mention Cython, which according to the project’s website is, “an optimising static compiler for both the Python programming language and the extended Cython programming language”. Although Cython’s compiler can in fact improve the performance of unmodified Python code, the biggest gains come from adding static type declarations to the code before compiling. Interestingly, combining PyPy and Cython can lead to impressive speedups it some use cases, as the benchmarks from this article demonstrate. We’ll be covering Cython in a future article.

What is PyPy?

PyPy is JIT compiler (like Pyston) which is largely compliant with both Python 2.7 and 3.5, allowing it to run many Python libraries including Django and NumPy. While PyPy does mostly support the CPython C API, CFFI is the better choice for integrating C code (like some Python modules) into code that’s run on PyPy.

Just like Pyston and Cython, PyPy’s main goal is to make existing Python code execute faster and consume less memory than CPython. The PyPy project runs nightly speed tests and posts the results, currently showing that on the whole PyPy is 7.6 times faster than CPython.

In the next article, we’ll dive deeper into why PyPy is able to thrive while Pyston couldn’t even survive, how it speeds up Python code, and discuss how coders can best harness PyPy’s potential from their existing codebase.