Google has solved the concurrency limitations introduced by the Global Interpreter Lock existent in CPython by creating a new Python runtime in Go.
Google uses Python for YouTube’s front-end and APIs running on CPython 2.7, the reference implementation of the language’s interpreter. The code has grown to millions of LoC over the years, and generally served them well after some performance tweaks of the runtime. But they have come against one problem that kept showing up: “it’s very difficult to make concurrent workloads perform well on CPython,” according to Google.
After investigating a number of other possible solutions, Google has decided to try and run Python code on Go’s runtime due to its strong support for concurrency. The Grumpy project was born. With Grumpy one can trans-compile Python code into Go source code which is then compiled into native code based on the target platform. The intermediary Go code is a series of operations applied to data structures representing Python objects.
