Crosstown

Pre-compiled wheels and installation recipes for simple, speedy, and slim Python deployments.

NOTE: This service is in its early stages and shouldn't be relied on in production!

Problem Being Solved

Crosstown solves a few problems:

Simplified System Dependencies

Building wheels and managing system dependencies across development environments, CI, and Dockerfiles is a hassle and wastes developer time.

Given the below set of requirements which Dockerfile would you rather write and maintain?

# requirements.txt

psycopg2-binary
flask
sqlalchemy

Without Crosstown:

FROM python:3.8-alpine

COPY requirements.txt /
RUN apk update && \
    apk add postgresql && \
    apk add musl-dev python3-dev gcc postgresql-dev --virtual=.build-deps && \
    python -m pip install -r requirements.txt && \
    apk del .build-deps

COPY app/ /app

With Crosstown:

FROM python:3.8-alpine

COPY requirements.txt /
RUN python -m pip install crosstown && \
    crosstown install -r requirements.txt

COPY app/ /app

You'll receive the same size container except you don't have to worry about system dependencies and the wheel for psycopg2-binary will come pre-built so the Docker container takes seconds to build rather than minutes.

Container Build Speed

Having to re-compile wheels every time you change your application dependencies or upgrade versions is a painful process for developers and CI alike. Caches can only save you if you don't change anything, which as we know isn't true often in software. :)

Container Size

A common work-around for solving the binary / system dependency problem for Python, especially in the scientific world, is to use Conda.

One of the most popular images for Miniconda and Python 3 on Alpine 3.10 starts at ~80 MB but after installing a single dependency, numpy, becomes 1.5 GB. For comparison Python 3.7 on Alpine 3.10 starts at 32 MB and after installing numpy with Crosstown is 172 MB.

Save the Environment

Google Cloud is carbon-neutral and hosts all the infrastructure for Crosstown as well as the VMs where all wheels are compiled. This means all the wheels from Crosstown are green wheels. :)

This can help drive down energy costs for your business as well as save the environment.

Links

Prior Art

This project is heavily inspired by Piwheels and Conda

About the Name "Crosstown"

"Crosstown" is the local name for Hennepin county road 62 in Minnesota. I probably drive on the road at least once a week. I felt the name fit well because of Python 'wheels' and 'speeding things up'.