A goal to dive into CPython

Some time towards the end of the semester, I made it my goal to work my way up to contributing regularly and meaningfully to CPython.

At the time when I first starting reading about the code base, it was not much more than an exciting form of procrastination: instead of doing my Neural Systems readings, I’d find myself staying up watching videos on CPython internals. It took awhile for me to realise what was happening, and now I’ve decided to go down a path that I have no desire to alter. I’ve decided to dedicate the last years of my university education to chasing this code base.

Why do I feel so strongly about this? It’s hard to pinpoint, but a few ideas have been stuck in my head that likely contributed to this decision.

Recently, I had a conversation with my computational probability instructor Professor Geman. He gave me two important pieces of advice: follow your nose, and get to the bottom of things.

Reflecting on this, I realised that for some time now, I haven’t exactly been following my nose. I’ve been going where others have expected me to go, and I’ve been chasing scents that other people seem to enjoy, even knowing that I’d been enticed by spicier things before.

More concretely, I realised that I’d been straying towards programming projects that I found less and less exciting. Still technically challenging, and still interesting, to be sure. But it became more uncommon for me to obsess over my work, and to feel hungry to code. This wouldn’t do. So I started to think back to the kind of programming that excited me, and noticed that I still felt that burning obsessiveness on occasion. I hadn’t been paying attention, but I’d felt it during my course on database management systems.

I am attracted to systems programming, and doing work that brings me closer to the metal. Such work, I’ve discovered, tends to make me feel like I’m… relaxing into the machine. It is a highly rewarding experience.

So to follow Professor Geman’s advice, I’m determined to chase this feeling down to the bottom of things. But to what problem should I devote my attention towards? Database systems? Tempting. I do plan to take a seminar with Professor Zdonik next semester to explore more topics in databases, and admittedly that could well evolve into something that might then take over my attention. But in the meanwhile, I came across the CPython repository.

I can’t remember what brought me there, maybe I had been trying to answer some student’s questions during office hours. But I do remember thinking that this project was humongous. I was also reminded that in my younger days, I did have a fascination with programming languages and compilers that I’d put aside as I pursued data science and machine learning at Brown. My interest was piqued.

It was then that I remembered an old blog post by Cal Newport on The Pyramid Method. The idea is simple: to develop one’s interest and talent into expertise, pick a venue that accepts entry-level work, but with ample opportunities for growth as well as clear metrics for progress. Then, put your head down and get to it.

I am ambivalent about my development as a programmer and computer scientist. I know that I’m effective, and that I’m comfortable enough with machines that I can do many otherwise daunting things on the computer. At the same time, I know that I’ve also barely stretched my wings, and that as a fledgling I have much more to learn. One of my weaknesses is not having much experience working on truly massive and complex code bases. I’ve had some opportunity during my last internship experience, but since then, it’s been back to school work and personal projects. I believe that to mature as a programmer, it’s imperative to learn how to grapple with such code bases, and the onus is on me to develop these skills.

CPython is truly massive. It’s been around for some 26 years, and as of this writing has 1,424 contributors on Github, as well as 1.4k open pull requests, and ~54k reported issues. Following the python-dev and python-ideas mailing list, it is also clear that this project has many ideas for short/medium/long-term development. As the standard implementation for what’s become one of the most popular programming languages and one of the go-to languages for new programmers, the success of CPython is important to a staggering number of people. Moreover, the Python community is friendly towards newcomers and accepts entry-level work.

This means that a goal of contributing to CPython is both productive as a venue for polishing one’s programming skills, and meaningful in terms of helping humanity.

In the past week I made my first contribution to the project. It was a minor bug, but the intent was to get a better understanding of the development process. I also wanted to kick myself into action as I’d been studying CPython and reading other books on programming in preparation for some time, but to no effect. Small as it was, I took pride in making sure I understood what change I was making, and added the relevant tests and documentation accordingly. I made the pull request with no small amount of trepidation. Over the next few days my pull request went through the review process, and polling how I felt internally, I realised that this was fun and, most importantly, it felt right. Like I was following my nose.

Now my plan is to finish a series of lectures on CPython’s internals (in addition to getting distracted by other materials on Python), then choose one area (maybe importlib?) and go for it. Next semester, I also plan to finally take the Compilers course at Brown.

Final thoughts

Of course, it’s hard to dedicate yourself to one thing. We are all creatures of many interests and conflicting motivations. We also like to hedge our bets. I had just started talking with a machine learning research group at Brown on a potential new project, and my humanity-centred robotics professor offered to work with me next semester to continue work on another project for a conference. These are both opportunities that I would’ve jumped on at the start of the fall semester.

But my nose is pointing to another direction, and I find that I must chase the name of this wind. I refuse to get lost again. I have to pursue the endeavours that set my heart on fire.