Exploring Large and Unfamiliar Python Projects in Emacs


Posted by alex


As programmers, we spend much more of our time at work reading code and doing code audits than writing it. Some of the code you're reading may not have been written by you. Reading code is an important skill that must be developed by every programmer in order to be successful in their job.

I'm not talking about reading code samples here. I'm talking about being handed a 50,000 line repository of someone else's code. How do you get up to speed with such a repository? How do you figure out which bits and pieces need changing? Perhaps a client is coming to you with an emergency, in which case you don't have time to get up to speed with all 50,000 lines. You need to be able to go directly to the source of the problem, diagnose it, and fix it.

Like any skill, exploring large codebases is one that is developed over time with practice. Nothing can replace experience, but there are certain tools that can help improve your workflow significantly.

I use Emacs as my editor of choice. I have been using Emacs for probably about 10 years now, but it wasn't until I started this job that I really took the time to configure Emacs to help me explore code. Many of these tools will work with any language, but as a Python developer, I include tools that help me explore Python code specifically.

First, I would like to introduce Helm and Projectile (and the glue than binds them together, predictably called Helm-Projectile).

Helm-Projectile Demo From https://tuhdo.github.io/helm-projectile.html

Helm is a tool for fuzzy searching through anything and everything. It doesn't come pre-loaded with things to search through. This is where Projectile comes in!

Projectile is an interface for searching through a git repository. When configured correctly, Projectile populates Helm with the list of files in the repository. You can then use Helm to filter them with fuzzy matching and open the file that you want in a buffer.

Once you find the file you're looking for, jedi.el comes into play. Jedi.el has a huge number of useful features for working with Python code, but by far the most useful feature to me is the ability to jump to the definition of almost anything. Put your marker above a function call and (in my case) hit C-. and the marker will jump to the definition of the function if it's in the same file, or to the import statement if it's not. Hit the same chord again on the import statement and Emacs will open the file that the function is imported from and scroll down to where the function is defined. It will even jump into 3rd party libraries to show you the function definitions.

Jumping back is just as easy. Every time you press C-., your position is pushed to a stack. Popping locations off the stack is made possible with C-,, which takes you back one jump.

The tricky thing with using Jedi is that it has to be able to run your Python code to be able to give suggestions and provide the jump-to-definition function. In keeping with Python development best practices, I keep all of my various projects siloed in their own virtual environments. This makes it difficult to configure Emacs' and Jedi's Python interpreter to use the correct virtualenv. Projectile helps us a little bit here. I use this snippet in my .emacs file to configure the Python virtualenv based on the project I'm working on:

    (defvar my:virtualenv-directory "~/.virtualenvs/"
      "The directory of virtualenvs.")

    (defun my:configure-python-venv ()
      "Set `python-shell-virtualenv-path' to the virtualenv directory."
      (require 'projectile)
      (let* ((project-name (projectile-project-name))
               (concat my:virtualenv-directory project-name))))
        (when (file-directory-p virtualenv-path)
          (setq python-shell-virtualenv-path virtualenv-path))))

Finally, I use Magit to create commits in Git and to push and pull from repositories. I don't understand it as well as I understand the command line, so I find myself occasionally popping back to the terminal to manage a complex merge or rebasing.

Viewing git state with Magit

I've only scratched the surface of what's possible. Reading the documentation of these projects will reveal a whole lot more.

Learn more about Fusionbox, a python development company.

Return to Articles & Guides