Lint, Linting, and Linters (and Vim)

The Theory of Lint, Linting, and Linters

The Oxford Universal Dictionary (1955) defines a "linter" as "A machine for removing short-staple cotton-fibre from cotton-seed after ginning; the fibre thus obtained, used in making mattresses, etc." In our more modern context, "lint" isn't just the unwanted fluffy stuff out of the dryer, it's also bad or unnecessary stuff in programs. Lint doesn't usually cause show-stopping bugs, but cleaning it out can often prevent small or annoying misbehaviours in programs.

A simple example is "HTML Tidy," a linter for HTML. It notifies you of things like missing open/close tags, missing alt tags for images, id anchors defined more than once or id anchors that start with numerical values. Most browsers would still render your HTML fine - but inconsistent HTML means inconsistent rendering across different browser engines, and duplicate id anchors are guaranteed to cause unexpected behaviour.

The Philosophical Side of Lint

I was amazed at how fast linting (ah, that snuck up on you - it's a verb, too) becomes philosophical. One of the linters for Python (there are several, as there are for most established languages) is flake8, which combines PyFlakes and pep8 (two other linters). pep8 is a linter based on PEP 8, which is a style guide for Python. What's interesting about this is how widely this style guide is enforced: in most languages, style guides are considered to be suggestions: in the Python community, PEP 8 seems to be almost required if you're going to present your code in public. One of the things that attracted me to Python in the first place was the REQUIRED indenting of code blocks: in most languages, code blocks are delimited by braces (or some form of marker) and indentation is recommended but a programmer choice. But not in Python: no braces, and you indent that loop or it isn't a loop. A language like that is hard-ass about coding conventions - so I'm not sure why I'm surprised PEP 8 is so strongly enforced.

If you run flake8 against your code (or at least my code ...) it complains about things like "variable defined but never used" (that's a pyflakes thing). I would argue that's a legitimate usage: it's available for a feature that I intend to develop. It also says "at least two spaces before inline comment" and "line too long (82 > 79 characters)" - both of which are PEP 8 things. The first is hardly onerous, but the second is annoying: I have an inline comment, and over-running the line length by three characters doesn't seem egregious. And another: "multiple spaces before operator":

variable = "something"
var2     = 5

I think that's an entirely legitimate way to improve the readability of code - apparently PEP 8 doesn't agree. You can't do this:

variable = "something"
    var2 = 5

This is completely illegal per the language itself because an indent means a new code block. So you're screwed for making it pretty/readable if you want to follow PEP 8 - even though readability and consistency are the stated aims of PEP 8. (I could of course actually READ PEP 8, but I haven't done that yet because I suspect I'd be snoozing within two paragraphs.)

Getting Linting Working in Vim

Linters were initially written to be run by hand, usually with something like this:

$ flake8 program.py
program.py:124:9 E221 multiple spaces before operator
program.py:125:12 E221 multiple spaces before operator
...

This can be useful, but most linters are now designed primarily to be used inside an editor, with the ability to jump to the next warning or error and fix it per the listed error (which will simultaneously be visible on screen).

My editor of choice is Vim. For better or worse, Vim has a complex enough plugin architecture that there are multiple package managers for Vim ... the one I use is Vundle, so that's the solution shown here. Likewise, there's more than one way to get syntax checking in Vim, but I've chosen Syntastic. But Syntastic doesn't do syntax checking/linting on its own: it requires language-specific linters to be installed. So this isn't a simple process. It's not difficult, but there are many steps. Let's start with getting Syntastic:

" ~/.vimrc
set nocompatible
filetype off                  " required by Vundle
" set the runtime path to include Vundle and initialize
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
" alternatively, pass a path where Vundle should install plugins
"call vundle#begin('~/some/path/here')

" let Vundle manage Vundle, required
Plugin 'VundleVim/Vundle.vim'
" Airline:
Plugin 'vim-airline/vim-airline'
" NEW!! Syntastic:
Plugin 'scrooloose/syntastic'
" All of your Plugins must be added before the following line
call vundle#end()            " required
filetype plugin indent on    " required

I've recreated the section of my ~/.vimrc that has the Vundle settings in full: the only line that's new is Plugin 'scrooloose/syntastic'. Once that line is added, start Vim and run ":PluginInstall".

Next step is to get the linters. Follow this link to Syntastic supported linters, and click through to your version of Syntastic. Or (simpler but less robust) just continue to follow my instructions.

I generally try to go with the linter for a language that's packaged for Debian: it's faster and cleaner to install, and the fact that it's packaged implies (just that, nothing more) that it's well regarded and well supported.

For bash:

# apt-get install shellcheck

For Python:

# apt-get install python3-flake8 python-flake8

For HTML:

# apt-get install tidy

VimScript is trickier. First, neither of the currently supported linters are packaged for Debian (not too surprising, it's a relatively obscure language). Second, despite the fact that the Syntastic documentation says it supports vint, it doesn't automatically detect it. First step, install it:

# pip install vim-vint

[2018-10 Update: Fedora at least has a OS-level 'vint' package: I now prefer that as I find I keep up on my OS-level upgrades much more reliably than I track the 'pip' stuff - this may be different for you.]

Yup, that's a Python package. Now, add a line to your ~/.vimrc:

let g:syntastic_vim_checkers = ['vint']

And finally add a configuration for the linter itself (this isn't necessary: mostly I'm increasing the number of warnings from the default):

# ~/.vintrc.yaml
cmdargs:
  # Checking more strictly
  severity: style_problem

  # Enable coloring
  color: true

Which points out that linters have their own configurations, so if I wanted to read up on it rather than griping, I can probably switch off the PEP 8 "multiple spaces before operator".

There are many choices for JavaScript, the easiest for me was this:

# apt-get install closure-linter

This includes the gjslint executable which Syntastic recognizes but doesn't automatically turn on:

" add this to ~/.vimrc
let g:syntastic_javascript_checkers = ['gjslint']

The linter for Python's RST files was automatic: if you have the python-docutils package in Debian (and you probably have to have it if you're dealing with RST files), it includes rst2pseudoxml which Syntastic automatically uses as a linter for RST.

There are many, many more linters for many languages, these are just the ones I tracked down and tested in the last couple days.

Your Starter Commands

:help syntastic is a great place to start. The commands I most wanted to know were :lnext and :lprevious to move quickly between the tagged errors/warnings. I recommend mapping these to easier commands (I used "-y" and "-Y" respectively, where "-" is my mapleader). :SyntasticToggle is good too, to just turn it off (or at least put it in passive mode).

Gripes

Linters can be annoying, as noted above. An example from shellcheck: "Double quote to prevent globbing and word splitting." Usually a correct suggestion, but when applied to if [ ${#PWD} -gt ${4} ] it's incorrect because these should both be numerical values. But it doesn't issue a warning on if [ $# -eq 3 ] because it's recognized those as numeric.

Another example: I used ls -l | grep "^-" | wc -l and it issues a warning: "Consider using grep -c instead of grep|wc." This was totally correct, and I fixed it later, but first I tried an experiment, changing it to command ls -l | command grep "^-" | command wc -l. It ceases to recognize the problem because (I suppose) it only parses the first command, which is now literally command, not grep and wc.

vint can't currently deal with ^[ - the escape sequence for the Escape key that can be generated by typing <Ctrl-V><Esc>. (This is a known bug.) If you have that character in a VimScript file, vint will fail with:

.vim/vimrc:0:0: Cannot detect encoding (binary file?): .vim/vimrc (see no reference)

This can be "fixed" (in quotes because this is vint's problem, not yours) by replacing the escape sequence with the literally typed <Esc>.

Despite all of which, I know it's going to at least somewhat improve the quality of my code, and an improvement - even a small one - is welcome.