The Unavoidable Command Line
Over the years I have noticed an interesting phenomenon when it comes to developers and the use of the command-line interface (CLI). In most mainstream media, the depictions of “hardcore” developers involve the subject sitting in front of one or more screens, surrounded by, and working with, massive amounts of text. There can be graphs, images, and all sorts of interesting eye candy on the screens but the meat of what the developer is doing is clearly some insane amount of text, usually scrolling by far faster than most people could process.
In the real world of development, at least in the mainstream, this is not the case. I know and have worked with (and still do), many very capable developers who avoid using the terminal at all costs. They’ll engage with it only when it’s unavoidable and get out as soon as possible. I feel confident in saying that a large number of developers today feel uncomfortable, and in some cases, intimidated, frustrated, lost, or all of the above.
To be sure, there are always exceptions. The outliers, that actively avoid graphical interfaces, eschewing modern IDEs for a terminal text editor. But that’s exactly what they are; outliers. The reality is that every person who works as a developer for any length of time has to deal with the command line to some degree. It’s simply unavoidable.
There’s a social aspect that’s often overlooked as well. Developers are still people, regardless of stereotypes or personal inclinations. Age discrimination in the software industry is a serious topic, and it does exist. I’ve been on the receiving end of it once or twice. But judgment based on perceived skill with the CLI is a thing. Now, to be clear, I’m not saying that age discrimination is a trivial subject or that judgment over CLI skills is on par with that kind of social issue. One of the reasons for that is because, in mainstream conversation with anyone not involved with software development, it rarely comes up, and when it does I’ve usually seen it handled like this: “That’s only a thing in the movies, real devs don’t work that way. In reality it’s not that exciting”, and the conversation moves on. However, I have seen developers in conversation go from exuberant to silent when topics involving knowledge of the CLI come up. It’s almost as if they have a sudden bout of impostor syndrome because they aren’t the CLI genius that pop culture demands superstar developers must be.
Having said all of that, the CLI is just another tool in your kit as a developer. The only reason that GUI-based IDEs aren’t intimidating is that it’s easy to poke around menu options, and icons representing certain tasks tend to be the same regardless of what the IDE is, e.g., “Run program” is almost universally represented by some variation of an arrow. Most people don’t worry too much about being judged for not knowing a particular tool, be it an IDE, a web application, or an office app, if they’ve never used it extensively. CLIs should be regarded in the same light. Actually, it’s much more difficult to master than an IDE or spreadsheet. It’s more like learning a new language. It’s so incredibly expressive, and has so many little bits, that no one really “masters” it, not completely. The same thing can also be said for GUIs but the big difference here, again, is the perception, or maybe expectation, is a better word. While people understand that there is a difference between the skill levels of someone that opens IntelliJ the first time, and someone who has used IntelliJ for years, no one judges them for that difference.
This brings me back to the point of this article. Using the CLI is unavoidable, and frankly, it’s becoming less avoidable as the year's pass. Since sometime roughly around 2010, the use of the terminal has become more commonplace. This interestingly coincides with the rise of cloud computing. At that time, AWS had launched EC2 a few years earlier and it was starting to become extremely popular. Microsoft launched Azure, Rackspace and NASA launched OpenStack, and Google introduced Google Compute Engine. Over the past decade, the mainstream use of all of these cloud entities has continued to grow at a breakneck pace. The thing that all of them have in common? You guessed it, command-line use.
The CLI has undergone a resurgence over the past decade. There are simply things that can’t be done in a convenient or timely fashion on those platforms without the use of the CLI, and the existence of tools like gcloud, AWS CLI, and Azure CLI, make it clear that the providers understand the need. Even before all of this, containers like docker require at least some use of the CLI to be productive. There are countless articles extolling the virtues of the CLI ad nauseam, I’m not going to preach to you that the command line is better. Go find one of them if that’s what you’re interested in. I’m simply saying unless your career as a developer is extremely short, or extremely specialized, you’re going to deal with the command line. Having said all of that, let me be clear; I’m speaking in the context of greenfield development. If I’m talking about automating a CI/CD pipeline, I’m probably using API calls.
Like articles espousing the superiority of ad nauseam the CLI, there are articles ad nauseam that tell you how to learn your way around a terminal interface. I don’t know you, your learning preferences, and IMO that’s a topic better suited to a book anyway, so I’m not going to speak to that. What I do want to tell you is that regardless of your personal level of knowledge, the tools available through the terminal are virtually infinite, and much like the universe, they continue expanding all the time. From the shell that you’re using to the commands you have access to, it’s all larger than you think. There are certain things that I personally feel are relevant regardless of what I’m trying to learn, or the medium I’m using to learn it. One of those things is, that truly learning something new, or improving on what you already know, requires a deliberate effort. For me, one of the things that means is, if I don’t know how to accomplish it with the tool I’m currently using, do I know that tool well enough? I’ll give you a concrete example from something I did this weekend that most developers have probably experienced in some form.
I had a set of personal goals I wanted to complete this weekend because I wanted to play with some technology I’d never worked with before, specifically Redis. I had two GCP compute engine instances set up that I wanted to use because I don’t have the hardware to run meaningful performance testing on my laptop. I wanted to install Redis on both instances, populate instance 1 DB with some data from memtier-benchmark, and have that information replicated on instance 2. I also wanted to write a rudimentary client that I could use to insert data into instance 1, and read that data from instance 2, simply to demonstrate the replication worked remotely and get a feel for the libraries. How did I access these instances you ask? I used SSH to connect. Now I’m firmly rooted in the CLI universe, running bash as my shell. I downloaded and installed Redis OSS on each instance. How did I download it, you may ask? I used wget. Compiling Redis OSS on the first instance took more than a few seconds so I wanted to start compiling it on the second GCP instance while the first instance was still compiling, but if I dropped my connection to the first instance, I would have killed the job and aborted the compile. So just killing the connection and logging into the second instance to start the compile would have been pointless. My solution? I used tmux to create a second session in my terminal window and used the new session to connect to the second instance. There are a number of ways to accomplish that same task, I was simply already using tmux so that was the laziest way and I could easily keep an eye on progress. Once they had compiled, the redis.conf configuration file had to be configured. I used vim for that. Now I had to run memtier-benchmark, the command-line tool for Redis perf testing, to load some data in and get a basic sense of expected performance. At this point, I needed to inspect the contents of both DBs to make sure that the contents had replicated as expected across instances. The way to do this was using the redis-cli command-line tool. Only one problem, I had run the servers from the command line. They were both sitting there throwing out logs to my terminal window. I could have just opened 2 more tmux sessions and had 4 ssh sessions going, 2 running a server, and 2 available for interacting with the command line. I had two problems with this; one, I have a peeve about too many windows open, it bugs me, and having 4 panes in a tmux session is a bit extreme for me to care about for more than a few minutes, and two, I knew linux had job control commands but for the life of me, the only thing I could remember was that I should have started the server using `./redis-server &` to start it as a background task. After a minute of googling to refresh my memory, I used the bash command `Ctrl+z` to suspend the process and get the command line back, then used the bash shell command `bg` to start the suspended process as a background job. This freed up my terminal to use start redis-cli to query the DB, and did not require me to open a new terminal.
If I had been uncomfortable with the CLI, at best it would have taken me significantly more than the 5-ish hours to complete all the tasks I had set out for myself. At worst, I may not have gotten any of my goals completed at all. I get that the terminal can be intimidating, there’s a lot of stuff to get familiar with, but if you look back at the afternoon I just described, you might see that terms “terminal”, or “command line”, hide a lot. The terminal is just the text interface I’m using to interact with the shell. The shell is just the interface that I’m using to interact with the OS. Tools like ssh, and tmux, are just the interface I’m using to interact with both the shell and operating system. We’re programmers, and to make good software we break things down into parts that can be easily composed. So to learn it break it down. I like to do it by deciding what my current issue is and asking if I can’t do it, is that because I need another tool or don’t know this one well enough? Is it my knowledge (or lack thereof) that makes this painful, or am I using the wrong tool for the job? What tool do I want to learn, what’s most beneficial? The shell? A terminal editor? I obviously can’t answer those questions for you. What I can say is that I think you should ask those questions of yourself and think about a good answer.
