Forging your own PATH

If you’re anything like me, once in a while you just want to get on with programming stuff, without diving into the minutiae details of the setup. This usually leads to copy-pasting the commands from the various setup instructions without much thought.

Some of these steps are so common that after a while you stop to question why they are necessary and what do they actually do. With so much software coming out these days, it’s only natural to leave some areas unexplored. Sometimes, however, it pays off to stop for a moment and consider what you are doing, and why is it necessary. Deeper understanding of how software works is rarely a pointless endeavor in the trade of software development.

The realization became apparent when I’ve recently caught myself nodding along some installation instructions. All was going fine until it came to a point where it asked to add some additional entries to a PATH in bash_profile on my MacBook. Adding stuff to PATH? I can do that, yessir! In fact, I’ve done it multiple times!

…but what does it actually meant to add things to PATH? I’ve edited the PATH many times over the years, yet for most of them I had no clear idea why it was necessary. What does it enable? What can go wrong? How can it be useful for my own flows, aside from installing software?

Let us explore it together.

What is PATH?

Have you ever considered how does something like man <command> or pwd work on your system? These commands are built in with most of the Linux and/or MacOS distributions, and they just seem to work when you type them on your shell. Obviously, there is some code that implements these commands. Yet, how does the OS know what code to execute? How does it know where the code is located? Could we persuade the OS to run our commands in such a fashion?

It turns out that PATH environment variable is an answer and a solution to these questions. Simply put, the PATH specifies where to find various executable programs. It lists a bunch of directories in which the OS should look for executable programs. Thus, instead of using the full path to the script and running /usr/bin/man ls, you can simply type man ls.

To get a feel of what’s going on, issue the following command:

> echo $PATH

Which should produce something like:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

(the exact result will depend on your operating system, it’s configuration, and installed software - do not be surprised if the result is substantially different on your machine)

Issue the following command:

> which ls

On my system, I get the following result:

/usr/bin/ls

Note that /usr/bin directory is included as one of the entries in the PATH that we got before. ls is the script that we’re running. Thus, when I run ls, the operating system consults PATH to see in which folders it should look ls command for. It will then go through each of the folders, from left to right, until it locates (or fails to locate) the script. Once it finds the script named ls, it stops the search and runs the script. A simple, elegant, and powerful mechanism.

Editing PATH

Now that we have a basic idea of how it works, can we bend the PATH to our will? In fact, this is quite easily, and often necessary, as proved by multiple installation instructions urging us to edit the PATH.

As we have seen, adding new folders to it will make the system look for the commands inside those folders. To put theory into practice, let’s try it out. Issue the following command:

> export PATH="$PATH:/Users/<username>/<folder>"

This will make it so that the PATH is redefined to whatever it was before ($PATH part), with /Users/<username>/<folder> appended to the end of it. Note the colon (:) after the $PATH. It is used to indicate the end of one directory and the beginning of the other.

Editing the PATH in this way will ensure that the system will consult /Users/<username>/<folder> directory when searching for executables. Note, however, that it will only look at it if it has not found the executable in the directories that come before it. If you wanted to make this directory the first place to look for any scripts, you could do so by appending the rest of the PATH to it instead:

> export PATH="/Users/<username>/<folder>:$PATH"

There is one gotcha with what we’ve been doing up until now, however. While what we’ve done works, it will only last until the current terminal session is terminated. Thus, if we were to close the terminal, all our configuration would be lost. To make our changes permanent, we can edit .bash_profile (or .bashrc/.profile, depending on a few factors) file, and defining the PATH variable there. For example, if I wanted to include Python 3.12 executables on a PATH on my MacBook, I’d write something like this:

PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:${PATH}"

Note that with this method you’ll need to reload the file you’ve just edited (or start a new terminal session). This is so because it’s read once, on Bash startup. To reload it, simply run:

> source ~/.bash_profile # or .bashrc, .profile, etc

This will execute the script (yes, the file is a script), which will define the PATH for later use in the terminal session.

On the other hand, if you do not wish to edit the PATH variable you could add your scripts to the directories that are already defined. Since the directories are already on the PATH, the system would pick the scripts right away - no need to source anything.

Example

So far, the discussion was quite abstract. To make it more palpable, let’s create a simple script, add its location to a PATH and see what happens.

First of all, let’s create a file in a folder that is for sure not among the ones in the current path:

> cd /tmp
> mkdir test && cd test
> touch testscript

Now that we have a file, let’s edit it to have the following content:

echo "Hi there, stranger!"

Now, let’s make it executable:

> chmod +x testscript

Before proceeding further, let's double check which directory we’re in. While we’re at it, we can also verify our PATH does not have it:

> pwd
/tmp/test
> echo $PATH
# Should print something like the following, though probably longer:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

Once we are sure it’s not in the PATH, let’s add it. First, we’ll do it in the current Bash session so that if something goes wrong we can simply close it and open another one:

> export PATH=$PATH:/tmp/test/

Let’s double check if it’s actually added to a PATH:

> echo $PATH
# Should print something similar to the following:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/tmp/test

Now let’s see if we can run scripts contained inside our /tmp/test directory. First, we’ll move to some other directory (so that we’re sure we’re not just picking up the script from current one), and then we’ll run our testscript command:

> cd ~/
> testscript
Hi there, stranger!

Great success! Our script is being picked up. Note that the script has no extension. This is intentional - writing command feels more natural than writing command.sh, but you could also have a file extension if you so wished.

Now that we’ve ensured it works, let’s make it permanent. Edit the ~/.bash_profile (or its equivalent), and add the following line:

export PATH=$PATH:/tmp/test

Alternatively, if PATH is already defined in the file, we can add :/tmp/test to the end of it.

Once the PATH is adjusted, we need to re-read the .bash_profile file so that updated PATH would be loaded. We can either run source ~/.bash_profile or simply open a new terminal session. Then, we can again run:

> testscript
Hi there, stranger!

And we’re done! The only thing that’s left is to actually create a useful command to run. Don’t forget to remove the /tmp/test from the PATH, though, once you’re done experimenting! Keeping PATH clean and focused is a good habit to keep.

Dangers

Customizing your system wouldn’t be as fun if you couldn’t shoot yourself in the foot. Editing PATH is no exception. There are multiple amusing scenarios you might find yourself in if you’re not careful when messing with PATH:

Removing the PATH variable. Some commands will stop working. For example, ls, man, python, gcc, and so forth. Try it out in the current terminal session:
```
> unset PATH
> ls
-bash: ls: No such file or directory
```
You can climb out of this hole by opening a new terminal session.
Adding a folder that has a script with the same name as an existing script. For example, you might add a directory that has a python script, and then realize one of the following:
- You can’t run the python shell or execute Python files. This might happen if a new script directory was put before the Python directory in the PATH.
- Your script is not picked up, and an already existing one is executed instead. This could happen if a new script directory was put after the Python directory.
Simplest solution is to rename your script.
Adding current directory (.) to PATH. You’ll never be sure whether the command you’re running is a proper system command, or some random folder that might contain malicious (or just plain wrong and confusing) commands [1].

Closing Words

While often there is a great temptation to simply blindly follow the instructions to be able to get on with your real task, it often pays off to spend some time to consider the reason behind them. Basic as it might be, for a long time I was not aware of what exactly PATH does and how it interacts with the rest of the system. Experimenting with it and reading up on it made me feel like I’ve managed to fit a small piece of a puzzle in its place. It’s a lovely feeling, and I hope this post will help you with putting your puzzle pieces together, too.

Sources

https://cets.seas.upenn.edu/answers/dot-path.html

Forging Your Own PATH