Few days ago I was playing with a Django application, trying to containerize it. Everything was going smoothly until I tried to run collectstatic in Dockerfile. Attempting to do so threw the following error:
This took me a little bit to figure out, mostly because I missed a command in Dockerfile. I failed to reflect on the basics of Linux filesystem and how Docker works which made the bug hunt longer than it should have been. However, spending some time on experimentation made the mistake obvious and drove the lesson home.
Project Structure
For the purposes of not missing the forest for the trees, this is the simplified project structure:
Dockerfile
This is the simplified Dockerfile:
The Issue and the Fix
An observant reader will note the issue immediately. However, for mere mortals like me, this is the relevant bit of the Dockerfile:
What is happening here? Well, first of all, we create a static/ directory, which matches the static directory specified in the settings.py file. This is where we intend the static assets to live. However, note that we have this command before specifying that we wish to switch to app user (USER app). Thus, the command runs as root. Therefore, it is owned by root, and the app user has no permissions to write to it. The simple fix is to edit the line to read as follows:
Now, static/ folder belongs to app, and app can create additional folders and files inside of it.
Further Exploration
One might wonder if we perhaps could simply not create the folder and let collectstatic do it - after all, we are running it as app, so it would make sense that whatever we run after switching to app user should belong to app. While this may indeed be true (provided we have a Dockerfile as defined above) it would still not solve the issue. To understand why we can exec inside the container and view the permissions of the file system:
Note the following line:
This means that the current directory is owned by a root user. Hence, app user has no power in there, including trying to create directories (which collectstatic would attempt to do). To verify that this is the case, let's break down drwxr-xr-x 1 root root 4096 Aug 13 09:04 .:
d- directoryrwx- permissions the owner has over the directory (read, write, execute)r-x- permissions the group has over the directory (read, execute)r-x- permissions others have over the directory (read, execute)1- number of hard links to the directoryroot- user that owns the directoryroot- the group that the directory belongs to4096- size in bytes of the directoryAug 13 09:04- date of last modification
As we can see, only the owner of the directory (root) has write permissions necessary to create files and folders - all others can either read or execute them, but not write new ones. We can double check that:
As expected, app doesn't have permissions to create the file. This is good for security purposes - we don't want rogue processes creating and/or modifying random files - but it also means that we need to be deliberate about what the user can and can't do, which folders it should own, and which folders/files we should create manually.
Alternative Solution
After short exploration of the permissions, we can naturally ask ourselves if we could make the app own /app folder. The answer is yes:
This will create a directory for the app and change its owner to app. Then, running ls -la we get the following results:
Now the ~/app directory is owned by app, and it has all the permissions it needs. And indeed, if we run collectstatic we'll be able to create the necessary folders and files:
Conclusion
Despite Docker abstracting away a lot of complexity of creating the images, we still need to understand the building blocks on which it operates. And even if we do understand it, we need to be careful to not overlook small but important details in our Dockerfiles. On the flip side, the best way to learn these things is to fail and spend time figuring out why.