Many people say SELinux is like big scary monster that only make Linux hard and break everything. I also think like that before because every time I install something, it stop working and I get very angry. But my teacher say the best way to learn SELinux is you must break it on purpose first, see how it fail, and then you fix it. This is very true. Today we will install Nginx web server and we will change the port to non-standard port 8888, and also we change the website folder to custom folder in /srv. This will make SELinux very angry and it will block Nginx. I will show you step by step how to trigger this block and how we can fix it in three different ways so you can understand SELinux much better.
First we need to install the Nginx server on our machine. I am using Fedora but you can use Red Hat or CentOS because they use SELinux by default. If you use Ubuntu, this tutorial not work because Ubuntu use AppArmor which is different. We write this command to install Nginx:
sudo dnf5 install -y nginx
This command will download Nginx and install it on the computer. Do not start the service yet because we need to make our new website folder first. If your Nginx is already running for some reason, please stop it with sudo systemctl stop nginx so we can start clean.
Now we must make our new custom webroot folder. Usually, Nginx look for website files inside /usr/share/nginx/html or /var/www/html. SELinux already know these folders are for web server, so it allow Nginx to read them. But we want to test SELinux, so we make new folder in different place where SELinux does not expect web files. We will make it in /srv/web/myapp using this command:
sudo mkdir -p /srv/web/myapp
The -p flag is very important because it make the parent folders if they do not exist yet. Now we need to put some HTML content inside this new folder so we have something to see when it works. We can use this command to write simple index file:
echo "<h1>Hello SELinux</h1>" | sudo tee /srv/web/myapp/index.html
This command write the text “Hello SELinux” inside a file called index.html inside our new directory.
Next step is we must edit the Nginx configuration file to use this new port and new folder. The configuration file is located at /etc/nginx/nginx.conf. I like to use vi editor but you can use nano if you think vi is too hard for you. Run this command:
sudo vi /etc/nginx/nginx.conf
Now you must find the block that say server { inside the file. Inside this server block, you will see lines about listen and root. We need to change these lines. Find the lines that look like this:
listen 80;listen [::]:80;
And you change them to use port 8888 like this:
listen 8888;listen [::]:8888;
Also you must find the line for the root which is the folder where Nginx look for files. It usually look like this:
root /usr/share/nginx/html;
Change that line to our new folder:
root /srv/web/myapp;
Now save the file and exit. If you use vi, you press Esc key, then type :wq and press Enter. If you are worried you made typo, you can check Nginx configuration with sudo nginx -t. If it say syntax is okay, then you are ready.
Now we do the fun part where we try to start Nginx and watch it fail. Run this command to start Nginx:
sudo systemctl start nginx
You will see that the command takes some time and then it fail with error. It say “Job for nginx.service failed”. To see what happened, we can check the system log with this command:
sudo journalctl -xeu nginx
If you look at the logs, you will see error message like “bind() to 0.0.0.0:8888 failed (13: Permission denied)”. This mean Nginx cannot use port 8888 because SELinux is blocking it. SELinux has security rules and it thinks only specific services can use port 8888, and Nginx is not one of them.
SELinux writes all these blocks in audit log. The file is at /var/log/audit/audit.log. It is very hard to read because it has many numbers and technical codes. But we can use very nice tool called sealert to make it easy. If you run this command:
sudo sealert -a /var/log/audit/audit.log
This tool will read the log and tell you exactly why SELinux blocked Nginx and it even gives you instructions on how to fix it. This is very helpful. Now I will show you the three ways to fix this, starting with how to fix the port first, and then how to fix the files.
The first way to fix the port problem is using semanage. SELinux labels everything, including port numbers. The label for web server ports is http_port_t. Port 80 and port 443 have this label by default, but port 8888 does not. We need to add port 8888 to the http_port_t label so SELinux knows Nginx is allowed to use it. Run this command:
sudo semanage port -a -t http_port_t -p tcp 8888
Let me explain this command. The -a means we want to add a new rule. The -t http_port_t means the type of label we want to use. The -p tcp means the protocol is TCP, and 8888 is the port number. This command can take some seconds to finish because SELinux is updating its policy database in background. Please wait.
After the command finish, we can try to start Nginx again:
sudo systemctl start nginx
This time, Nginx will start successfully! You can check if it is running with sudo systemctl status nginx. But wait, we are not finished. If you open your web browser or use curl to see the website:
curl http://localhost:8888
You will get “403 Forbidden” error. Nginx is running, but it cannot read our index.html file in /srv/web/myapp. This is our second SELinux block. Nginx process has a label called httpd_t, and it is only allowed to read files that have web content labels like httpd_sys_content_t.
If we check the label of our new folder with this command:
ls -Z /srv/web/myapp
You will see something like unconfined_u:object_r:srv_t:s0 index.html. The important part here is srv_t. This is general label for files in /srv folder. SELinux does not allow Nginx to read general files because of security. If a hacker takes control of Nginx, we do not want them reading other files on the system.
Now we must fix this file label. We have two choices. We can use chcon command which is temporary fix, or we can use semanage fcontext which is permanent. If we use chcon like this:
sudo chcon -t httpd_sys_content_t /srv/web/myapp/index.html
It will work immediately and you can see the website. But if the system relabels files later or if you reboot, the label will go back to srv_t and Nginx will break again. So we must use the correct permanent way.
To make the label change permanent, we use semanage fcontext to write a new rule in SELinux policy, and then we use restorecon to apply it. Run this command first:
sudo semanage fcontext -a -t httpd_sys_content_t "/srv/web(/.*)?"
This command tells SELinux that any file or folder inside /srv/web must always have the label httpd_sys_content_t. The regex (/.*)? means everything inside this folder recursively. After you run this, the files still have the old label because SELinux only wrote down the new rule but did not apply it yet. To apply the rule, run this command:
sudo restorecon -R -v /srv/web
The -R flag means recursive so it does all files and subfolders, and -v means verbose so it shows you what it is changing. You will see output saying that the label for /srv/web/myapp and /srv/web/myapp/index.html was changed from srv_t to httpd_sys_content_t.
Now if you try to access the website again using curl:
curl http://localhost:8888
It will work and you will see <h1>Hello SELinux</h1> on your terminal!
There is also a third way to fix SELinux problems when you have very complicated application and you do not know what labels to use. This is using audit2allow tool. This tool reads the error logs and automatically creates a custom policy module to allow whatever was blocked.
To use this way, you can search for the Nginx errors in audit log and pipe them to audit2allow like this:
sudo grep nginx /var/log/audit/audit.log | audit2allow -M mynginxpolicy
This command creates a file called mynginxpolicy.pp which is a compiled policy module. Then you can load this module into the kernel using this command:
sudo semodule -i mynginxpolicy.pp
Now SELinux will allow Nginx to do those specific things that failed. But please remember, this lazy way is sometimes bad for security because it can allow Nginx to do things it should not do. It is always better to use semanage for ports and files if you can because it keeps your system much safer.
In conclusion, we learned how to troubleshoot SELinux by breaking Nginx on purpose. We saw how changing the port to 8888 and moving the files to /srv triggered two different SELinux denials. We fixed the port restriction using the semanage port command, and we fixed the file access restriction permanently using semanage fcontext and restorecon commands. This is much better than turning off SELinux completely because now our server remains secure while our application runs without any issues.