SELinux used to be one of those things you'd disable immediately upon installing a new RHEL/CentOS box for all the troubles it would cause. But default policies have evolved a lot, making this the wrong thing to do, for a few years now. But people still do it out of habit.
If you ignore SELinux, it won't cause issues besides the ocasional need to run "restorecon" (which one gets into the habit of doing whenever an "access denied" error happens when permissions seem otherwise correct).
But one problem still remains. SELinux is (very) complex and people (myself included) have a very hard time groking its base concepts. This limits adoption greatly, and I'm still to find a decent document that starts from the simple stuff and lets one build a mental concept of how it works before jumping into the more complicated (real-world) use cases.
I worked at Red Hat when SELinux was made mandatory. Every single customer was hit with 'avc denied'. Having to explain that avc was 'Access Vector Cache' and that 'Access Vector Cache' was part of SELinux, and trying to convince somebody to patch the messages to just say 'selinux denied' was nightmarish.
That said: SELinux is one of the only things that can make shared Docker hosting (ie, where the containers are actually isolated from the host and each other) possible.
You're absolutely right. The problem is that it really doesn't matter if it is amazing an works fine out of the box: people won't use it because SELinux is generally perceived as a technology that is too complicated and will break things if you don't disable it.
At this point, very few people will bother to learn the few bits you need to know to troubleshoot and fix any issues you might encounter; and any benefits you get from using it are not worth the effort.
I used Fedora from 12 to 21 (? I think) and always left SELinux enabled, and it just works. The few things that failed (I remember two issues, one with an experimental build of Chromium and another one with OpenVPN and certificates in $HOME), I submitted a bug report and created my own rule to workaround the issue.
If I managed to run a desktop system with SELinux on, it should be possible (and potentially easier) to use it properly on a server.
> people won't use it because SELinux is generally perceived as a technology that is too complicated
It's amazing how bad things have got with the majority of developers and admins that they refuse to learn things that are difficult and instead simply turn it off. It's not like all of them are too young to remember when nothing in their system was easy and they actually had to learn about what they were doing.
I'm looking at disabling selinux on some of the systems I work on. But I don't think it's for a lack of trying to use and understand it. I don't think it's just a matter of being complicated, to me I find the system cryptic, and information about how to do things correctly very difficult to find information for.
While I agree with your sentiment, I think I have tried to give selinux a fair chance, but I've reached the point where it seems to add more overhead than it benefits us. This is combined with, I'm also likely making mistakes with my configuration, that make it too permissive, because with my limited understanding, I'm putting together modules that make my stuff work, but I don't actually understand the implications of some of the decisions I'm making for permissions.
And my entire team seems to be struggling with selinux, and a little bit fed up with it, because we keep running into it blocking things, simple things, against our intuitions of the way our system and selinux should work.
It might be the perfect access control system, but to me UX is horrible. Or maybe I just hate to admit it, but I've utterly failed at building a mental model for how it works, and how I can effectively interact with it to do what I need to do.
So while you might be right, at least in the case of selinux, I'm not sure I agree that it's simply a matter of developers and admins refusing to learn things that are difficult. In the selinux case, I think it goes deeper, where it creates a cognitive load, that someone needs to invest a significant amount of effort to truly learn it.
I agree that the UX is bad, and I think that's because they were wearing heavily SELinux-tinted glasses when they designed it. As an example, even if you correctly expanded `chcon` to "change context" the name of the command itself tells you nothing about it being SELinux related. In contrast, `aa-enforce` and friends hint that they're all AppArmor commands. (This is more useful when trying to remember the name of the command you typed a few months ago.)
This is of course difficult to get right - one would need to be aware of what everything does detail, but still know what it would be like to use it with no prior knowledge.
Just an agreement. I make a point of using some LSM everywhere I can, but in practice selinux is still hard to debug properly. The level of abstraction in module doesn't help here. System-wide integration means that patching single app profile sometimes involves patching the system profile package, sometimes just one module. And that's just on redhat-like system - anything else gives no guarantees about selinux working at all.
Sure, apparmor has fewer options, but it's trivial to manage a profile along with the package itself. The learning and reporting system is much simpler. And there's no need for debugging the "where did I miss the context setting this time" problem in deployment.
I can deal with both and either one is needed. But selinux simply wastes my time way too often.
> it creates a cognitive load, that someone needs to invest a significant amount of effort to truly learn it.
To benefit from SELinux, I don't think you need to "truly learn" it... but you do need to invest some effort developing troubleshooting skills that are specific to SELinux. But the thrust of your point is right. We need to invest some effort to benefit from it.
I think postings like RedHat's are useful because they show us in a concrete way why it might be worth the small effort to develop the troubleshooting skills, or even worth the large effort to really understand SELinux.
There is a line between something being difficult and something causing any part of the system to randomly stop working with no explanation whatsoever except -if you're lucky- a single log line somewhere that will give no result when searched for on google.
Well - devs and admins spend the minimum effort required to reach their primary goal. And I believe most of the time that it is in line with their employer's policy.
I'm too young to remember how it was in the old times (would be glad to hear a story or 2), but since my very first job, the internal policy (the real one - not the one presented to customer or auditors) has always been "scew firewall, selinux, principle of least privilege. Disable everything so it works right now and grant the dev team root access to the prod environments - they need it for an urgent customer issue!".
Humans are designed to obtain their goals while conserving as much energy as possible. As a rule, people will always take the lowest-effort route to get what they want. At a macro scale, this can only be counteracted by designing systems that strongly discourage specific low-effort-but-harmful routes, and even then, there will always be some sector of the population that doesn't grok the downside of taking the "easier" way.
I think one of the big things that has impacted SELinux adoption is that everyone has sort of seen it as a proactive booster rather than a really necessary part of a secure environment. I'm sure a lot of that is because lots of people are used to administering systems that were around before SELinux (and similar) was. For something that's perceived as a bonus point, it interferes far too often.
Most people don't realize SELinux is on until it does something really bad like stopping a database from restarting correctly or otherwise harming what's supposed to be a stable environment. When those are the stakes, SELinux does not have, or at least does not make immediately clear, a sufficient value proposition to incentivize the admin to fix the rules rather than just turning the whole thing off.
For example, to contrast with the OP, when SELinux stops Docker from doing something, the impulse is not going to be "Yay, SELinux stopped Docker from doing something dangerous! Thank you glorious SELinux!" Instead, it will be "Ugh, SELinux again getting in the way of stuff. I just need to disable that. I get enough headaches from Docker as-is and SELinux probably just isn't modern enough to handle my uber-charged stack with all of it's new-fangled features. Disabling!"
I agree with your post, except for one point: system design shouldn't be focused on strongly discouraging harmful low effort routes, but on ensuring that the good routes are less effort than the harmful ones. That is, while it can be achieved by raising the effort of the harmful routes, it can also (and might be better) achieved by lowering the effort of the desired routes. For example, if journalctl showed the relevant AVC messages when showing the log of a failing unit, it would remind the administrators they may need to adjust it. (Except actually mention SELinux in the message, nobody knows what AVC is.)
To be fair, documentation on SELinux sucks. Git is also complex, but there is a lot of good documentation. When I first tried to develop an SELinux policy it took me a month to learn. Too many things are undocumented, or are documented but not clear where.
> that they refuse to learn things that are difficult and instead simply turn it off.
I wish I could do this with our Logstash cluster. It requires so much hand-holding, and troubleshooting can be quite opaque. Last night the logstash indexing service had 'just stopped', and was sending 57000-character-long json loglines into its own log.
And if they do fix a bug, you can't just upgrade one component, you have to upgrade logstash and elasticsearch and kibana... and maybe whichever beats you're using.
"I used Fedora from 12 to 21 (? I think) and always left SELinux enabled, and it just works."
I've had all sorts of weird errors over the years when I set up new CentOS machines.. one thing or the other. And then I remember to turn off SELinux, and it suddenly works. Over and over again.
It's going to take a lot of convincing to get me to leave SELinux on when it's caused so many problems over the years.
Did you consult the audit.log? Did you pump it through audit2allow? The audit2allow tool will even tell you if the issue can be fixed by setting a boolean on the SELinux config. Most of the stuff I have run into recently can be fixed by setting a boolean.
Yeah, SELinux does require learning it, but it adds a lot. I recently helped a friend of mine fix his PHP CMS because it was hacked. The PHP was high jacked and it started attacking other instances in hosting providers network. If only he had not turned off SELinux, it would have prevented outgoing connections from the http server.
The thing that kills it for me is that Fedora ships with broken default configuration.
Install Fedora, reboot and log in. Chances are within minutes you'll start seeing "selinux denied" messages popping up, complaining about services, files and policies you've never heard of. How is anyone but a seasoned RHEL admin supposed to know what to do with that?
Sort of. I tried the latest workstation release recently and the installer was broken.. you need a bug tracker account to file a bug, so you need your browser to work, which is rough when your system doesn't.
That's the issue with LSMs in general. They all work, but historically have been difficult to configure and impossible to maintain. That's changing.
There used to be only one (SELinux), however, there's competition now from other LSMs. Smack, AppArmor, Tomoyo, etc. In part, that's why SELinux is improving.
I've tried them all and settled on Tomoyo. The documentation is outstanding and it is (to me at least) the easiest LSM to reason about and configure.
> I'm still to find a decent document that starts from the simple stuff and lets one build a mental concept of how it works before jumping into the more complicated (real-world) use cases.
That sounds pretty cool, but where do rules come from?
Suppose some author wrote some daemon. Is the packager responsible for writing the rules? It sounds like having packagers understand SELinux rules is a lot of responsibility, and if upstream is cross-platform, they might not care about such specific needs so as to provide it.
Also, what happens if I write some small app? Do I need to write its rules? If it has no rules applied to it, then it's basically game over, because SELinux sounds like it works ONLY if it applies to all processes.
A packager is generally responsible for selinux policies if they aren't suitable for inclusion in the core policy, as a developer if you want to write them please do but certain aspects rely on things like where binaries and application data are stored so you can't always write a policy that won't need tweaking on a specific distribution.
SELinux policies take some time to write the first time or two, but typically running in permissive mode and running your app with a permissionless context will give you everything you need to include in one.
Admins have the hard job when they move default data directories around, takes time to get used to running 'semanage fcontext' in addition to setting file system permissions.
Read that a while ago and it didn't make it click for me.
If I had just stumbled across it without a recommendation and without recognising the bame of the author I'd easly dismissed it as some kind of trolling but thats maybe just me.
What I want is:
1. How to do tasks x, y and z (with explanations on why).
2. Complete documentation of all commands, settings files etc.
Usually 2. is somewhat covered in the official docs and 1. is available in blog posts etc. Last I checked 1 was not covered in any way when it comes to SELinux.
Making a coloring book out of it is nowhere on my list.
To late to edit but let me be perfectly clear that I am totally fine with people making coloring books etc, i just wish there was more how-to, but that might just be me.
It's very hard to come back from such a popular image of "100% broken and must be disabled immediately". Even if SELinux evolves to absolute perfection, the damage to its image is done, and that will take a long time to change, regrettably.
It should not have been shipped so green.
> If you ignore SELinux, it won't cause issues besides the ocasional need to run "restorecon" (which one gets into the habit of doing whenever an "access denied" error happens when permissions seem otherwise correct).
Sound like it's still pretty broken, IMHO. I should never see an "access denied" error on a host I control, unless I misconfigured it.
The truth is, the defaults MUST work on all common scenarios all of the time for these things to be successful. Otherwise, people will only see the downsides (and the upsides are rarely visible, and rarely outweigh the downsides).
> I should never see an "access denied" error on a host I control, unless I misconfigured it.
What do you mean? Even stock UNIX will give a permission denied error if you try to run an executable without `chmod +x`-ing it or `rm -rf /boot` as a regular user.
> SELinux is (very) complex and people (myself included) have a very hard time groking its base concepts.
I wonder what a clean slate OS design would look like. One that satisfied the same requirements without any concerns about backward compatibility with POSIX history.
The MLS model was too difficult to adapt to commercial use. Biba was good for stopping malware from overwriting files. They still preferred something more flexible. SCC then invented type enforcement in another high-assurance system:
Flask architecture was combining that tech with a microkernel. SCC, acquired by McAfee, added type enforcement to a BSD OS for their Sidewinder firewall. The next work by Mitre was proof-of-concept for OSS by adding it to Linux. That and a pile of incremental additions is called SELinux. I'm sure you'll find the LOCK design a lot cleaner as it was originally intended. ;)
Also worth noting are the KeyKOS system (esp with KeySAFE), the capability-security machines, and one language-based mechanism:
How about Magenta [0] by Google, on top of Little Kernel (LK), a neat and modern microkernel design? (It's part of Google's work-in-progress complete OS named Fuchsia, which appeared briefly a while ago on tech news sites.)
Microsoft's Midori project. It never saw the light of day, but there are some very interesting blog posts about it. The Redox project has some leanings in this direction.
Robigalia is interesting but not nearly ready: https://gitlab.com/robigalia (their website has a cert error right now. It seems like I've seen a lot of those these days)
It's a rust userland built upon SEL4. SEL4 is very simplified in order to meet their verification goals so robigalia has to implement some interesting resource sharing primitives on top of it to get things to work. It could be interesting.
> If you ignore SELinux, it won't cause issues besides the ocasional need to run "restorecon" (which one gets into the habit of doing whenever an "access denied" error happens when permissions seem otherwise correct).
You also have to put things in the correct place. For instance, your VPN certificates should be in $HOME/.cert so restorecon knows they should have the home_cert_t label.
When I asked them about this and why they do not even tell the customer about this, they said it is so that they can reset the password when requested through the control panel.
You need to be in the context that will allow the change of password. For example, if this is an agent, you need to be able to execute code in the agent. If this is from a DHCP hook in early boot, you need to execute in that hook.
That's totally not equivalent to the use of "setenforce 0".
Yes, you are right. However, when I calculating risks and protection/price ratio, I see no difference. System with `setenforce 0` is equal to system with the rule for first two significant digit, because root password is most significant and risk of attack through webui is same and high too. Other risks are less significant than that and SELinux does not reduce them significantly.
It sounds like you're saying the security position can be simplified to one dimension. It can't. Unless you're able to provide risk and exposure for all users and somehow make them the same for everyone (they're not).
Then you write that SELinux doesn't reduce the risk. It definitely does that for webapps executing wrong commands, utility services being exploited for local access, many attempts at race conditions via shared directories, etc. For example almost all interesting use cases for imagetragick exploits are severely limited by properly configured selinux. Once in a while there's going to be an issue with a trivial exploit which everybody and their dog will use to scan the whole internet before you have time to patch. This is what LSM is great to protect against.
I'm not saying that risk is not reduced. I'm saying SELinux is not effective, so it's better to spent my time/money on something that can reduce risk significantly.
I saw no 0-day exploits to date which are stopped by SELinux. For example, a trojan can use apache process as malware host, without reading/writing to disk at all. SELinux will not stop that even in theory.
It will not. But looking at what exploits normally do - that would be uncommon and targeted. A lot of common stuff will only drop a stage 2 downloader and try to execute it. Doesn't work - move to the next target.
For most of automated exploitation, selinux is perfectly capable of intervening.
If you ignore SELinux, it won't cause issues besides the ocasional need to run "restorecon" (which one gets into the habit of doing whenever an "access denied" error happens when permissions seem otherwise correct).
But one problem still remains. SELinux is (very) complex and people (myself included) have a very hard time groking its base concepts. This limits adoption greatly, and I'm still to find a decent document that starts from the simple stuff and lets one build a mental concept of how it works before jumping into the more complicated (real-world) use cases.