I installed Apache from the Rocky repository. After about a week I have all the path problems resolved BUT when I start it [systemctl start httpd] Apache will start and run for about 5 minutes but then stops with status message:
systemctl status httpd
× httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/httpd.service.d
└─php-fpm.conf
Active: failed (Result: timeout) since Mon 2023-05-22 16:52:04 EDT; 2min 12s ago
Docs: man:httpd.service(8)
Process: 1160 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=0/SUCCESS)
Main PID: 1160 (code=exited, status=0/SUCCESS)
CPU: 73ms
[John@prod02 ~]$ systemctl status httpd
× httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/httpd.service.d
└─php-fpm.conf
Active: failed (Result: timeout) since Mon 2023-05-22 16:52:04 EDT; 2min 40s ago
Docs: man:httpd.service(8)
Process: 1160 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=0/SUCCESS)
Main PID: 1160 (code=exited, status=0/SUCCESS)
CPU: 73ms
Also, why is the default service file set to run Apache in the foreground?
[-DFOREGROUND]
Apache is stopping due to a timeout trying to load a module or configuration. You will need to try to disable modules or configurations to try to isolate where the issue is.
The reason why apache is ran in the foreground is so that systemd can properly manage the service. This way some logs (if configured or applicable) will appear in the systemd journal.
This may help:
apachectl configtest
to find out where the errors in the config might be at least.
Thanks Ian.
That results in “Syntax OK”. The other help folks suggested that one of the
modules is not getting loaded and so far everything seems to be as expected. I
tried httpd -M and today my plan is to see if one of the desired modules is NOT
loaded. There seems to be no reason to suspect that.
One of the problems is that it’s hard to debug things when the daemon keeps
stopping almost immediately so I can’t get a full status.
The log is sort of useless too; all I get in the error log is:
[Tue May 23 11:41:12.197529 2023] [core:notice] [pid 7565:tid 7565] SELinux
policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Tue May 23 11:41:12.208368 2023] [mpm_event:notice] [pid 7565:tid 7565]
AH00489: Apache/2.4.53 (Rocky Linux) OpenSSL/3.0.1 configured – resuming normal
operations
[Tue May 23 11:41:12.208387 2023] [core:notice] [pid 7565:tid 7565] AH00094:
Command line: ‘/usr/sbin/httpd -D FOREGROUND’
[Tue May 23 11:42:42.228697 2023] [mpm_event:notice] [pid 7565:tid 7565]
AH00492: caught SIGWINCH, shutting down gracefully
which looks to me like a normal startup followed by a graceful halt. The signal
name is a bit misleading since SIGWINCH has nothing to do with this situation.
Regards,
John
When continuing to chase this around it occurred to me that I have no idea where
the pid file is written. Is this possibly a pid file SELinux permissions
problem? If so how would I prove it since SELinux doesn’t seem to have a log
file that I can find.
John
Easiest way would be temporarily put the system in permissive mode with:
setenforce 0
selinux policies will then be only logged and not blocked. If you find apache starts behaving, you can then take a look with the selinux tools, so:
audit2allow -a /var/log/audit/audit.log
and see what is reported there. Once the issues are addressed, you can then switch back to enforcing with:
setenforce 1
Is your apache configuration using ssl certificates, if so when there is a mismatch between key and certificate apache will fail to start?
Yes, we do have an SSL certificate but no messages indicating there is a problem
there. Same cert and same CA authority cert as on the old server where it has
been running happily.
Thanks for the idea though.
Regards,
John
May be a bit off-topic here, I don’t really know, but what I did was create the /.autorelabel file and reboot. What I was attempting to resolve is access to the web pages which has the following authorities:
ls -alZ /httpd/iliffe
total 4
drwxr-xr-x. 2 root root unconfined_u:object_r:default_t:s0 27 May 22 21:40 .
drwxr-xr-x. 4 root root system_u:object_r:default_t:s0 32 Apr 30 13:03 …
-rw-r–r–. 1 root root unconfined_u:object_r:default_t:s0 113 May 22 21:40 homepage.html
I created this directory and page so I have something to experiment with but shouldn’t it be associated with httpd after relabelling?
The autorelabel will only work for the default directories, so in this case /var/www/html. Since you created your own, outside of this, you will need to set the contexts yourself on those directories and files. Take a look at the context of /var/www/html:
ls -Z /var/www/html
although the audit2allow command that I gave you before, should have highlighted that already.
Thank you. I just got back to the office today and tried that to see what came
up. Lot’s that is important I think but beyond my understanding.
Here is the full log:
#============= httpd_t ==============
allow httpd_t default_t:file getattr;
#!!! This avc can be allowed using one of the these booleans:
httpd_can_network_connect, httpd_graceful_shutdown,
httpd_can_network_relay, nis_enabled
allow httpd_t http_port_t:tcp_socket name_connect;
allow httpd_t i18n_input_port_t:tcp_socket name_bind;
#!!! This avc can be allowed using the boolean ‘httpd_can_network_connect’
allow httpd_t i18n_input_port_t:tcp_socket name_connect;
#!!! This avc can be allowed using the boolean ‘nis_enabled’
allow httpd_t unreserved_port_t:tcp_socket name_bind;
Since my last note I have found the setsebool and getsebool commands but they
don’t seem to resolve the problems noted in the log.
For example, the log notes problems with http_can_network_connect
getsebool:
getsebool httpd_can_network_connect
httpd_can_network_connect → off
so I tried to turn it on:
setsebool http_can_network_connect on
Could not change active booleans: Invalid boolean
Same thing happens with http_can_network_relay so I am assuming that I am
misreading the man page and also the Red Hat SELinux docs.
What is the exact format of this commend and do I have to shut something down to
make it run?
John
Try:
setsebool -P httpd_can_network_connect 1
the P parameter makes it permanent, otherwise after restart of server you revert to previous setting if you don’t use P.
Incidently yours above was missing the d. You wrote http instead of httpd like mine. Therefore you can also do:
setsebool -P httpd_can_network_connect on
You can find a list of variables with:
getsebool -a | grep http
you’ll notice they are all httpd and not http.
Thanks for the suggestions. I turned off Selinux completely in the configuration file and rebooted. httpd comes up as expected but still crashes after about a minute so there is some other problem not Selinux. Apache’s error log is now reporting path problems on paths that seem correct to me so more investigation required. Thanks for your help.
John
I noticed that while Selinux was turned off in the config file httpd was still reporting it as active so I made another change to the configuration (set selinuxtype to minimal) and tried a reboot but now the system refuses to boot.
Any ideas what I may have broken? … and what I will have to do to get it back?
John
Try launching Apache manually from the command-line and then watch what happens until it crashes and follow messages as well.
If you disable selinux, then it will not put contexts on files, you need to autorelabel, or put selinux in permissive mode. Disabling selinux is not needed, it could have been put in permissive mode which is practically the same as disabled, but it still logs warnings to the log files so that any selinux issues can be seen and addressed before switching back to enforced.
That is why I mentioned using setenforce in my previous post to do that without requiring a reboot or editing files to disable it completely.
Apache ie httpd runs under systemctl and there for is better running in foreground.
What I don’t understand is, how did you make it crash?
There are missing details.
we need the output of:
find /etc/httpd
and also:
httpd -S
please replace any sensitive data with something global.
I have httpd working in production on-top of 9.2 and back for a while now.
One more thing, what I would also do for the new location you are attempting to use:
chown -R apache:apache /httpd/iliffe
check/replace the path accordingly, or do for just /mttpd and everything underneath it. Just in case it’s a permissions issue.
The problem that I have at the moment is that Rocky became unbootable when I changed the type to “permissive” so I can’t do anything at the moment. Since I will probably have to re-install to get out of this mess, and that calls for pulling the server out of the rack because the network is unreachable, and I will be away for much of the next three weeks, I’m going to just step back and re-analyse the whole environment before I do that.
Thanks for all your help; I’ve never gotten into a mess like this before and I think I’ll turn off Selinux permanently on the next install. There are independent ways of finding out if we have been hacked that we use at present.