Let’s Encrypt Renewal

Yeah yeah, SSL certificates expired like what, a month and a half ago? Anyway, I needed to renew it but it attached a bunch of domains to the certificate on this host and I needed to start over. There are a bunch of options but the most straighforward for me was to just delete everything and start over. So, in the shell:

# cp /etc/letsencrypt /etc/letsencrypt.backup -r
# rm -rf /etc/letsencrypt/live
# rm -rf /etc/letsencrypt/archive
# rm -rf /etc/letsencrypt/renewal/YOURCERTIFICATEDOMAIN.conf
# certbot certonly

At this point it’ll ask you for webroot an’ stuff so just answer the questions and nobody gets hurt. These instructions work for Apache but I’m using NginX, so then you have to edit the configuration for the site(s) in /etc/nginx/sites-available/. There will be a couple lines going like:

ssl_certificate /etc/letsencrypt/live/OLDCERTIFICATEDOMAIN/fullchain.pem; # managed by Certbot
ssl_certificate /etc/letsencrypt/archive/OLDCERTIFICATEDOMAIN/fullchain.pem; # managed by Certbot

Just update them to point to the most current certification location, then restart NginX with service nginx restart

nginx + hhvm, continued, part three

It’s starting to sound like a bad Hollywood movie. You think you’re done with it and then BOOM it comes out of nowhere straight to DVD. Back when movies did that.

So after I played around with everything, bloody HHVM kept crashing — I’m starting to think it’s a fucking piece of shit just like facebook, which begat it — and all I could get out of the various things in /var/log were that it kept running out of memory. htop didn’t really say anything, nor ps aux nor netstat

Using systemctl status hhvm.service or journalctl -xn reported only that something was wrong, but didn’t say what. Which is annoying since fucking piece of shit systemd wants everyone to use its own tools for managing the system.

Gods, I dislike systemd. But carrying on…

Checking /var/log/messages manually led me to believe that hhvm was running out of memory and not failing gracefully. So that led me to looking around. Then I got this snippet from this site, which I added to my /etc/hhvm/server.ini:

hhvm.jit = false

Then
# service nginx restart && service hhvm restart

Since I did that I haven’t had to restart hhvm. Sure, I lose performance, but since this is my personal blog and I’ve had issues with php-fpm and hhvm I would much prefer to have reliability.

PS: Yes, I work on root. I’m a real sysadmin, not like y’all wankers who work with sudo.

nginx + hhvm, continued

So after my last post on this, HHVM stopped working again, making nginx give a 502 as usual.

This time, a comment on a forum (who knows which, I looked at a lot of pages) gave me a little snippet that helped me pinpoint the issue:
# netstat -plunt | grep hhvm

This should have told me that HHVM was listening on a TCP port (since that’s what the default for HHVM is). But netstat reported nothing. After checking my configuration, I had forgotten that I had iptables drop all traffic to 127/8 that doesn’t use loopback. Here’s the snippet I use in my iptables configuration file
-A INPUT -d 127.0.0.0/8 -j REJECT

Commented the line out, imported the new ruleset into iptables and iptables didn’t block traffic sent from nginx to HHVM anymore, letting everything work as is supposed to.

Now I’m hoping it doesn’t die again due to some other reason. This was really annoying.

nginx + hhvm

I’m not sure what I did, but I finally got it to work. My previous configuration had nginx watch out for a 502 BAD GATEWAY error to switch from HHVM to php-fpm.

Too bad that every single PHP request invoked a 502 error. But my blog worked and I didn’t check the logs cos I happily thought everything was working.

Until it didn’t.

MariaDB bitched. Then php-fpm started using 100% CPU until it killed MariaDB. nginx didn’t care cos it doesn’t use that much memory to begin with. But the end result of all this was that my blog didn’t work.

Ended up reinstalling everything from scratch. Then it somehow works. I should write up about it later so I don’t forget.

But right now I need to shower so I can be at work in a half hour. Yes, at 0630. And now WordPress is bitching about something going wrong. I’ll probably just end up switching back to good ol’ Apache with the standard PHP module in.