Apache websockets proxying

I needed a way to proxy traffic from external mobile users to an internal (i.e. on-premises) RocketChat server. These are not normal https requests, they’re websockets requests that often remain open for long periods of time. This requirement could just as easily be adapted to AWS-hosted apps, or even just making an internal websockets app appear to be part of your existing website by only proxying certain URI’s.

I’ll throw an additional caveat into the mix in that I wanted the users to use certificate authentication, including CRL checking. AND, load a new CRL daily in case it has changed.

This is what I came up with:

<IfModule mod_ssl.c>
        <VirtualHost *:443 >
          ServerName myrocketchat.domain
          UseCanonicalName On
          Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;"
          Header set Connection "upgrade"
          DocumentRoot "/var/www/html"
          CustomLog "/var/log/httpd-rocketchat/access_log" combined
          ErrorLog "/var/log/httpd-rocketchat/error_log"

          SSLEngine on

          # Load myrocketchat.domain production SSL cert, DO NOT use SSLCA* directives here
          SSLCertificateFile /etc/mysslCA/myrocketchat.domain.2023.crtkey
          SSLCertificateChainFile /etc/mysslCA/myrocketchat.domain.2023.crtkey

          # Load the certificate authority file that all
          # client certs have been signed by.  Certs signed by any
          # other authority won't be accepted.
          SSLCACertificateFile /etc/mysslCA/Rocket-CA.crt

          # Require client certificate validation for all access
          SSLVerifyClient require

          # Check that end-user certificate has not been revoked
          # via file-based CRL.
          SSLCARevocationCheck leaf
          SSLCARevocationFile /etc/mysslCA/123hash123.r0

          <IfModule mod_proxy.c>
          <Location />
            Order allow,deny
            Allow from all
          </Location>
            RewriteEngine On
            RewriteCond %{HTTP:Upgrade} =websocket [NC]
            RewriteRule /(.*)           wss://192.0.2.1/$1 [P,L]
            RewriteCond %{HTTP:Upgrade} !=websocket [NC]
            RewriteRule /(.*)           https://192.0.2.1/$1 [P,L]
            SSLProxyEngine on
            SSLProxyVerify none
            SSLProxyCheckPeerCN off
            SSLProxyCheckPeerName off
            SSLProxyCheckPeerExpire off
            ProxyPassReverse / https://myrocketchat.domain/
          </IfModule>

        </VirtualHost>
</IfModule>
Code language: PHP (php)

The next issue I came up with is apache won’t want to restart when these persistent websockets connections remain open. This prevents the new CRL file from being loaded. It even prevents you from simply restarting the daemon interactively because the command will hang indefinitely, but all new connections will be down. Finally, if you begin killing the daemon to accomplish the restart, you end up with orphaned semaphores that will ultimately prevent it from restarting, perhaps during a time you aren’t available.

Cron job to solve that; set to frequency of your choosing:

#!/usr/bin/bash
/usr/bin/killall -9 httpd
ipcs -s | tail -n +4 | head -n -1 | tr -s ' ' | cut -d' ' -f2,3 | while read -r id owner; do [[ $owner == "apache" ]] && ipcrm -s "$id"; done
systemctl start httpd
Code language: PHP (php)

Leave a Reply

Your email address will not be published. Required fields are marked *