Nginx HTTP/3 proxy server displays content from the wrong virtual host
At the end of May 2025, I published the metalhead.club song via a Faircamp website at music.metalhead.club. However, not all users were able to follow my links to the Faircamp site without problems. In a few cases, users reported at least one of the following errors to me:
- When users accessed the album page: Error 404 - not found
- When users accessed the main page: The watch.metalhead.club globe was displayed
After a few attempts, I was able to reproduce the error sporadically myself. In the access logs of the Nginx proxy, I noticed that all erroneous requests were made with HTTP/3.
Since the proportion of HTTP/3-compatible web browsers is steadily increasing, I had decided a few months earlier to use a very recent version of Nginx with HTTP/3 enabled on the server. However, I had not enabled HTTP/3 for music.metalhead.club. This raises the question: …
Why is HTTP/3 used?
For my Mastodon instance at metalhead.club, HTTP/3 was enabled as the only virtual host on the Nginx server, primarily to reduce connection setup time for users located further away and thus shorten loading times. For this virtual host, the Alt-Svc header was also set to “h3” (i.e., HTTP/3). However, this setting did not apply to other virtual hosts such as music.metalhead.club – so why did various web browsers still send HTTP/3-based requests to the non-HTTP/3 virtual host music.metalhead.club?
To explain this phenomenon, it is important to know that HTTP/3, like its predecessor HTTP/2, has a feature called “connection coalescing.” If …
- IP address
- port, and
- SSL certificate
are the same, “old” QUIC connections are reused (see also RFC 7540 “Connection reuse” – also known as “Connection Coalescing”). So even though a different virtual host on the server may be addressed later, an existing QUIC connection to the server is used if the three parameters match. Whether this “other” virtual host supports HTTP/3 at all is no longer checked. In my case, the following apparently happened:
- A user first visits metalhead.club. The browser recognizes HTTP/3 support via the “Alt-Svc” header
- Further connections to metalhead.club are now established using HTTP/3 – a QUIC connection is set up for this purpose
- The user then navigates to music.metalhead.club. This other virtual host uses the same IP/port/SSL certificate combination (because of the wildcard certificate). The browser recognizes this and uses the existing QUIC connection
As a result, the web browser now also communicates with the web server using HTTP/3 when accessing music.metalhead.club.
… but music.metalhead.club does not speak QUIC!
However, there is no QUIC listener in the Nginx configuration for music.metalhead.club. And this is where the second pitfall comes into play: If a virtual host that does not have a QUIC listener is requested via a QUIC connection, Nginx searches for the “default” QUIC virtual host. The “default” vHost is the one that has been explicitly defined as such – or the one that (as in my case) has been implicitly defined by Nginx through the configuration order. In my case, there was only one vHost that speaks QUIC: the vHost for metalhead.club.
Requests to music.metalhead.club were ultimately processed by the vHost for metalhead.club because there was no other vHost with QUIC enabled.
Since this vHost is only a proxy that points to another Nginx instance (“Nginx app server”), the request is forwarded again. The app server Nginx instance in my setup finally receives the request for music.metalhead.club – but cannot do anything with it. After all, it does not have a vHost that could be responsible for music.metalhead.club (see diagram). So once again, a “default” vHost is selected. And since watch.metalhead.club is the first vHost that appears in the configuration of the second Nginx, the request is forwarded to watch.metalhead.club and answered there: The user sees the globe on the home page… or a 404 error page, because the requested album overview does not exist on watch.metalhead.club, of course.
Red: The incorrect route between the two Nginx instances. Green: The intended route. (1) Due to the missing QUIC listener, the metalhead.club vHost is addressed. (2) This in turn now points to the wrong Nginx app server.
The solution
Mystery solved! But how can the problem be fixed? The core problem is that web browsers assume that QUIC connections can also be reused for other virtual hosts if the IP address, port, and certificate are the same. Therefore, we should create the appropriate conditions for this on the server side. The solution to the problem is therefore to enable QUIC for all virtual hosts that share an IP address/port/certificate combination. In my case, I also enabled QUIC for music.metalhead.club (and watch.metalhead.club) on the proxy Nginx. This solved the problem.