Nimble

nimble • quick and light in motion, Merriam-Webster

Almost four weeks ago Kyriakos Zarifis wrote to the dev mailing list:

"I am experimenting with contention between lower/higher priority HTTP/2 streams, and I think I am noticing that high-priority frames are not given high priority (quickly enough)"

And he had the numbers to prove it. He configured Apache httpd to PUSH a large image when serving a page with another image. The PUSHed images was configured to have low priority.

What he, correctly, expected was that the html and contained image would be served with priority, while the PUSHed one gets much less bandwidth. What he however observed was:

The dark green indicates frame sent for a stream. The light green the time the stream was open. Surprisingly, the server sent only frames of the low priority PUSHed images and the high priority images was completely ignored during this time.

Looking at the server logs on very verbose, he saw that both requests had been received by mod_h2 and even the answer for the second was ready to be served. Only this did just not happen until the previous image was done.

WTF!

In my ambition to make streaming out data faster and faster I had created an addict. mod_h2 was just so happy it had data to send, it no longer cared about anything else. Only when the data ran dry did it ever look up to see if anything else needed doing.

This is very bad, as Kyriakos showed in his second test setup:

Here, a user opens a first page and 2.5 seconds later navigates to another. Unfortunately, it takes very long for that page to be sent, because mod_h2 wants to send all other output first before it cares for other responses. The user needs to wait more than two seconds after her click. Clearly unacceptable.

(Side note: does this affect your setup? Well, if you serve pages with a lot of static files, probably yes. If you have Apache as proxy, probably not.)

Once the problem became clear, finding a solution was easy. From release v1.8.8 and onwards, mod_h2 checks frequently for new data and evaluates priorities, even when a lot of other things are ongoing. The page navigation now serves the next page 5-20 ms after receiving the reuqest:

and in the PUSH case from above, priorites are obeyed:

In other news, I have now a tattoo on my forearm saying:

"Be fast and nimble!".

Many thanks to Kyriakos Zarifis for his work. He also provided the graphics shown here. If you find something or have questions, talk to us at dev at httpd.apache.org.

As usual, you can throw this into your 2.4.25 Apache with v1.8.11 from github. Which also has tons of other improvements.

Münster, 28.01.2017,

Stefan Eissing, greenbytes GmbH

Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. This file is offered as-is, without warranty of any kind. See LICENSE for details.