Olivier Dutilleul

... and Beware of the Age header too !

Blog Post created by Olivier Dutilleul Employee on Nov 7, 2016

Note: This post has been written in continuation of this article on the "Via" header and its potential impact on content compression.


You may want to pay attention to the infamous "Age" Header as well. The purpose of this header is to synchronise cache TTL when you stack several caching elements between the origin and the end users, but it has some nasty side effects when misused.


Initial situation

Let's imaging you have a CMS as origin which generates site pages, then a local Varnish (which cache locally generated pages when serving them), then a CDN (which also caches and returns pages) and finally the end user.

      (End User)   <--->  (CDN)   <-->  (  (local Varnish)  <-->  (CMS)  )


A certain TTL will be configured on the varnish (let's say 1d). So when the page is generated by the CMS, the Varnish will cache this content, and will be allowed to serve it from cache for a maximum of 1 day (after which, he will go back to origin to check if some more up-to-date content is available)

Similarly, a certain TTL will be configured on the CDN (let's say 1d as well), so when the CDN retrieves the content, it will keep it in cache and will also be allowed to serve it from cache for a maximum of 1 day.


Why Age can be useful

If you add up the Varnish TTL and CDN TTL, at maximum you could get a global TTL of almost 2 days (if CDN retrieves the content just before the expiration of the content on Varnish). So if you want to avoid this efect, you can use on your varnish the Age header. The purpose of this header is simply to indicates to downstream elements (CDN) the age of the content, meaning how long passed since the Varnish got the content. In our example, if CDN requests a content that has been cached 2 hours ago in the Varnish, in the response, the Age header value will be : 7200s. Therefore, the CDN will honor this header and the TTL on the CDN will be reduced to: 1d - 2h = 22h.


But what's the trap ?

The problem mainly arises when the TTL configured on the Varnish and the CDN are not alike, especially if the TTL on Varnish is higher than TTL on CDN - let's say 7d on Varnish and 1d on CDN.

For the first day, CDN will deduce the Age header from it's TTL when caching the content, so the TTL for retrieved content will decline from 1d to 0. From the 2nd day onward, the Age header value will be higher then the CDN TTL, meaning the content will be marked as older than the maximum allowed duration configured on the CDN ! From that moment onward, the CDN will simply stop caching content, and all offload benefit will be lost !

This situation will clear itself only when the TTL on Varnish expires, and the Age header resets to 0...


Below, the offload report from a real situation, pretty scary isn't it ?