Advanced Rails Caching.. on the Edge

When trying to scale a portal to millions of users with complex personalization, we had to rethink the application design. We needed partial page cacheability, and we wanted that close to the Edge... and recently the idea has been catching on. First with Components Are the New Black, and the idea of Nginx, SSI, and Memcache, what I'd like to briefly describe below is a partial page caching strategy that has worked well for the past year.

First, we use a technology called ESI. ESI stands for Edge Side Includes, which is a simple markup language for describing dynamic assembly of applications. At the core, its similar to SSI, but its a more versatile spec that has been accepted/implemented by a half dozen cache servers, both open source (Squid, Mongrel-ESI) and commercial (Oracle Web Cache, Akamai, among others). It also includes an invalidation protocol, exception handling on the edge, and a few other features.

The ESI W3C specification has been out for 6+ years, so these ideas are not new, but ESI is a diamond in the rough. No one seems to be using it.

A simple example of ESI in your rails application is including a common header. If the header is static, using a shared layout, or rendering a shared partial across applications, could be sufficient. But if you use the common idiom of a sign-in bar, like "Welcome Bob - Signout", on a page you want to fully cache, or a common header you want to share across multiple applications, then you may consider another approach.

If you leverage ESI, you can put the below in your application layout.

<div id="header">
  <esi:include src="/header" max-age="300"/>
</div>

The <esi:include> tag will inject the response from /header into the page and cache that fragment for 300 seconds. You can also add a variety of options, request header parameters (which can be used to personalize the request) along with a dozen other optional parameters, some of which I will outline below. This is simple SSI (Server Side Includes) .

But if we take it a step further, lets look at how we can apply it to a site like Twitter.




If you look at the image above, the majority of the page can be fully cached. I've outlined two green boxes, which appear to be dynamic content, which can have different TTLs. Using ESI, your markup could be:

<div id="latest">
  <esi:include src="/latest" max-age="5"/>
</div>
...
<esi:include src="/featured" max-age="3600"/>

The ESI enabled cache server would parse this markup, make 2 separate HTTP requests (/latest, and /featured) and cache those with their corresponding TTLs. You can furthermore cache the wrapping template with a Surrogate-Control header, which tells the cache server to keep a cached copy of the template. A request to this page 4 seconds later would make 0 requests to your rails infrastructure. 8 seconds later would only hit the /latest, returning the rest of the page from cache. You can also envision a application pool of servers just to handle /latest, but I'll get into sharding applications via ESI in a later article.


Exception Handling

If you take this a step further, you can also define timeouts and exception behavior. As defined below, If the /latest request takes more than 1s, The cache server will give up, and retrieve the static snippet defined in the except block from your static web host.

<esi:try>
  <esi:attempt>
    <esi:include src="/latest" max-age="5" timeout="1"/>
  </esi:attempt>
  <esi:except>
    <esi:include src="http://static.foo.com/latest" max-age="5" timeout="1"/>
  </esi:except>
</esi:try>

We call these "Sorry" modules, because they often say "Sorry, this feature is having trouble", but the rest of the page may surface meaningful content. Even better, write out more meaningful content to disk once a day and serve that from Apache.


Invalidation

The other benefit of ESI is Invalidation support. There are various mechanisms to invalidate content, but my favorite is Inline Invalidation. Consider the common rails idiom of updating data. You post to a controller which redirects to a view of that data. Since the HTTP redirect (301) bubbles all the way back to the browser, any content in the body of the redirect can be parsed by the ESI server. Therefore you can put the invalidation xml is the redirect response body and not conditionally dirty your view logic.

<esi:invalidate>
  <?xml version="1.0"?>
  <!DOCTYPE INVALIDATION SYSTEM "internal:///WCSinvalidation.dtd">
  <INVALIDATION VERSION="WCS-1.1">
    <OBJECT>
      <BASICSELECTOR URI="/foo/bar/baz"/>
      <ACTION REMOVALTTL="0"/>
    </OBJECT>
  </INVALIDATION>
</esi:invalidate>

The above is a simple single URL example, but the specification supports regex, which would work well for restful resources (e.g. /people/#{person.id}/*), among other things.


FragmentFu

FragmentFu is a plugin that enables ESI support in rails applications. At this point it is an extraction from our internal plugin for modules, caching, and ESI. Its in active extraction/development, but I wanted to involve the community in gathering ideas, suggestions and comments.


Some sample snippets:

<% render :esi => widget_url(1) %>

<%= render :esi => latest_url, :ttl => 5.seconds, :timeout => 1.second %>

<%= render :esi => featured_url, :ttl => 10.hours, :except => '/some/static/path' %>

We've also toyed around with the idea of fragment respond_to's. Since ESI supports adding request header parameters, we can mimic Prototype's
X-Requested-With behavior to implement:

def latest
  ...
  respond_to |wants| do
    wants.html { do something }
    wants.fragment { do something }
  end
end

How to get started

Mongrel-ESI is a great starting place. It was built in-house and released a few months ago on Google Code. It supports a large subset of ESI, ESI Invalidation, and its a great tool for developing applications that utilize ESI. You can grab Mongrel-ESI, download the Oracle Web Cache Standalone or even Squid 3.0... and when you're site gets really big, its time to give Akamai a call. :)