Advanced Rails Caching.. on the Edge

When trying to scale a portal to millions of users with complex personalization, we had to rethink the application design. We needed partial page cacheability, and we wanted that close to the Edge... and recently the idea has been catching on. First with Components Are the New Black, and the idea of Nginx, SSI, and Memcache, what I'd like to briefly describe below is a partial page caching strategy that has worked well for the past year.

First, we use a technology called ESI. ESI stands for Edge Side Includes, which is a simple markup language for describing dynamic assembly of applications. At the core, its similar to SSI, but its a more versatile spec that has been accepted/implemented by a half dozen cache servers, both open source (Squid, Mongrel-ESI) and commercial (Oracle Web Cache, Akamai, among others). It also includes an invalidation protocol, exception handling on the edge, and a few other features.

The ESI W3C specification has been out for 6+ years, so these ideas are not new, but ESI is a diamond in the rough. No one seems to be using it.

A simple example of ESI in your rails application is including a common header. If the header is static, using a shared layout, or rendering a shared partial across applications, could be sufficient. But if you use the common idiom of a sign-in bar, like "Welcome Bob - Signout", on a page you want to fully cache, or a common header you want to share across multiple applications, then you may consider another approach.

If you leverage ESI, you can put the below in your application layout.

<div id="header">
  <esi:include src="/header" max-age="300"/>
</div>

The <esi:include> tag will inject the response from /header into the page and cache that fragment for 300 seconds. You can also add a variety of options, request header parameters (which can be used to personalize the request) along with a dozen other optional parameters, some of which I will outline below. This is simple SSI (Server Side Includes) .

But if we take it a step further, lets look at how we can apply it to a site like Twitter.




If you look at the image above, the majority of the page can be fully cached. I've outlined two green boxes, which appear to be dynamic content, which can have different TTLs. Using ESI, your markup could be:

<div id="latest">
  <esi:include src="/latest" max-age="5"/>
</div>
...
<esi:include src="/featured" max-age="3600"/>

The ESI enabled cache server would parse this markup, make 2 separate HTTP requests (/latest, and /featured) and cache those with their corresponding TTLs. You can furthermore cache the wrapping template with a Surrogate-Control header, which tells the cache server to keep a cached copy of the template. A request to this page 4 seconds later would make 0 requests to your rails infrastructure. 8 seconds later would only hit the /latest, returning the rest of the page from cache. You can also envision a application pool of servers just to handle /latest, but I'll get into sharding applications via ESI in a later article.


Exception Handling

If you take this a step further, you can also define timeouts and exception behavior. As defined below, If the /latest request takes more than 1s, The cache server will give up, and retrieve the static snippet defined in the except block from your static web host.

<esi:try>
  <esi:attempt>
    <esi:include src="/latest" max-age="5" timeout="1"/>
  </esi:attempt>
  <esi:except>
    <esi:include src="http://static.foo.com/latest" max-age="5" timeout="1"/>
  </esi:except>
</esi:try>

We call these "Sorry" modules, because they often say "Sorry, this feature is having trouble", but the rest of the page may surface meaningful content. Even better, write out more meaningful content to disk once a day and serve that from Apache.


Invalidation

The other benefit of ESI is Invalidation support. There are various mechanisms to invalidate content, but my favorite is Inline Invalidation. Consider the common rails idiom of updating data. You post to a controller which redirects to a view of that data. Since the HTTP redirect (301) bubbles all the way back to the browser, any content in the body of the redirect can be parsed by the ESI server. Therefore you can put the invalidation xml is the redirect response body and not conditionally dirty your view logic.

<esi:invalidate>
  <?xml version="1.0"?>
  <!DOCTYPE INVALIDATION SYSTEM "internal:///WCSinvalidation.dtd">
  <INVALIDATION VERSION="WCS-1.1">
    <OBJECT>
      <BASICSELECTOR URI="/foo/bar/baz"/>
      <ACTION REMOVALTTL="0"/>
    </OBJECT>
  </INVALIDATION>
</esi:invalidate>

The above is a simple single URL example, but the specification supports regex, which would work well for restful resources (e.g. /people/#{person.id}/*), among other things.


FragmentFu

FragmentFu is a plugin that enables ESI support in rails applications. At this point it is an extraction from our internal plugin for modules, caching, and ESI. Its in active extraction/development, but I wanted to involve the community in gathering ideas, suggestions and comments.


Some sample snippets:

<% render :esi => widget_url(1) %>

<%= render :esi => latest_url, :ttl => 5.seconds, :timeout => 1.second %>

<%= render :esi => featured_url, :ttl => 10.hours, :except => '/some/static/path' %>

We've also toyed around with the idea of fragment respond_to's. Since ESI supports adding request header parameters, we can mimic Prototype's
X-Requested-With behavior to implement:

def latest
  ...
  respond_to |wants| do
    wants.html { do something }
    wants.fragment { do something }
  end
end

How to get started

Mongrel-ESI is a great starting place. It was built in-house and released a few months ago on Google Code. It supports a large subset of ESI, ESI Invalidation, and its a great tool for developing applications that utilize ESI. You can grab Mongrel-ESI, download the Oracle Web Cache Standalone or even Squid 3.0... and when you're site gets really big, its time to give Akamai a call. :)

Code Digest #2

When you program for a living, you write lots of code. There is often some code that you are fond of. We started the Code Digest series to present such code written by the RHG developers. We encourage other teams and individual developers to share similar snippets in their blogs so we all can learn from each other and become better rails developers.

Mai Nguyen

Simple AJAX error messaging

When simple javascript validation is not enough and your product managers insist on ajax for error messaging, guess what. You have to implement ajax error messaging. One such case is 'username availability'. This simple example displays an error message on blur of a the username field. It doesn't hit the server unless the value of the field is well-formed (at least that will save you *some* network traffic ...)

Controller would look something like:

class FooController < ::ApplicationController
  
  def is_username_taken
    
    unless Person.find_by_username(params[:username])
      render :nothing => true
      return
    end

    message_html_id = params[:message_html_id]
    render :update do |page|
      page.replace_html message_html_id ,"Username is not available, please choose another."
      # if you want error styling associated with the failed state
      page << "document.getElementById('" + message_html_id + "').className = 'failed'"
    end

  end
  
end

View would look something like:

<dl>
  <dt><label>Username: </label></dt>
  <dd id="username_input">
    <%= text_field :person, :username, :class => "input", :type => 'text', :id => 'person_username_input' %>
  </dd>
  <dd id="username_messaging"><%= @person.errors.on(:username)%></dd>
</dl>

Your javascript would look something like:

var UserNameUnique = Class.create();
UserNameUnique.prototype = {
    initialize: function( field_id, message_id) {
        this.message_id = message_id;
        this.field = document.getElementById( field_id );
        if (typeof this.field == 'undefined') {return;}
        // Observe blur on field    
        Event.observe(this.field, 'blur', this.checkName.bindAsEventListener(this));
    },
    checkName: function() {
    if( typeof this.field != 'undefined' )
    {
            var name = this.field.value;
            // don't hit server unless username is well-formed
            var re = /^[A-Za-z])[a-zA-Z0-9]{2,25}$/;
            if( name.match( re ))
                new Ajax.Request('/foo/is_username_taken', {asynchronous:true, parameters:'username='+name+'&message_html_id='+this.message_id});
            }
    }
};

new UserNameUnique('person_username_input', 'username_messaging');

You could also use Rails helper observe_field instead of the writing your own javascript, but there is more flexibility in writing your own javascript (such as the need to check well-formed values before hitting the server).

Mark Brooks

Creating the options list for select tags can be annoying, especially if there is a requirement that the current "option" be displayed as the default option. With javascript, it isn't such a big deal, but one must take care of the degraded case as well.

In any event, the base object is an array of two-item hashes representing both the name of a video channel and the number of videos in that channel. By way of example:

@channelinfo = [
  {"name"=>"Fitness","count"=>4},
  {"name"=>"Diabetes", "count"=>1},
  {"name"=>"Pregnancy", "count"=>11}
]

The option values are the name fields of each hash. The key is, we want the current option value to be the first one in the option list, and the rest to be in alpha order by name.

So let's create an option builder from the bottom up.

First, we only need the channel names for the select tag. This gives us what we need:

options = @channelinfo.collect { |channel| channel['name'] }

That gives us a list of channel names. Using the list above, it would be ["Fitness", "Diabetes", "Pregnancy"].

However, we need to make sure that the current channel is first in the list. Let's say that current channel is Diabetes. Since we already have that value in @current_channel, we can exclude it from our options list:

options = @channelinfo.collect do |channel|
  channel['name']
end.reject do |channelname|
  channelname == @current_channel
end

Now the resulting list will look like ["Fitness", "Pregnancy"]. However, we still need the current channel to be at the front of the lift, so we add it back as the first element:

options = @channelinfo.collect do |channel|
  channel['name']
end.reject do |channelname|
  channelname == @current_channel
end.unshift(@current_channel)

Now our options list looks like ["Diabetes", "Fitness", "Pregnancy"].

Two points. First, we want to make sure that, while the current channel is at the head of the list, the rest of the list items are in alpha order, so we add a sort directive in the appropriate place. Also, it is probably a good idea to exclude any nils that might pop up in the collection on the original data object, since it comes from a service and it is possible, however unlikely, that a hash might get spit out without a 'name' property. The more rigorous code looks like this now:

options = @channelinfo.collect do |channel|
  channel['name']
end.compact.reject do | channelname |
  channelname == @current_channel
end.sort.unshift(@current_channel)

Now we can generate our options list using:

options.collect do |channelname|
  "<option>" + channelname + "</option>"
end.join(",")

although if you want to, you can simply combine the whole thing as follows:

@channelinfo.collect do |channel|
  channel['name']
end.compact.reject do |channelname|
  channelname == @current_channel
end.sort.unshift(@current_channel).collect do |channelname|
  "<option>" + channelname + "</option>"
end.join(",")

to get the same result, and eliminate the unnecessary options binding.

Todd Fisher

Here is an extension trying to execute a block multiple times before giving up. Handy for network operations and such.

module Kernel
  def could_fail(retries = 3, &block)
    tries = 0
    begin
      yield
    rescue Exception
      tries += 1
      if tries < retries
        retry
      else
        raise
      end
    end
  end
end

Call it as result = could_fail { some_operation_that_might_fail_first_time }

Todd Fisher

When you are writing a script that needs to auto install gems, you are likely to run into a problem that it stops because there are multiple platform versions available (jruby, win32, etc) and the gem command expects you to pick one that matches your platform. This patch forces to use a specific platform so no user interaction is needed. Original idea from Warren updated to support rubygems 0.9.4.

module GemTasks
  def setup
    
    return if $gems_initialized
    $gems_initialized = true
    Gem.manage_gems

    # see => http://svn.bountysource.com/fishplate/scripts/debian_install.pl
    Gem::RemoteInstaller.class_eval do
      
      alias_method :find_gem_to_install_without_ruby_only_platform, :find_gem_to_install
      
      def find_gem_to_install( gem_name, version_requirement, caches = nil )
        if caches # old version of rubygems used to pass a caches object
          caches.each {|k,v| caches[k].each { |name,spect| caches[k].remove_spec(name) unless spec.platform == 'ruby' } }
          find_gem_to_install_without_ruby_only_platform( gem_name, version_requirement, caches )
        else
          Gem::StreamUI.class_eval do
            
            alias_method :choose_from_list_without_choosing_ruby_only, :choose_from_list
            def choose_from_list( question, list )
              result = nil
              result_index = -1
              list.each_with_index do  |item,index|
                if item.match(/\(ruby\)/)
                  result_index = index
                  result = item
                  break
                end
              end
              return [result, result_index]
            end
            
          end
          
          find_gem_to_install_without_ruby_only_platform( gem_name, version_requirement )
        
        end
      end
    
    end
  
  end
end

Val Aleksenko

Since class-level instance variable are not inherited by subclasses, you need to go some extra steps when writing a plugin using them. Depending on the amount of such variables, I have been either defining a method instead of class-level instance variables or forwarding them to subclasses.

Example #1. acts_as_readonlyable needs to provide a single class level instance variable. Defining a method instead.

def acts_as_readonlyable(*readonly_dbs)
  define_readonly_model_method(readonly_models)
end

def define_readonly_model_method(readonly_models)
  (class << self; self; end).class_eval do
    define_method(:readonly_model) { readonly_models[rand(readonly_models.size)] }
  end
end

Example #2. acts_as_secure uses a bunch of variables. Forwarding them to subclasses.

def acts_as_secure(options = {})
  @secure_except = unsecure_columns(options.delete(:except))
  @secure_storage_type = options.delete(:storage_type) || :binary
  @secure_class_crypto_provider = options.delete(:crypto_provider)
  @secure_crypto_providers = {}
  extend(ActsAsSecureClassMethods)
end
    
module ActsAsSecureClassMethods
  def inherited(sub)
    [:secure_except, :secure_storage_type, :secure_class_crypto_provider, :secure_crypto_providers].each do |p|
      sub.instance_variable_set("@#{ p }", instance_variable_get("@#{ p }"))
    end
    super
  end
end