Facade To The Rescue

Eddie talked in detail about Facade in this blog before. I would like to provide a couple of examples how that great plugin helps with writing runtime-specific implementations.

User Story: Jim's AWS S3 plugin (test/development/production all use different implementations)

For his rails application, Jim wrote a plugin that allows storage on Amazon S3 (not knowing there is already a very good one). Not wanting to hit the service in test and development mode, he is using mocks with Hash storage for tests, and ActiveRecord for development. He put the mocks in corresponding app/mocks directories and everything works as expected. There is something bothering him though. First, he does not like an idea that various implementation of the same functionality are scattered across the application structure. He would prefer to have a hierarchy that indicates an implementation type (i.e. mocks, active_record, service). More importantly, he is going to release the plugin on rubyforge so he needs a clean way to package all implementations with it. He read about Facade so he decided to give it a try.

First, Jim creates an implementations hierarchy in his plugin:

vendor
  plugins
    aws_s3
      lib
        s3.rb
        service
          s3.rb
        active_record
          s3.rb
        mock
          s3.rb 

Then, he adds facade configuration (vendor/plugins/aws_s3/config/facade.yml):

production:
  s3:
    backed_by: service

development:
  s3:
    backed_by: active_record

test:
  s3:
    backed_by: mock

He modifies the top s3.rb to define the interface:

class S3 < Facade::Base
  backend_class_method :create, :find
end

No changes required for implementation classes, so for example his test mock stays as it was:

class Mock::S3

  @@storage = {}

  def self.create(key, data)
    @@storage[key] = data
  end

  def self.find(key)
    @@storage[key]
  end

end

Jim extracts a plugin, packages it, and publishes it to rubyforge for public consumption.


User Story: Betty's Crypto Library (packaging gem with its own mock implementations but allowing an override in applications)

Betty works for a large healthcare provider with many internal projects embracing Rails for web-based implementations (now you know this is a fictional story ;-) ). They lack a strong crypto library they can reuse across applications to keep them compliant with HIPPA regulations. Betty has experience with RubyInline C so she volunteers to write one. She releases it as a gem to an internal gem server together with a Facade based mock implementation using ROT13 instead of strong encryption, so rails developers have an easy way to recover encrypted data in unit tests. The gem contains config directory with facade.yml:

production:
  healthcare/scrypto:
    backed_by: healthcare/live

development:
  healthcare/scrypto:
    backed_by: healthcare/live

test:
  healthcare/scrypto:
    backed_by: healthcare/mock

Joe is a developer of a small internal application that utilizes the crypto library gem. He found that, since requirements for the library was to withstand to a brute-force attack by an average NSA computer for at least 48 seconds, the library is very CPU demanding and it makes development on his old iBook unbearably slow. Joe decides to override the development mode facade configuration for the gem so it falls back to the mock implementation for both development and testing. He adds facade.yml to his application's config directory with a single entry:

development:
  healthcare/scrypto:
    backed_by: healthcare/mock

Since Facade uses ConfigurationLoader for configuration files loading, application config is being merged with the one enclosed with the gem, thus providing a customized behavior for Joe's application.

[PLUGIN SEED] - Data sharder

In the spirit of beating a dead horse, I figured I would stick up this very incomplete plugin that I was fooling around with about 10 months ago.

The idea revolved around splitting up really huge tables into multiple databases, all while keeping "peer" data colocated. The 'parent' records get evenly distributed among databases based on ID, and all child records live with the parent so that you can eagerly load associations via the :include directive on AR finders.

Limitations/Pitfalls: It's probably only useful for very special purpose situations where you're perfoming almost no ad-hoc queries, sorting, etc. The reason is that ad-hoc queries must be joined with software after performing the query on all databases. Clearly this creates trouble with sorted, paginated queries.

Basically throwing it out there as a conversation piece to see if anyone else is exploring the idea of splitting up massive tables.


Here's some illustrative snippets:

class Forum < ActiveRecord::Base
  sharded_among :forums
  has_many :topics
end

class Topic < ActiveRecord::Base
  sharded_with :forum
  belongs_to :forum
end

development:
  adapter: mysql
  database: primary_development
  username: root
  password:
  host: localhost
  forums:
    - adapter: mysql
      database: forum1_development
      username: root
      password:
      host: localhost
    - adapter: mysql
      database: forum2_development
      username: root
      password:
      host: localhost
    - adapter: mysql
      database: forum3_development
      username: root
      password:
      host: localhost

Resultant console/mysql:

>> @forum1 = Forum.create :title => "Forum 1"
=> ..
>> @forum2  = Forum.create :title => "Forum 2"
=> ..
>> @forum3  = Forum.create :title => "Forum 3"
=> ..

>> @forum1.topics.create :title => "forum 1, topic 1"
=> ..
>> @forum1.topics.create :title => "forum 1, topic 2"
=> ..

>> @forum2.topics.create :title => "forum 2, topic 1"
=> ..
>> @forum2.topics.create :title => "forum 2, topic 2"
=> ..

>> @forum3.topics.create :title => 'Forum 3, topic 1'
=> ..
>> @forum3.topics.create :title => 'Forum 3, topic 2'
=> ..


mysql> use forum1_development;
Database changed

mysql> select * from forums;
+----+---------+
| id | title   |
+----+---------+
|  3 | Forum 3 | 
+----+---------+
1 row in set (0.00 sec)

mysql> select * from topics;
+----+------------------+----------+
| id | title            | forum_id |
+----+------------------+----------+
|  1 | Forum 3, topic 1 |        3 | 
|  2 | Forum 3, topic 2 |        3 | 
+----+------------------+----------+

mysql> use forum2_development;
Database changed

mysql> select * from forums;
+----+---------+
| id | title   |
+----+---------+
|  1 | Forum 1 | 
|  4 | Forum 1 | 
+----+---------+
2 rows in set (0.00 sec)

mysql> select * from topics;
+----+------------------+----------+
| id | title            | forum_id |
+----+------------------+----------+
|  1 | forum 1, topic 1 |        4 | 
|  2 | forum 1, topic 2 |        4 | 
+----+------------------+----------+
2 rows in set (0.00 sec)

mysql> use forum3_development;
Database changed

mysql> select * from forums;
+----+---------+
| id | title   |
+----+---------+
|  2 | Forum 2 | 
|  5 | Forum 2 | 
+----+---------+
2 rows in set (0.00 sec)

mysql> select * from topics;
+----+------------------+----------+
| id | title            | forum_id |
+----+------------------+----------+
|  1 | forum 2, topic 1 |        5 | 
|  2 | forum 2, topic 2 |        5 | 
+----+------------------+----------+
2 rows in set (0.00 sec)

Can do eager joins because child objects are sharded with their parents.

epf-lap:/Users/epf/ws/triplebeam $ ruby script/console
Loading development environment.
>>  f = Forum.find 3, :include => :topics
  Forum Columns (0.002027)   SHOW FIELDS FROM forums
  Topic Columns (0.001440)   SHOW FIELDS FROM topics
  Forum Load Including Associations (0.001279)   SELECT forums.`id` AS t0_r0, forums.`title` AS t0_r1, topics.`id` AS t1_r0, topics.`title` AS t1_r1, topics.`forum_id` AS t1_r2 FROM forums LEFT OUTER JOIN topics ON topics.forum_id = forums.id WHERE (forums.id = 3) 
  
?>  f = Forum.find 1, :include => :topics
  Forum Load Including Associations (0.000430)   SELECT forums.`id` AS t0_r0, forums.`title` AS t0_r1, topics.`id` AS t1_r0, topics.`title` AS t1_r1, topics.`forum_id` AS t1_r2 FROM forums LEFT OUTER JOIN topics ON topics.forum_id = forums.id WHERE (forums.id = 1) 

If you want to play with it, download it here

[PLUGIN RELEASE] ActsAsPartitionable

Posted on behalf of Jeffrey Damick:

Introduction

ActsAsPartitionable provides support for using multiple databases to support data partitioning schemes for existing models. This plugin allows models marked as acts_as_partitionable to specify database
partitions to and explicit names to reference those partitions using existing
models. As part of this plugin connection pooling is provided so that many
different models may share the same database for partitions without forcing each partition model object to inherit from a common base class.

Disclaimer

As part of our strategy to address data partitioning we wrote this plugin in preparation to using slave DBs but we are not going to
have those until May 2007. So even though the code is covered with tests (see svn://rubyforge.org/var/svn/actsaspartition/trunk/test/unit/partitionable_test.r
b), it has not yet been used in a production environment.

We will have a discovery period in May when the code is likely to be improved so for now you can use is at your own risk. Meanwhile, we would be happy to fix any issue revealed. Drop us a line at rails-trunk [ at ] revolution DOT com.

Using this plugin should not be your first step in application optimization/scaling or even the second one. Before installing it make sure you understand the implication of leveraging multiple DBs (for example, the potential for cross DB joins).

Example

Sample Model

class SomeModel < name =""> "partition_1", :access => :readonly, :db_config => :read_only

# use the default database for partition_2
   acts_as_partitionable :name => "partition_2"

# use the a specific w/r database for partition_3
   acts_as_partitionable :name => "partition_3", :db_config => :some_model_partition

# Specify the database configuration from a hash
   acts_as_partitionable :access => :readonly, :db_config => {:adapter => "mysql", :database => "partionable_db_test", :username => "root", :password => "", :host => "localhost"}
end

Sample DB Config

dbs:

 database: master_db
 host: master-host

   read_only:
     database: slave_db
     host: slave-host

   some_model_partition:
     database: slave_db_2
     host: slave-host

Usage

r = SomeModel.readonly.find(:first) # executes against the read_only db - slave_db
r.field = 'value'
r.save! # raises ActiveRecord::ReadOnlyRecord
r.readonly? # true

t = SomeModel.partition_3.find(:first) # executes against slave_db_2
t.field = 'some value'
t.save! # success to partition_3

Installation

As plugin:
script/plugin install svn://rubyforge.org/var/svn/actsaspartition/trunk/vendor/plugins/acts_as_partitionable


License

ActsAsPartitionable is released under the MIT license.


Support

The plugin RubyForge page is http://rubyforge.org/projects/actsaspartition

Unveiling Revolution Pages

With the official launch, we're pleased to unveil the beta of codename "fabric"—the "Revolution Pages" platform. Our users will create 'long tail' topic-oriented pages that will help other users find resources on both the arcane and the mundane. The premise is that a page on 'Chemotherapy in Des Moines' could be inherently more interesting/useful than a page on 'Cancer'.

We believe that this application really shows off the capabilities of our home-grown Ruby portal server. Within our portal, most resources are implemented as 'modules' (and any un-modularized blocks of functionality are quickly being converted.) Fabric is simply a layer of client-side functionality (i.e. a boatload of javascript) that sits on top of our existing rendering/page composition capabilities.

Fabric provides the following capabilities:

1. The ability to 'clip' any module on the site into your private clipboard. Try it for yourself—register, then visit any page on the site (try healthy living or conditions for target-rich environments) and click on the scissors (in the upper right, with the page tools.) The modules that are clippable will illuminate.

Demo:

2. The ability to syndicate most modules on the site (the capability exists for all, but we don't syndicate everything for a variety of reasons.) Enter clipping mode and hit syndicate instead—you'll be given a chunk of javascript to paste into your website/blog/etc.

3. The ability to perform full-text search against every module that has ever been rendered on the portal (with a few exceptions where we explicitly choose not to ingest.) This means that not only can we surface pages based on full text, but we can also surface individual components of a page independently. We're using the power of SOLR via acts_as_solr (along with some stomp) for this, and it came out really clean. Props to the SOLR guys (expect some acts_as_solr enhancement patches + commentary on this topic shortly.)

Demo:

4. The ability to build pages using modules gathered via #1 and #3, with a few 'special' modules that bring in external resources (RSS feeds, rich text editor, image upload, YouTube, etc.) We even give you two means to lay out pages—columns and float. For the technical audience, think static vs. absolutely positioned. In float mode, you get Photoshop style snaplines; enable columns and you get 'straightjacketed' slots.

5. The ability to browse pages built via #4, either by full-text search or tag selection.


We developed some interesting client-side techniques to pull this off, and we'll be releasing some of the more generic stuff eventually.

Feel free to poke around and create a page for yourself:

http://www.revolutionhealth.com/fabric

Watch the full demo (still under development):

Recommended Browsers: FF 2.0, IE 7.0
Supported Browsers: Recommended browsers + IE 6
Potentially reasonable, but unsupported: Late versions of Opera, Webkit, Safari

RevolutionHealth Officially Launched!

Its been 3 months since we launched our preview site, and today I am proud to say we've launched the new and improved RevolutionHealth!

I look forward to our folks sharing some of the details behind our products in the near future.

Also, a big thank to the rails community. There is no way we could have done this without Rails, the community, and all the great plugins/blogs/articles out there.

www.revolutionhealth.com

Aside from a full redesign, we have some great new products such as:

KnowYourRisk - "Give us 10 minutes, we'll give you 10 years": A state-of-the-art assessment pioneered by BioSignia. In just 10 minutes, by confidentially entering basic health information, the KnowYourRisk(TM) proprietary program can calculate a user's risk of contracting one of the nine most common diseases – and can provide personalized information on actions to help reduce that risk and live longer.

Medicine Chest - A tool to find drugs and treatments, as well as rate and share your experience with others. This service is in its infancy, but our philosophy is that if millions of people participate, it will emerge as a useful tool for people as they consider their treatment options.

Revolution Pages - Now people can create and customize their own health pages, including their top-rated articles, tools and resources, as well as photos and personal stories. This sophisticated self-publishing capability encompasses Web 2.0's promise of creating customized health content: patients and families dealing with health issues can share the benefit of their experiences with others -- and learn from others, too. See some examples: Sneezing, Type II Diabetes, or Bad Knees.

These are along with many of the awesome products we launched in late January.

Thank you!

[PLUGIN RELEASE] ActsAsReadonlyable

Introduction

ActsAsReadonlyable adds support of multiple read-only slave databases to ActiveRecord models. When a model is marked with acts_as_readonlyable, some of AR finders are overridden to run against a slave DB. The supported finders are find, find_by_sql, count_by_sql, find_[all_]by_*, and reload.

Finders can be forced to fall back to a default DB by passing the :readonly flag set to false.

Disclaimer

As our blog post points out, we wrote this plugin in preparation to using slave DBs but we are not going to have those until May 2007. So even though the code is covered with tests (see svn://rubyforge.org/var/svn/acts-as-with-ro/trunk/test/unit/read_write_model_test.rb), it has not been used outside of those. We would have a discovery period in May when the code is likely to be improved so for now you can use is at your own risk. Meanwhile, we would be happy to fix any issue revealed. Drop us a line at rails-trunk [ at ] revolution DOT com.

Using this plugin should not be your first step in application optimization/scaling or even the second one. Before installing it make sure you understand the implication of leveraging multiple DBs (for example, the potential for cross DB joins).

Usage

Add acts_as_readonlyable to your models backed up by slave DBs. If you want to apply ActsAsReadonlyable to all models, add this or similar code at the end of config/environment.rb:

class << ActiveRecord::Base
  
  def read_only_inherited(child)
    child.acts_as_readonlyable :read_only
    ar_inherited(child)
  end
  
  alias_method :ar_inherited, :inherited
  alias_method :inherited, :read_only_inherited

end

Example

Sample DB Config

dbs:

  database: master_db
  host: master-host

  read_only:
    database: slave_db
    host: slave-host

Note: There is no need for more than one read-only database configuration in your database.yml since you can leverage traditional load balancing solutions. If you still want to use database.yml to spread the load, define multiple entries there and use acts_as_readonlyable [:first_read_only, :second_read_only].


Sample Model

class Fruit < ActiveRecord::Base
  acts_as_readonlyable :read_only
end

Usage

r = Fruit.find(:first) # executes against the read-only db
r.field = 'value'
r.save! # executes against the read/write db

r.reload # executes against the read-only db
r.reload(:readonly => false) # executes against the read/write db


Installation

As plugin:
script/plugin install svn://rubyforge.org/var/svn/acts-as-with-ro/trunk/vendor/plugins/acts_as_readonlyable


License

ActsAsReadonlyable released under the MIT license.


Support

The plugin RubyForge page is http://rubyforge.org/projects/acts-as-with-ro