[PLUGIN RELEASE] ActsAsOverflowable

acts_as_overflowable

Written by Nicholas Lega

== DESCRIPTION:

Allows a column to overflow data into a secondary column if the data size exceeds the character limit. This is useful for fast indexing.

Instead of trying to index text blobs, you can specify a varchar column to be used for indexing. Any characters that do not fit into the base column will be automatically saved into or retrieved from an overflow field.


== SYNOPSIS:

Create overflow column migration:

 class CreateOverflowModels < ActiveRecord::Migration   
   def self.up         
     create_table :overflows do |t|       
       t.integer :overflowable_id       
       t.string :overflowable_type       
       t.string :overflow       
       t.timestamps     
     end

     add_column(:my_class_that_uses_overflows,
                :has_overflow,                 
                :boolean)     

     add_column(:my_class_that_uses_overflows,                 
                :overflowable_text,                 
                :string,                 
                :limit => 255)
   end

   def self.down
     drop_table :overflows

     remove_column(:my_class_that_uses_overflows,
                   :has_overflow)

     remove_column(:my_class_that_uses_overflows, 
                   :overflowable_text)
   end
 end

Add to model:

 class MyClassThatUsesOverflow < ActiveRecord::Base   
   acts_as_overflowable :overflow_column => :overflowable_text,
                        :overflow_limit => 255,
                        :overflow_indicator => :has_overflow,
                        :overflow_table => :overflows
 end

To get and set the value of the "overflowable_text" field as described in the test model, use:

 # #{my_field_name_here}_with_overflow is the method name
 long_text = overflowable_obj.overflowable_text_with_overflow

 overflowable_obj.overflowable_text_with_overflow = very_long_text

Simply using the getters and setters for "overflowable_text" without appending "_with_overflow" to the method name will return the fragment of the text contained in the base column.


== INSTALL:

sudo gem install revolutionhealth-acts_as_overflowable -s http://gems.github.com


== SOURCE:

http://github.com/revolutionhealth/acts_as_overflowable/tree/master


== FEATURES/PROBLEMS:

* Only tested on mysql and sqlite3

[PLUGIN RELEASE] ActsAsSeoFriendly

ActsAsSeoFriendly

== DESCRIPTION:

Create an SEO friendly field for a model automatically based on a given field.

So if you have a Blogs model, and you would like create an SEO friendly version
of the 'title' field, you would just add this to your model and then be able to
use the SEO friendly id as the unique id to the resource. The plugin will only
append an integer to the SEO id if there is a collision.


== SYNOPSIS:

Create seo column migration:

class CreateSeoTestModels < ActiveRecord::Migration
  def self.up
    create_table :seo_test_models do |t|
      t.string :name
      t.timestamps
    end
  SeoTestModel.create_seo_friendly_column()
  end

  def self.down
  SeoTestModel.drop_seo_friendly_column()
    drop_table :seo_test_models
  end
end

Add to model:

 class SeoTestModel < ActiveRecord::Base
  acts_as_seo_friendly :resource_id => :name,
              :seo_friendly_id_field => :seo_id, # default is :seo_friendly_id
              :seo_friendly_id_limit => 100 # default is 50
 end

To lookup the resource in the controllers use:

 SeoTestModel.find_by_seo_id(params[:id])

== INSTALL:

sudo gem install revolutionhealth-acts_as_seo_friendly -s http://gems.github.com

== SOURCE:

http://github.com/revolutionhealth/acts_as_seo_friendly/tree/master

To see the plugin in action see our recipes section, for example:

http://www.revolutionhealth.com/recipes/thai-miang-khem-style-salad


== FEATURES/PROBLEMS:

* Only tested on mysql and sqlite3

Capistrano Off the Beaten Path

Introduction

If you use Capistrano, most likely you use it to deploy rails applications by running it from the project directory. The plugem management tool piggy-backs on Capistrano to execute some recipes without using the current path because recipes operate on gems. Recipes do not, however, completely ignore the current directory, but instead use it for optional customization. Customization is not limited to a deployment recipe from the current directory but can be environment-specific across multiple sites if they each have a special extension gem installed.

Background

We use a bunch of ruby scripts at RHG for deployment and delivery of our numerous applications and shared component. When we started preparing some of that functionality for a public release, we needed to provide a way to customize the scripts. Warren Konkel suggested to migrate them to Capistrano since it was the most familiar tool for rails developers and it has a recipes hierarchy that can be used for customization. So, we wrote the plugem command tool that feeds its parameters to Capistrano via Capistrano::CLI.new(plugem_converted_arguments).execute!
after loading its own recipes.

The Code

    1 module Capistrano
    2   class Configuration
    3
    4     alias :standard_cap_load :load
    5
    6     def load(*args, &block)
    7
    8       standard_cap_load(*args, &block)
    9      
   10       if args == ["standard"]
   11
   12         load_plugem_deploy_recipes(File.dirname(__FILE__) + '/../..')
   13
   14         begin
   15           require 'plugems_deploy_ext'
   16           load_plugem_deploy_recipes(PLUGEMS_DEPLOY_EXT_DIR) # Overriding from extensions
   17         rescue Exception
   18           # No extension is loaded
   19         end
   20
   21       end
   22
   23     end
   24    
   25     def load_plugem_deploy_recipes(dir)
   26       Dir[File.join(dir, 'recipes', '*')].each { |f| standard_cap_load(f) }
   27     end
   28      
   29   end
   30 end

Ignoring lines 14-19 for now, all it does is inject the plugem deployment gem recipes to Capistrano. They are now available for execution by Capistrano. So when I call plugem update my_app it is being translated to cap plugem_update -s plugem_name=my_app(we use the plugem_ namespace for tasks and variables to avoid clashing with standard recipes). Since loading of plugem recipes is purposely not handled via the -f flag of capistrano, the standard deployment recipes like config/deploy.rb are still being loaded.

Customization

The package is extending Capistrano but we wanted it to be customized too. For example, you might want to define your own list of gem servers to download gems from. Capistrano allows to define recipes per project, user, or host. The plugems_deploy takes it a step further and provides a way to customize it per deployment environment, which might contain many hosts. It does this by expecting an optional plugems_deploy_ext gem to be installed on a system (lines 14-19). If it finds the extension gem, it loads recipes from there overriding the default ones.


Conclusion

Capistrano is not only a great deployment tool, it can be used as a base of highly-customizable general purpose tools.

[PLUGIN RELEASE] ActsAsSecure

Introduction

ActsAsSecure adds an ability to store ActiveRecord model's fields encrypted in a DB. When a model is marked with acts_as_secure, the :binary type fields are recognized as needed to be stored encrypted. The plugin does before_save/after_save/after_find encryption/decryption thus making it transparent for a code using secured models.

The plugin supports a master key approach as well as individual records encryption keys. It does not contain any crypto provider but allows to plug in any external one as long as it supports encrypt/decrypt methods.

The fields are converted to a YAML form before encryption. After description they are restored via YAML.load. Since fields are stored encrypted, the find usage is very limited.

Usage

Master Key Provider Usage

class SecureModel < ActiveRecord::Base
  acts_as_secure :crypto_provider => MasterKeyProviderClassOrInstance
end

SecureModel.create()
SecureModel.find(:first)

Individual Keys Provider Usage

class SecureModel < ActiveRecord::Base
  acts_as_secure
end

SecureModel.with_crypto_provider(SomeProvider.new(some_param)) { SecureModel.find(:first) }
SecureModel.with_crypto_provider(SomeProvider.new(some_param)) { SecureModel.create() }

Other Options

acts_as_secure :storage_type => :text -- changes the secure storage type from :binary to the supplied one
acts_as_secure :except => [:field1, :field2] -- disables the secure behavior for the :binary type fields in a supplied list


Example

Let's define three models: Fruit (not secure), SecretFruit (uses the master key approach), and UberSecretFruit (uses the individual key approach).

Rot13CryptoProvider represents a master key provider. SaltedRot13CryptoProvider depends on a salt individual for each record.

Models

class Fruit < ActiveRecord::Base
  has_one :secret_fruit
  has_one :uber_secret_fruit 
end

class CreateFruits < ActiveRecord::Migration
  def self.up
    create_table :fruits do |t|
      t.column :name, :string
    end
  end
end

class Rot13CryptoProvider
  class << self
    def encrypt(arg)
      arg.tr("A-Za-z", "N-ZA-Mn-za-m")
    end
    alias_method :decrypt, :encrypt
  end
end

class SecretFruit < ActiveRecord::Base
  acts_as_secure :crypto_provider => Rot13CryptoProvider
  belongs_to :fruit
end

class CreateSecretFruits < ActiveRecord::Migration
  def self.up
    create_table :secret_fruits do |t|
      t.column :name, :binary
      t.column :fruit_id, :integer
    end
  end
end

class SaltedRot13CryptoProvider
  def initialize(salt)
    @salt = salt
  end
  def encrypt(arg)
    @salt + arg.tr("A-Za-z", "N-ZA-Mn-za-m")
  end  
  def decrypt(arg)
    arg[@salt.size .. -1].tr("A-Za-z", "N-ZA-Mn-za-m")
  end  
end

class UberSecretFruit < ActiveRecord::Base
  acts_as_secure
  belongs_to :fruit 
end

class CreateUberSecretFruits < ActiveRecord::Migration
  def self.up
    create_table :uber_secret_fruits do |t|
      t.column :name, :binary
      t.column :fruit_id, :integer
    end
  end
end

Usage

>> f = Fruit.create(:name => 'passion fruit')
>> SecretFruit.create(:name => 'maracuya', :fruit => f)
>> puts f.secret_fruit.name
maracuya
>> secret = readline.chomp
uber_secret
>> crypto_provider = SaltedRot13CryptoProvider.new(secret)
>> UberSecretFruit.with_crypto_provider(crypto_provider) { UberSecretFruit.create(:name => 'Passiflora edulis', :fruit => f) }
>> UberSecretFruit.with_crypto_provider(crypto_provider) { puts f.uber_secret_fruit.name }
Passiflora edulis

DB

> select * from secret_fruits;
+----+---------------+----------+
| id | name          | fruit_id |
+----+---------------+----------+
|  1 | --- znenphln  |        1 | 
+----+---------------+----------+

> select * from uber_secret_fruits;
+----+-----------------------------------+----------+
| id | name                              | fruit_id |
+----+-----------------------------------+----------+
|  1 | uber_secret--- Cnffvsyben rqhyvf  |        1 | 
+----+-----------------------------------+----------+

Installation

As plugin:
script/plugin install svn://rubyforge.org/var/svn/acts-as-secure/trunk/vendor/plugins/acts_as_secure


License

ActsAsSecure released under the MIT license.


Support

The plugin RubyForge page is http://rubyforge.org/projects/acts-as-secure

Facade To The Rescue

Eddie talked in detail about Facade in this blog before. I would like to provide a couple of examples how that great plugin helps with writing runtime-specific implementations.

User Story: Jim's AWS S3 plugin (test/development/production all use different implementations)

For his rails application, Jim wrote a plugin that allows storage on Amazon S3 (not knowing there is already a very good one). Not wanting to hit the service in test and development mode, he is using mocks with Hash storage for tests, and ActiveRecord for development. He put the mocks in corresponding app/mocks directories and everything works as expected. There is something bothering him though. First, he does not like an idea that various implementation of the same functionality are scattered across the application structure. He would prefer to have a hierarchy that indicates an implementation type (i.e. mocks, active_record, service). More importantly, he is going to release the plugin on rubyforge so he needs a clean way to package all implementations with it. He read about Facade so he decided to give it a try.

First, Jim creates an implementations hierarchy in his plugin:

vendor
  plugins
    aws_s3
      lib
        s3.rb
        service
          s3.rb
        active_record
          s3.rb
        mock
          s3.rb 

Then, he adds facade configuration (vendor/plugins/aws_s3/config/facade.yml):

production:
  s3:
    backed_by: service

development:
  s3:
    backed_by: active_record

test:
  s3:
    backed_by: mock

He modifies the top s3.rb to define the interface:

class S3 < Facade::Base
  backend_class_method :create, :find
end

No changes required for implementation classes, so for example his test mock stays as it was:

class Mock::S3

  @@storage = {}

  def self.create(key, data)
    @@storage[key] = data
  end

  def self.find(key)
    @@storage[key]
  end

end

Jim extracts a plugin, packages it, and publishes it to rubyforge for public consumption.


User Story: Betty's Crypto Library (packaging gem with its own mock implementations but allowing an override in applications)

Betty works for a large healthcare provider with many internal projects embracing Rails for web-based implementations (now you know this is a fictional story ;-) ). They lack a strong crypto library they can reuse across applications to keep them compliant with HIPPA regulations. Betty has experience with RubyInline C so she volunteers to write one. She releases it as a gem to an internal gem server together with a Facade based mock implementation using ROT13 instead of strong encryption, so rails developers have an easy way to recover encrypted data in unit tests. The gem contains config directory with facade.yml:

production:
  healthcare/scrypto:
    backed_by: healthcare/live

development:
  healthcare/scrypto:
    backed_by: healthcare/live

test:
  healthcare/scrypto:
    backed_by: healthcare/mock

Joe is a developer of a small internal application that utilizes the crypto library gem. He found that, since requirements for the library was to withstand to a brute-force attack by an average NSA computer for at least 48 seconds, the library is very CPU demanding and it makes development on his old iBook unbearably slow. Joe decides to override the development mode facade configuration for the gem so it falls back to the mock implementation for both development and testing. He adds facade.yml to his application's config directory with a single entry:

development:
  healthcare/scrypto:
    backed_by: healthcare/mock

Since Facade uses ConfigurationLoader for configuration files loading, application config is being merged with the one enclosed with the gem, thus providing a customized behavior for Joe's application.

[PLUGIN RELEASE] Configuration Files Loader v1.0.0

Configuration Files Loader can be used as a gem or a Rails plugin to load various config files.

It finds config file fragments in a Rails config directory and in config directories of plugins and dependent gems. It then tries to merge them in the following order: gem<-plugin<-application. This allows overrides of global (gem/plugin) configs by individual applications. The supported types for merging are String, Hash, and Array. It caches the content of files by default. ConfigurationLoader is RoR environment aware and provides a shortcut (load_section) to load a section of a config file corresponding to RAILS_ENV. It is being used in another method provided by ConfigurationLoader - load_db_config. It loads a section from config/database.yml providing a convenient method for placing secondary DB entries in a code as seen here:

establish_connection ConfigurationLoader.load_db_config['secondary_db']

See the DRYing Up Configuration Files post in our team blog for additional details.


Setup

Installation

To install Configuration Files Loader as a gem:

Use gem install configuration_loader or download the gem file from the project page and install it.

To install it as a Rails plugin:

ruby script/plugin install svn://rubyforge.org/var/svn/config-loader/trunk/vendor/plugins/configuration_loader


Usage

require 'configuration_loader'

ConfigurationLoader.load_db_config # gets the RAILS_ENV section from database.yml

ConfigurationLoader.load('cfg.yml') # loads a YAML-processed config file

ConfigurationLoader.load('cfg.yml', :erb => true) # loads a ERB+YAML-processed config file

ConfigurationLoader.load('cfg.yml', :section => RAILS_ENV) # loads a current RAILS_ENV section from a YAML-processed config file

ConfigLoader.load('cfg.yml', :cache => false) # loads an uncached copy of a YAML-processed config file


License

Configuration Files Loader is released under the MIT license.

Update by Sean (6/20/2008)
This is now available on github as a gem.

To install:
gem install revolutionhealth-config_loader -s http://gems.github.com

Or simply add the following to your Rails 2.1-compatible application:
config.gem 'revolutionhealth-config_loader', :lib => 'config_loader', :source => 'http://gems.github.com'

DRYing Up Configuration Files

Introduction

Our post about deployment process explains that some configuration files are overridden at the time of deployment. Maintaining those files could quickly become a nightmare unless the development team constantly evaluates them to gather only pieces that differ into the deployment specific configs, i.e. keeps them DRY. It has been a long journey for us and we are still adjusting the config file usage for our project.


ConfigurationLoader

We access all configuration files from our code via ConfigurationLoader (and, yes, it is a plugem itself). It provides some convenience methods for major configs. For example, to establish a non-default connection from a model, this piece of code could be used:

establish_connection ConfigLoader.load_db_config['secondary_db']

The load_db_config method knows that database.yml is sectioned per environment so it loads the file and returns the corresponding to current RAILS_ENV section. ConfigLoader is an instance of ConfigurationLoader with caching enabled. Since we use a lot of configuration-driven parameters in our models and controllers, caching this info saves unnecessary file system calls and ERB parsing.


DRYing Up By Keeping Configs Close to The Source

Another property of ConfigurationLoader is that it looks not only in the application config directory but in configs of plugins and gems. All found configs are merged (for usual config ruby structures like Hash and Array) in the order gems<-plugins<-application. It allows us to keep the default configuration within a gem to be shared between multiple applications, and, at the same time, lets applications overwrite it if needed. Together with the power of deployment time overriding, which is applicable for shared components as well, it helps us to keep only application-specific entries in its config files. The following is a concrete example from today: We have been using the Browser Logger as a plugin in our applications. Aaron decided to convert it into a plugem. We obviously don't want it to be enabled in our production deployment environment but we want to be able to see logs through a browser on developers' workstations and in QA. The gem initializer loads browser_loggger.yml via ConfigurationLoader to determine whether it is enabled or not. It is not DRY to put browser_loggger.yml in every application that uses Browser Logger. Instead, the gem itself contains config/browser_loggger.yml, with the property enabled, config/deployment/prod-browser_loggger.yml, with the property disabled, and config/deployment.yml with a single entry 'files: browser_loggger.yml' which defines that config/browser_loggger.yml will be overridden on production boxes with the content of the second file (see When Capistrano Is Not Enough for details how deployment time configuration works).


DRYing Up By Extracting Changing Parts in A Separate File

Our rails applications call a lot of backend services. Services configuration is stored in the service.yml file. Service endpoints usually follow same naming conventions in respect to a port and a path between different deployment environments. So at some point, we extracted host names into a separate file (service-hosts.yml) and changed our deployment.yml to override it instead of service.yml at the deployment time. Since service.yml is ERB-processed, we leverage it for loading the host file:

<% hosts = ConfigLoader.load_file('service-hosts.yml') %>

service_cfg: &service_cfg

foo:
url: http://<%= hosts['foo'] %>:8080/services/foo
timeout: 5

bar:
url: http://<%= hosts['bar'] %>:8080/services/bar
timeout: 5

production: *service_cfg
development: *service_cfg
test: *service_cfg

Conclusion

Our experience has shown that combination of deployment and run time configurations is powerful enough to handle most use cases we have had so far. There is always room for improvement, however. For example, we have lately run into a problem when one of the production boxes needed one of parameters in service.yml set to a different value then the rest of production boxes. We can address it within the current framework by copying a content of prod-sevice-hosts.yml to prod-box-1-service-hosts.yml and changing the parameter there. Since the host name (prod-box-1) takes precedences over the host class name (prod) it would fix the issue but the maintenance cost of two almost identical files is high. An alternative solution would be to enhance the framework to allow additional runtime overriding of values of service.yml from service-override.yml. Then we can create prod-box-1-service-override.yml which would contain a single entry with the parameter it needs to override. Work in progress...

Gem-based Deployment and Delivery: Part 2 - Distribution via Gem-servers

Part of the Gem-based Development Process series:

The Idea

Once we packaged all of our applications and shared components as gems, it did not take us long to realize that we could leverage the native method of gem distribution - gem servers - to build an infrastructure for pushing products through the dev/qa/production deployment stack. The idea of gem promotion was born.

There are three layers of gem servers - one per deployment environment (dev/qa/production). Gems can be promoted from the upstream gem server to the downstream one. The direction of promotion is dev->qa->prod. The individual boxes within the deployment environment install gems only from the server that serves their environment. For example, when a QA person installs gems on qa7-rails & qa8-rails boxes, he uses the qa-gems server as the source (as in gem install xyz --source qa-gems --remote). The dev gem server is the base one where freshly built gems are pushed to.


The Implementation

Since our QA and Operations teams wanted to have full control over what they have on their gem servers, we adopted the push-pull model instead of a more simple push-only one (when a gem promotion works by pushing from the upstream to the downstream). In our current model, developers push gems to the base (dev) gem server first. After it is tested and is QA ready, the manifest file, containing a specific gem name and version (together with names/version of all dependent gems), is generated and sent to QA. A QA installer uses the manifest to pull gems from the upstream dev gem server to the downstream QA one. He then goes to the individual QA boxes and upgrades the gems there. When QA clears the build, the manifest is sent to Operations to repeat the procedure in production with the qa gem server being the upstream one.

We greatly simplified the usage by hiding decisions about upstream/downstream, the source gem, etc. behind a single tool that was deployment environment aware. For example, to pull down gems based on the manifest to the qa gem server, the installer issues a command rhg pull manifest_file on the qa gem server box. The rhg tool queries the hostname of the machine it is being run on, and based on its name (qaX-YYY), it picks the upstream server (dev-gem). Running rhg up gem_name gets the gem (and all of its dependencies) from the environment specific gem server, with the base (dev) gem server being the default option so developers can use the same command for updating their local gem repositories.

Under this scheme, each environment has full control over which gems are installed there and the only piece of information needed for promotion is the manifest file declaring specific versions to promote.


The Reality

We built the tools and dev/qa gem servers and started pushing gems to QA using the described approach. When the time came to implement it in production, our operations team rejected it. They had a reason: they wanted a unified approach for delivering all packages they need to install. Since we have Java and many other non-ruby applications while the ops requirement was to use RPMs for package distribution, we had to adapt. We retained the dev gem sever and continued to allow developers to push gems to it. We continue to generate the manifest file, but instead of delivering it as is, we use it to pick up the specific versions of gems and repackage them as RPMs (using a modified version of gem2rpm). They are then delivered to QA and later to production.


The Future

We still believe that using the 'gem servers' hierarchy is the right way for plugem-based applications distribution. Most likely the original scripts will be a part of the near future plugem public release. We hope that one day it might become the preferred method of rails applications distribution.

Locking down a deployed application

A single host might have multiple applications installed each on a different release track. Since all our applications are distributed as a bunch of gems, there is a potential issue with shared components. Even though the whole portal is being tested when some application or component is being pushed through the stack, there is always a possibility of incompatibility of on of the upgraded shared component with another dependent application. In some other environments there would only one solution - to roll back the whole new application. In our environment we have another option - to lock down the affected application. When an application is deployed, one of the deployment steps is to create a gem repository inside the application structure. The application gem repository has the sources gem installed and the rest of gems sym-linked as in this example:

actionmailer-1.2.5 -> /usr/lib/ruby/gems/1.8/gems/actionmailer-1.2.5
actionpack-1.12.5 -> /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5
...
sources-0.0.1

Gems put into an internal gem structure are those it was tested with, so they are guarantee to work. All is needed to lock down an application now is to set the GEM_HOME environment variable on a web server instance, serving the application, and to point it to the application internal gem repository. After that the application won't use the latest versions of the gems but those specific it is locked to.

Plugems

Part of the Gem-based Development Process series

Implementations:

Runtime

Deployment

*****

We started with a small application. A few controllers, a few models. Soon the company grew, the development team grew, and with that the application grew. Luckily, we namespaced applications early on, so it was mostly clear where one application lived from another, but the rails project became too big. As one coworker (Eddie) put it, "My TextMate file find is getting too slow!". It was time to split applications. But for a portal with a single look and feel, how do you share things across applications? Here is how we do it (this week).

We wanted to use gems, as described in Part I, but without gems being true 1st class citizens in rails, there was no way to make it happen. So what was important to share and more importantly version?

I. Code
Thats obvious, rubygems already does (most of) this for us.

II. Views
With a common header, or commonly shared assets, we'd like to leverage those.

III. Rake Tasks
We have lots of rake tasks to do all sorts of things, we need to share those too.

IV. Assets
We'll cover this later. This gets complicated, and needs deployment input.

V. Plugems as Plugins
Using gems, as plugins, in your plugin dir, and the A/B/C problem.




I. Code
We didnt do (much of) anything to get this. Thanks guys who wrote RubyGems!

II. Views
We needed to load views from a few places, and it took some diving into rails internals to figure out how. There were a couple things we needed to figure out.

1. Where to look for views.
2. The order in which we look.
3. Caching. Globbing for files is slow... especially across many directories. Cant do this a lot.

1. So we first overwrote the ActionView::Base's implementation of finding files. That was straightforward. We also fixed ActionController::Layout to allow layouts to live in gems as well. Once we provided ActionView and ActionController with a bigger list of partials, it could then load the partial from anywhere. Now you'll see the path to a file in a gem in certain stack traces, just to prove that where to find it.

2. So why is the order important and powerful? Lets say you decide to use our base_ui gem and its default layout.


<%= render :partial => "header"%>
<%= @content_for_layout -%>
<%= render :partial => "footer"%>



In our gem,
/gems/base_ui/
/views/layouts/
default.rhtml
_header.rhtml
_footer.rhtml

You've been told to build the FooBar application, whos pages dont have the same header. So in your application, add a
/app/views/layouts/_header.rhtml, and put whatever you want there. It just works.

So the dependency order is: app ==> plugins ==> gems. There is almost always true. I'll explain the exception later when "developing" things that are gems.

Note: The compiled template method names get kinda ugly, but who cares.. You only see those in some stack traces.

3. Caching was quickly important. Disclaimer: We only look for views in gems and plugins at startup. After that, bounce your app to pick up new files. Thanks to Zed, mongrel_rails restart is wicked fast.

III. Gem Rake Tasks
So the easiest way to include rake tasks from gems was to add a bootstrap.rake into /lib/tasks/bookstrap. We grab the manifest file for the application, and for each of the gems defined, 'load' the rake files. This was a quick fix and could probably be more elegant.

IV. Assets
This conversation gets complicated quick. We'll cover this later.

V. Plugems ( Plugin + Gem = Plugem!)
So now you've taken all your favorite plugins and made them gems. In the next article, I'll cover developing gems as plugins, and the A --> B --> C dependency loading problem, where A and C are gems, and B is a plugin.

Gem-based Rails Dependency Loading: Part 0 - Introduction

Part of the Gem-based Development Process series

*****

Once you begin to build a lot of plugins (shared across numerous Rails applications), cross-plugin dependencies start to creep in. If you've ever renamed your plugins to effect load order, then you know what I'm talking about. Our idea, which has been extremely successful, is to package interesting dependencies as RubyGems. These dependencies include proprietary shared plugins as well as library dependencies.

Since our applications care to share a lot more than bundles of Ruby classes, we decided to beef up plugins as well.

• Part I: Justification
• Part II: Bundling a plugin as a Gem and getting it all to work with Rails
• Part III: Extracting views, partials, tasks and layouts into plugins and gems
• Part IV: Retaining developer convenience in a Gem-driven Rails application

Next time: Our justification behind this approach

A New RHG Developer's Illustrated Primer

Part of the Gem-based Development Process series

*****

Dan had just completed his last project and was looking for another job where he can apply his skills in Rails and AJAX, when he run into a blog discussing challenges of an enterprise RoR-based application. He never worked on a large-scale Rails project so he was eager to try. He applied to a position at Revolution Health Group and after going through series of interviews, received an offer. People he talked to on campus, and his own scrutinizing of the web-site on which he was supposed to work, made him decide to accept the offer. He started on Monday.

Now it's Wednesday. Dan has spent his first two days at RHG reading the development's wiki, talking to coworkers, and configuring his new development workstation, a MacBook Pro. He was just assigned a bug, so he is eager to prove that he was the right choice for a job.

First, he decides to install a copy of one of the applications (rop) locally. Armed with instructions from wiki, he installs the deployment support tools:

$ gem install rhg_deployment --remote --source http://gems.revolutionhealth.com:8808

He now has two new commands - rhg and rhgcontrol. He sets up a runtime environment for the application:

$ rhgcontrol setup
Setting up the runtime environment
mkdir -p /opt/rhg/applications/etc
mkdir -p /opt/rhg/applications/tmp
...
mkdir -p /opt/rhg/applications/log

He downloads the latest version of the application with all its dependencies to his workstation:

$ rhg update rop
Bulk updating Gem source index for: http://gems.revolutionhealth.com:8808
Installing [ actionwebservice, 1.1.6 ]
Installing [ activesupport, 1.3.1 ]
Installing [ rails, 1.1.6 ]
...
Installing [ rhg_ui, 1.5.40766 ]
Installing [ rhg_migrations, 1.0.37541 ]
Installing [ rop, 1.4.40837 ]

Dan has all he needs now so he deploys the application, converting it from a static gem to a live site:

$ rhg deploy rop
Deploying rop-1.4.40837
Loading /usr/lib/ruby/gems/1.8/gems/rop-1.4.40837/config/deployment/database.yml as database.yml
Executing: ln -nfs /usr/lib/ruby/gems/1.8/gems/rop-1.4.40837 /opt/rhg/applications/rop
...
Executing: ln -nfs /opt/rhg/applications/log/rop log

It is a rails app, so it needs a DB which he creates using mysqladmin. He then issues a command to add a structure and populate with data his development DB:

$ rhgcontrol migrate rop
Running migrations for rop
cd /opt/rhg/applications/rop
Executing: rake db:migrate
...

Dan plans to use both lighttpd and mongrel to host the application, but first he goes with the default one, lighttpd, using a config supplied with the deployment tools:

$ rhgcontrol add rop
Adding lighttpd configuration for rop_8001

$ rhgcontrol start rop
Executing on rop_8001
Executing: /usr/sbin/lighttpd -f /opt/rhg/applications/etc/rop_8001.conf

Dumping the runtime manifest

He points his browser to http://localhost:8001 and plays with the application. He feels like doing some coding. He checks out the latest application code from subversion and runs mongrel from the top of the source tree:

$ mongrel_rails start -d

He fixes some code and navigates his browser to http://localhost:3000 to see the changes. It works but he needs to do some more fixes, this time in a shared component rhg_ui. He checks out the latest component code, sym-links it to the vendor/plugin directory of the application making it temporarily a plugin, and changes some code to see the immediate result. The bug is resolved, and, after running unit tests, he checks in the modified code for both the application and the component.

The changes he made are in a latent state. They are in the source tree but no gems were built off them yet. Dan knows that QA usually builds application gems off the latest code but not component ones. He decides to build a gem for rhg_ui himself. He navigates to the top of the component source tree and runs a command to tag, build, package, and publish the component as a gem to our local gem server:

$ rhg publish
...
Committed revision 40967.
...
Checking out tag to /tmp/rhg_ui-1.5.40966
Changing directory to /tmp/rhg_ui-1.5.40966
Building gem from tag
...
Successfully installed rhg_ui, version 1.5.40966
Pushing gem to development gem server via rhg tool: rhg push rhg_ui
Bulk updating Gem source index for: http://gems.revolutionhealth.com:8808
Bulk updating Gem source index for: http://gems.revolutionhealth.com:8808/archive
SSH User: dsmith
SSH Password: XXXXXXX
Publishing gems...
...
Refreshing the gem server indexes at /opt/rhg/gems

His changes are now packaged and ready to be picked up by QA for testing and pushing to production. How that part is done is for another time...

Gem-based Development Process

The gem based development with multiple applications and shared components was our solution when we could not bear anymore the pain of working on a mammoth monolithic application. The code built to support that change makes it easier for developers, QA team and Operations to work on, deploy and deliver the whole application as well as individual components. Next installments, written by people who made it possible, will reveal the ideas and implementation details of how pushing gem usage to extreme let us survive and get the the site to where it is now - open for visitors, helping them to better manage their health and the health of their families.

Coming soon:

• A New RHG Developer's Illustrated Primer
• Gem-based Rails Dependency Loading Series
• Plugems - Gems as First Class Rails Citizens Series
  
• Gem-based Deployment and Delivery Series