Friday, May 04, 2007

[RELEASE] Plugems packaging

The Plugems runtime is enough to justify their existence, but it does not stop there. Since we already defined our dependencies in a gem-like fashion, it is only a small step to start using them for packaging as well. All that is needed for this is the plugems_deploy gem that can be installed from rubyforge. After the gem is installed, a new plugem command comes up. It piggy-backs to capistrano to provide some plugems-related recipes. The recipe packaged with the initial version is build. This action allows you to package your project as a gem.

Let's take for example our sample application's config:

:version: [1, 0]
:name: "cool_application"
:description: "My First Plugemified Application"
:dependencies:
- ['some_gem', '~> 1.0']
- ['other_gem', '> 2.0']
- ['one_more', '2.0.1']

When we run the plugem build on the top of the application, we get a gem built:

$ plugem build
* executing task plugem_build
rm -rf pkg
mkdir -p pkg
Successfully built RubyGem
Name: cool_application
Version: 1.0.0
File: cool_application-1.0.0.gem
mv cool_application-1.0.0.gem pkg/cool_application-1.0.0.gem

If you looked inside, you would find that the attributes and dependencies from the manifest file were translated to corresponding gem attributes and dependencies.

Where did the build (micro) revision come from?

You might noticed that the manifest defined only major and minor revisions but there was a micro added during the packaging time. The plugem packaging process follows the Rubygem's rational versioning policy giving you full control over the build revision. The full version can be defined in the manifest file a-la-Rakefile (i.e. :version: [1, 0, 1]). You can derive it dynamically from a source like svn revision. You just set the capistrano gem_micro_revision variable in a deployment recipe (like config/deploy.rb). An svn based example is:
set :gem_micro_revision, `svn info`.grep(/^Revision:/).first[/(\d+)/][$1]

And you can always overwrite the full version via the '--version' flag of plugem: plugem build --version 3.2.1

The choice is yours.

You might wonder why would you package your rails application as a gem. The rationale is that it allows you to utilize the only ruby-native packaging and distribution system to distribute and deploy your application. The next plugem_deploy releases and this series installments would provide the tools and guidance how to do that.

But it is not just for packaging...

Since you have you dependencies clearly defined, it is really easy to update them to the latest version. Just run plugem update from the project directory:

$ plugem up
* executing task plugem_update
Updating some_gem (~> 1.0)
* executing task plugem_install
Bulk updating Gem source index for: http://gems.rubyforge.org
Installing [ some_gem, 1.2.2 ]

You also have a full control which gem servers to use. Set the variable in your deployment recipe:
set :gem_servers, [ 'http://gems.mycompany.com:8808', 'http://gems.rubyforge.org' ]

and see the difference:

$ plugem up
* executing task plugem_update
Updating some_gem (~> 1.0)
* executing task plugem_install
Bulk updating Gem source index for: http://gems.mycompany.com:8808
Bulk updating Gem source index for: http://gems.rubyforge.org


To Be Continued ...

Wednesday, May 02, 2007

CSS: CSS Browser Selector


Each browser has its quirks. Having a clear way of organizing the work arounds
for those quirks is a challenge. Using CSS selectors is not a new idea, but I thought it might be helpful to others to give an example of the technique and how we've been able to successfully deploy it for revolutionhealth.com.



First, for IE (the browser that usually requires a hack), we can rely on conditional comments. This is good because it means we don't need to depend on Javascript. For other browsers we'll have to rely on a document.write solution. For Safari, Opera, and Firefox, we rely on the script from http://rafael.adm.br/css_browser_selector/ and for IE, conditional comments.


Here's what we include at the top of our document. (The browser_detect_start partial.)



<!--[if lt IE 7.]>
<div class='ie ie6'>
<![endif]-->
<!--[if IE 7]>
<div class='ie ie7'>
<![endif]-->
<script type="text/javascript">//<![CDATA[
var d = browserCSSDetection();
if( d.browser != "ie" ){ document.write( "<div class='" + d.browser + " " + d.os + "'>" ); }
//]]></script>

And here's what we do for the end of the document. (The browser_detect_end partial.)

<!--[if IE ]>
</div>
<![endif]-->
<script type="text/javascript">//<![CDATA[
var d = browserCSSDetection();
if( d.browser != "ie" ){ document.write( "</div>" ); }
//]]></script>


The browser detection in Javascript. This could be enhanced further, but for us this allowed us to get the site working relatively easy in Konqueror. As well it enabled us to fix our menu's so that they float over flash in Linux using this techinque.



function browserCSSDetection()
{
// see: http://rafael.adm.br/css_browser_selector/
var ua = navigator.userAgent.toLowerCase();
var is = function(t){ return ua.indexOf(t) != -1; };
var b = (!(/opera|webtv/i.test(ua))&&/msie (\d)/.test(ua)) ?
('ie ie'+RegExp.$1) :
is('gecko/') ? 'gecko' :
is('opera/9') ? 'opera opera9' :
/opera (\d)/.test(ua) ? 'opera opera'+RegExp.$1 :
is('konqueror')?'konqueror' :
is('applewebkit/') ? 'webkit safari':
is('mozilla/')?'gecko':'';
// see: http://www.mozilla.org/docs/web-developer/sniffer/browser_type.html
var os = (is('x11')||is('linux'))?' linux':is('mac')?' mac':is('win')?' win':'';
var css = {browser:b,os:os};
return css;
}


Finally, to make all this fit nicely into a layout:

<head>
<%= javascript_include_tag 'browser_detect' %>
</head>
<body>
<%= render :partial => "browser_detect_start" %>
<%= @content_for_layout %>
<%= render :partial => "browser_detect_end" %>
</body>


Update:
Here's a simple example of what this enables. The advantage of this over a conditionally included file is it keeps everything about the login box isolated to one place. You don't need to worry about openning up a separate file to make your IE fixes. We do use iefix specific CSS for our site, but only for very large features like menus



#login {
padding:0px;
}
.ie6 #login {
padding: 2px;
}

Plugems - Why views in plugems?


When we started to develop Plugems we had multiple applications that all shared a similar look and feel. One concept that emerged recently for me was that layouts can act like objects. In our suite of applications, we have a base layout that each application can derive from to satisfy it's unique set of presentation requirements. For example our UI plugem has the following:



ui/views/layouts/default.rhtml

Using a little magic from fora.pragprog we can extend our default layout writing the following:

<% inside_layout 'default' do -%>
<%= @content_for_layout %>
<% end -%>



As well as with layouts, there are also smaller blocks of UI that make up a page that can be shared. I've always thought of these as widgets, which in many cases can be defined as a set of partials and helpers. An application may place a widget on a view, feeding it the data needed to bring it to life. This is different from components because there is no controller logic. Really the presentation side of a widget can be expressed by a set of partials, Javascript, CSS and a healthy dose of ruby [e.g. helpers]. In this way partials are the basic building block on which many UI components can be built. By themselves they are incomplete, but within the context of an application they are alive...




Plugems enables you to share common UI widgets across multiple applications. Common assets (css, js, images) can live in a plugem, but be shared across applications at deployment time. I hope to post more in depth on this topic in near future.

Plugems - More than dependency management

In our previous post, we attempted to compare plugems to various other techniques in the community. James Adam, of Rails Engines responded here and here, and raises some interesting points I'd like to address.

First, all of the comments comparing engines referred to the engines plugin that enhances rails, as described in this post, but I added an updated note to the Engines comparison section for clarification.

That being said, we believe versioned dependencies provide more value than vendoring all dependencies for medium to large rails applications. We do suggest you start all shared code as plugins, and move towards a plugems approach when multiple applications want to leverage it.


Version Policy Adherence

Our group strongly follows the versioning policy outlined in the Rubygems User Guide . Which is:

  • Versions shall be represented by three non-negative integers, separated by periods (e.g. 3.1.4). The first integer is the "major" version number, the second integer is the "minor" version number, and the third integer is the "build" number.

  • A category 1 change (implementation detail) will increment the build number.

  • A category 2 change (backwards compatible) will increment the minor version number and reset the build number.

  • A category 3 change (incompatible) will increment the major build number and reset the minor and build numbers.

We have a test harness that wraps each plugem, that helps our team adhere to the backwards compatibility, and as always TEST!!! TEST!!! TEST!! Our applications are safer because we lock most plugem dependencies pessimistically, e.g.:

'foo', ' ~> 1.0.0'

This states we'll load any plugem where the revision is: 1.0.0 > revision < 1.1.0

If the author of the plugem has non-backwards compatible changes, he can simply branch the plugem, update the manifest by incrementing the minor, and redeliver. All existing applications will NOT be affected, even if its installed on the same host.

In the majority of cases, the machine-wide updates are for picking up pure bugfixes (which should be safe) via the increment of micro. Bigger releases aren't automatically picked up due to the pessimistic lock.


Plugem Deployment

But Plugems is not just dependency management! If you vendor everything, you lose the ability to push micro changes out quickly to other developers and the ability to deploy with dependencies.

Lets say I have a foo application, with a bar dependency. When i've made a change to the bar dependency that I'd like to share with others, I do plugem publish bar, which pushes the change to our gem repository. Then when Eddie is working on the foo application he can run: plugem up foo . If the recently published bar gem conforms to the manifest in the foo application, he'll get an update. He restarts his application, and he can go on developing.

Even better, the foo application is also a plugem! So I can publish the foo application in the same way. plugem publish foo. Then on each deployment host, plugem up foo. There are obviously more details, but this gets why we think having dependency information in the application is important. And we cant take credit for this. It was Dave Thomas at RailsConf 2006 that had the idea of using gems for deployment.

ScribeMedia has a great video series from RailsConf 2006. See Part 5



We've talked about it a bit before
, and plan to dive into the details of plugem deployment in the next few days and look forward to feedback from the community.

Tuesday, May 01, 2007

[RELEASE] Plugems Runtime

One sentence summary: "Everything is a gem, most things work from within the gem, and all dependencies are accounted for."

How are Plugems different from...

Plugins

  1. Plugems are a superset of Rails plugins.
  2. Plugems are bundled as Rubygems and are installed on the server hosting your application. Plugins are directories of files that get copied into the vendor/plugins directory of every application that leverages them.
  3. Plugins aren't dependency-managed. If one plugin depends upon another, then you have to handle the dependencies in a very implicit way. Plugems supports full dependency management: applications declare dependencies, dependencies declare their own subdependencies. Plugin load-path hacks aren't enough because they fail to address the issue of 'behavioral dependencies' (i.e. one plugin's init needing run after another.)
  4. Thanks to Rubygems, Plugems supports revision/release conventions. It was important for us to version dependencies explicitly or implicitly. Now you have full control.
  5. Plugems can be distributed through the standard Rubyforge repository (but they're still backwards compatible with plugin install.)
  6. Plugems adds support for the loading of views from gems (in addition to rake tasks, models, etc.)
  7. With plugins, you can't update a plugin for all apps on a box without pushing the applications. With Plugems, you simply install the new gem and bounce the apps. All applications that are not hard-locked to a particular revision of that plugem will pick up the latest version automatically.
Note: We still recommend plugins for quick and dirty things that you don't expect to undergo a heavy rate of change or become a shared dependency. At RHG, most Plugems start their life as plugins (and get promoted once multiple applications want to share it.)

Engines

  1. Engines have no support for dependency management or revision/release conventions (see #2, #3 for plugins.)
  2. With Engines, you can't update an engine for all apps on a box without pushing new application code. (see plugins #7)
  3. The Plugems library is distributed as a ruby gem -- no plugin or svn external to maintain. Simply update your gem when you're ready to take a new version.
  4. Plugems intentionally don't support migrations -- It was a fundamental choice we made early on to avoid the temptation to build apps that share databases (resulting in highly-coupled applications with the database as the integration point.)
  5. Plugems intentionally don't support runtime asset loading. We believe the coalescence of assets is a deployment-time issue (plugem deployment is forthcoming in this series.) For local development, something like this is needed, though. We use a mongrel handler for local development.
  6. Plugems intentionally don't support to routes. We decided that applications own urls, not shared libraries.


Recent related core discussion
  1. This patch does not account for rake tasks, views, etc. from within plugin gems (albeit probably intentionally for consistency with plugins.)
  2. With the latest patch, gems are bundled with the application, preventing live-uptake of new versions (see Engines #5)

Other notable features

  1. Plugems is accompanied by a tool for examining, updating, building, and 'deploying' plugems to other developers on the team (or developers on other teams that share your plugem.). Think `svn up; plugem up`. These capabilities leverage the existing Rubygems platform. We're still busy preparing these for release--this release is only the runtime environment.
  2. Support for applications-as-gems (more useful with the release of the deployment-time tools.)
  3. You can develop a plugem without loosing its dependency knowledge. You can check out or symlink the plugem into your plugins directory for active development and Plugems will ignore the installed gem but still load that 'bypassed' Plugem's dependencies.

Installation Guide:

Install it on your box:

As a gem:
gem install plugems

Create your plugem configuration:

Add config/manifest.yml describing your plugem and its dependencies:
:version: [1, 0]
:name: "cool_application"
:description: "My First Plugemified Application"
:dependencies:
- ['some_gem', '~> 1.0']
- ['other_gem', '> 2.0']
- ['one_more', '2.0.1']

Bootstrap the plugems:

Add the plugems requirement to config/boot.rb at the bottom right before the initializer call:
  # Add this line:
require_gem 'plugems', '~> 1.0'

Rails::Initializer.run(:set_load_path)
end


You are all set to enjoy the power of plugems!

License

Plugems released under the MIT license.


Support

The plugin RubyForge page is http://rubyforge.org/projects/plugems

We've been using this internally for quite a few months, and are really interested in feedback from the community. Please tell us what you think!