>>> import shlex
>>> shlex.split('foo "bar baz" qux')
['foo', 'bar baz', 'qux']

It’s useful for creating your own mini-languages or external DSLs that need to parse quoted strings like the one shown above.

We recently built an inventory tracking application in Ruby that has a user interface for selecting search filters. To expose the same search capabilities as a web service, we created a simple query language. I was looking for an shlex equivalent in Ruby.

It turns out that the Ruby Standard Library has a module called Shellwords:

>> require 'shellwords'
=> true
>> Shellwords::shellwords('foo "bar baz" qux')
=> ["foo", "bar baz", "qux"]

Shellwords is a little less capable than shlex but handles the most common use case just fine. It’s a convenient solution for a problem that comes up too often.

Supervisor as a Platform

I will quickly introduce you to Supervisor and the immediate benefits of running your server processes under it. We will then dive into how applications written specifically for Supervisor can take advantage of it as a platform — writing your own event listeners to observe Supervisor and the process lifecycle, controlling with XML-RPC, and extending the Supervisor core with your own Python extensions.

This will be an expanded version of the talk I gave with Chris McDonough at PyCon 2008. Since PyCon, there’s been quite a few interesting developments in Supervisor like the ability to extend supervisorctl and progress made on configuration reloading. We’ll touch on these as well, so if you attended the PyCon talk there will still be new and interesting material in this talk for you.

Integration Testing PHP Applications

While more PHP developers are recognizing the importance and benefits of unit testing, the uptake of PHP developers using automated integration or acceptance testing is relatively slow. This testing is equally crucial to maintaining the integrity of applications.

I’m going to help get you started testing at the application level with practical tips and source code. We’ll look at how to structure your HTML markup so it’s more easily testable, making tests easier to write and maintain with CSS selectors, organizing your tests, and testing with or without a browser.

We write a lot of PHP applications with this kind of testing at Maintainable. Before the conference, we plan to release some open source PHP code to help you test that we’ll cover in the talk as well. I’d also suggest you check out Sebastian Bergmann’s tutorial session on PHPUnit’s integration with Selenium RC.

I’m pleased to have been able to contribute a recipe to Mike Clark’s new book, Advanced Rails Recipes. The concept presented in my recipe, “Fail Early”, is that you can use initializers to prevent your application from starting up under certain conditions.

Rails applications typically run under persistent application server processes, like mongrel or thin. When a Rails application starts, it goes through a startup procedure that is performed only once. The startup includes reading the environment configuration files and running any initializers that have been set up. This can also be used as an opportunity to detect potentially dangerous situations and bail out.

“Fail Early” uses the case of pending migrations to demonstrate. If the application is started while there are pending migrations for the production database, the results can wreck production data. Instead, an initializer detects this condition and exits by calling Ruby’s Kernel.abort.

Here’s another case where this idea is useful. It’s well-known that the Ruby-based MySQL driver included with Rails isn’t suitable for use in production. In fact, Rails will produce this warning in the log if it is in use:

WARNING: You’re using the Ruby-based MySQL library that ships with Rails. This library is not suited for production. Please install the C-based MySQL library instead (gem install mysql).

This can go unnoticed in the log. Instead, we can write a short initializer that detects this condition and aborts the application start if the production server is misconfigured.

config/initializers/check_mysql_driver.rb:

if RAILS_ENV == 'production'
  config = ActiveRecord::Base.configurations['production']
  if config['adapter'] == 'mysql' 
    ActiveRecord::Base.require_mysql
    if Mysql::VERSION.to_s.include?('-ruby')
      abort "Ruby-based MySQL driver is not suitable for production"
    end
  end
end

When the initializer above is run in a production environment that has the Ruby-based MySQL driver instead of the C-based one, startup will be aborted.

$ mongrel_rails start -e production
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with production environment...
Ruby-based MySQL driver is not suitable for production
$

You can run many other safety checks like this at startup. Since they will be run only once and not per-request, your application incurs no performance penalty by doing so.

About two months ago, we began a push to build quality documentation for Supervisor using Docbook. After this was complete, we set out to build a new web presence. Our efforts culminated at PyCon 2008 when we unveiled the new Supervisor website, built by the team at Maintainable Software.

PyCon was also very productive, with hacking on Supervisor, Repoze, and general WSGI fun. I worked on a new feature to allow Supervisor’s process group configurations to be reloaded without restarting Supervisor. Meanwhile, Chris made supervisorctl pluggable so extensions can add new commands to the Supervisor command line.

I also attended quite a few interesting talks, met some new people, enjoyed hanging out with usual the Zope and Plone folks, and overall just had a good time. PyCon 2008 was great conference and I’m looking forward to next year.

Rails for PHP Developers is a new website that’s a companion to the new book by the same name. Like the book, it’s aimed towards PHP developers who have an interest in Rails and Ruby. The website features articles that alternate between Ruby and PHP focus, so PHP developers that aren’t interested in Ruby should still find it useful.

Changes in release 0.2:

Renamed addProcessToGroup() to addProgramToGroup(). The method
now supports program definitions with numprocs > 1 and will add
all resulting processes from the program definition. It also
fixes a bug where the process config was not added to the group
properly. Requested by Roger Hoover.

Added new method log() that writes an arbitrary message to the
main supervisord log. This is useful for recording information
about your twiddling.

You can download the new version and read the updated documentation on the Supervisor Twiddler page of Maintainable Software’s open source project pages.

Last week, I purchased the RTV VEG. This device converts composite video to VGA and also has a TV tuner. The photo above shows my Commodore Plus/4 computer connected to a Samsung 17″ LCD panel through the device. A CMD FD2000, an aftermarket 3.5″ floppy drive, rounds out the setup.

Commodore monitors were never known for reliability. Even now, two of my Commodore 1084 monitors are waiting for me to repair them. With devices like the RTV VEG, modern LCD video displays can be used with the older computers. The RTV VEG is capable of converting both composite video and separated chroma/luma (S-Video) to VGA. This works for many Commodore computers like the VIC-20, TED series, and C64.

The Commodore 128 also outputs RGBI video, which is not supported by the device. The C128 requires a monitor that can be switched between composite and RGBI such as the 1902 or 1084. Kevin Krausnick has combined with the RTV VEG with another converter so that both modes can be used on a VGA monitor at the flick of a toggle switch. This is very clever and I intend to do this as well for my C128.

When switching to 80-column (VDC) mode on the C128, you also have to switch the monitor. On the old monitors, this switch is in the front. On Kevin’s setup, he has a toggle switch on his homebuilt hardware. Years ago, I modified my 1902 monitor by replacing the video mode switch with my own microcontroller to do it automatically. I intend to do something similar again with the goal of having a C128 with an LCD panel that displays both video modes and switches between them automatically.

Chuck Hagenbuch started the library as an adaptation of Spyc around six months ago. Since then, he and I have been quietly using and improving it. Along the way, we fixed many issues, added support for pecl/syck, and wrote a test suite with PHPUnit.

There are a couple of other libraries also derived from Spyc, notably the sfYaml class from the Symfony framework. Since these efforts also found and corrected issues, we incorporated as many of these fixes as we could find and added them to the test suite as we went along.

At Maintainable Software, we frequently use YAML files for configuring our custom applications because our clients tend to like the format more than the alternatives. We’ve been using Horde/Yaml successfully for quite some time so we think it should generally work well for you also.

There’s a nice tutorial on working with YAML in PHP 5 over on the new Rails for PHP Developers website. It includes everything you need to get started with Horde/Yaml.

Supervisor is a tool for managing UNIX processes. Supervisor starts arbitrary processes at its own startup and allows otherwise unprivileged users to start, stop and restart its subprocesses and view its subprocess’ logs using a command line or web interface. Other programs exist which do this, but what sets Supervisor apart is that it is written in Python and built with extension points that can be leveraged by Python developers. In this talk, we’ll look at Supervisor as a platform, and how Python programs written to run under Supervisor can use its unique capabilities.

Our talk will cover basic usage but will focus on the more advanced features we’ve added recently, such as adding custom RPC interfaces and event listeners.

PyCon 2008 will in Chicago on March 13-16th. From the list of talks, it looks like this is going to be a great conference. I am especially looking forward to pounding out some new code during the sprints following the conference on the 17-20th.

I was pleased to learn today that I am a winner in the Rails Hackfest.

The Hackfest is a contest where your ranking is primarily determined by how many of your patches get accepted into Rails core during the contest month.