Mike Naberezny

Repoze: Retooling Zope Nov 1, 2007

Posted by Mike Naberezny in Python

During the last couple of weeks, I have been privileged to help out with Repoze, a new open source Python project. Repoze is “plumbing Zope into the WSGI pipeline” but also much more.

I’m not a Zope developer but I’ve benefited from Zope indirectly in several ways. The most significant for me has been Supervisor, a process management system. It’s not part of Zope itself but it originated from a need to keep production Zope servers running smoothly. I started using it for other kinds of servers, and fast forward a couple of years and now I’m very involved and help develop it.

Supervisor is interesting to many Zope outsiders. It might have grown out of the needs of running Zope servers, but it’s a tool that many others can benefit from using. I think parts of Zope itself also have similar potential for outsiders.

Zope has been around for a long time and for many was thought to be the “killer app” for Python. Philip J. Eby wrote about the significance of Zope in Where Zope Leads, Python Follows. I’m interested in Zope not because of the application server as a whole but because it has many technologies like an object database, transactions, and object publishing that I’d like to explore.

It seems that quite a few technologies that are gaining mindshare now through new developments have been available and stable in Zope for many years. As an outsider, I want to be able to consume bits of Zope technologies for other kinds of projects.

The problem that Zope has is that its technologies aren’t accessible to outsiders in the same way as Supervisor. To the rest of us, the Zope world is largely a mysterious monolith. Projects like Grok make it easier to get started by bringing Rails-like conventions to Zope development, but many of the core Zope technologies remain largely inaccessible for new purposes without putting in a lot of effort.

Repoze is changing that in a significant way. Repoze is packaging Zope technologies into components and middleware that can be easily deployed on WSGI servers. This is a win for Zope applications because it means better deployment options like Apache and mod_wsgi. Breaking up the monolith and providing more options will help the longevity of their platform.

For the rest of us, a more modular world of Zope means that eventually, many of the interesting technologies that Zope has to offer will be closer in reach and more practical through a combination of WSGI middleware and better packaging.

Repoze already seems to have great momentum and I’ll be watching it with interest. I look forward to exploring what the Zope world has to offer through Repoze.

Update: This post was featured on Repoze Notes.
Zend/PHP Conference 2007 Sep 23, 2007

Posted by Mike Naberezny in PHP

I will be presenting again this year at the Zend/PHP Conference, together with Sebastian Bergmann and Matthew Weier-O’Phinney.

PHP development by teams of developers introduces challenges beyond the mere coding of great algorithms. Agreements are needed about tools, specifications, architecture, design patterns, coding standards, testing, documentation, source code control, staging and deployment. This tutorial session aims to introduce developers to a set of best practices that will help them deliver applications fast and move them to deployment with confidence.

We will present our full-day tutorial session, “PHP Development Best Practices”, on October 8th. I really enjoyed last year’s ZendCon and I look forward to this one.
New in Horde: Routes Sep 15, 2007

Posted by Mike Naberezny in PHP,Python

I’m pleased to announce the first release of Horde/Routes, a new URL mapping system for PHP 5. This package provides classes for mapping URLs into the controllers and actions of an MVC system, inspired by Ruby on Rails.

There are already quite a few existing libraries that do this sort of thing for PHP. Horde/Routes is a compelling alternative.

I examined most of these PHP solutions and found them all inadequate for various reasons, particularly because we wanted RESTful routing, named routes, sophisticated matching, PHP 5 E_STRICT, and extensive test coverage.

Since I do quite a bit of Ruby and Python programming, I surveyed the options in those languages and decided to do a full port of the Python library, Routes.

Horde/Routes provides these features and more:
- Supports route recognition and generation
- Sophisticated matching conditions like subdomains
- Named routes and RESTful route generation
- PEAR-style naming and coding standards
- PHP 5 E_STRICT compliant, web framework agnostic
- A comprehensive unit test suite
I decided to contribute the code to Horde’s Rampage project because Horde is one of PHP’s oldest and most successful projects. I’m using and contributing to other Horde libraries and I think Rampage is worth your attention.

The Python version has been around for some time and is very popular with different Python web frameworks. Horde/Routes is now part of that ecosystem now and has already resulted in patches being committed back to the Python version.

I’ve already used Horde_Routes on several applications. While Horde/Routes is relatively new, it is very feature-rich and well-tested.

Currently, Horde/Routes is a beta release. Over the coming weeks, we’ll be making some minor changes to the API and adding more documentation, and then it will quickly move to stable.

Update: The project now has its own pages on the Horde website and was featured on Chuck Hagenbuch’s blog.
Keep the Flash and Test it, too. Sep 8, 2007

Posted by Mike Naberezny in Ruby,Testing

The flash in Ruby on Rails is a special hash that is stored in the session. Its contents are available to the next request and cleared out immediately after.

This makes is very useful one-time events, such as an item being added to a shopping cart:
```
flash[:success] = "#{cart.items.size} items added to your cart."
```
The flash[:success] will be available for the next request only.

However, Ajax requests will also clear the flash. This can lead to the sometimes mysterious problem of the flash contents disappearing before their intended request because an intermediate Ajax request caused them to be cleared.

The solution is to explicitly preserve the flash contents in actions that could unintentionally clear it. This is done using flash.keep:
```
flash.keep
```
The current contents of the flash will then be preserved for the next request. You can also pass a specific key such as flash.keep(:foo).

On one application I work on, we make Ajax requests periodically on a timer. Putting flash.keep in these Ajax actions was a simple way to make sure they didn’t unexpectedly gobble the flash contents.

Of course, once adding flash.keep, we also want to add a functional test for it. You might try doing this:
```
def test_something_keeps_flash
  @request.flash[:foo] = 'bar'
  xhr :get, :something
  assert_response :success
  assert_equal 'bar', @response.flash[:foo]
end
```
You’d be close but wrong. That would seem like a natural way to do it but unfortunately, the flash can’t be populated that way due to how TestRequest was built. It doesn’t have a flash attribute and the session doesn’t have a FlashHash. However, that won’t stop us from writing our test.

We could try monkeypatching but I try to avoid clever things like that. This problem is easily remedied with a helper in test_helper.rb:
```
def make_flash_hash(with_contents = {})
  returning ActionController::Flash::FlashHash.new do |flash_hash|
    flash_hash.update with_contents
  end
end
```
Our test case now can now use the simple make_flash_hash helper to populate the request’s session with flash:
```
def test_something_keeps_flash
  @request.session['flash'] = make_flash_hash(:foo => 'bar')
  xhr :get, :something
  assert_response :success
  assert_equal 'bar', @response.flash[:foo]
end
```
In addition to using @request.session as shown above, you could also populate the session in the xhr method directly. I like the former for readability.

Whichever way you choose to do it, writing the functional test will ensure that you’ll continue to preserve the flash when the application changes later.
Better PHPUnit Group Annotations Sep 4, 2007

Posted by Mike Naberezny in PHP,Testing

Last week, Sebastian Bergmann wrote about the new support for TestNG-style grouping of tests in the upcoming PHPUnit 3.2. This feature allows individual test methods to be grouped with an @group annotation.

I typically organize my test case classes into high-level groups such as unit and functional. Method-level group annotations are inconvenient for us because we’d need to annotate every method of every test case class.

I was discussing this with Sebastian and not long after he had committed changesets 1293 and 1294 to the PHPUnit repository.

Now, @group annotations on the class inherit to all the methods of that class. When all the test methods of a class belong to the same group, just annotate the class:
```
/**
 * @group functional
 */
class FooTest extends PHPUnit_Framework_TestCase {
...
```
With that one annotation to the class, a command like this one will run all the tests in the class above and any other tests annotated with @group functional:
```
$ phpunit --group functional AllTests.php
```
That’s a great shortcut. It shouldn’t take long to put @group annotations on the test case classes of a project.

Having the ability to annotate individual test methods is still very useful because test methods (and even classes) may belong to multiple groups.

Sebastian pointed out a great use case for this in the form of testing bugs that span multiple test case classes. I was already commenting my tests with bug tracker numbers, now I’ve just converted them to @group annotations.

Whenever we receive a bug report from a client, we will create tests to reproduce the bug and then fix the implementation so the tests pass. This ensures that the bug can be reproduced and that we won’t repeat the same mistake again later.

Now, the test methods associated with a bug can be annotated like @group bug42. Running phpunit --group bug42 AllTests.php will run only the tests associated with bug #42, regardless of what files and groups those test methods span.

PHPUnit 3.2 is looking great.
Faster TDD with Stakeout.rb Sep 4, 2007

Posted by Mike Naberezny in PHP,Ruby,Testing

I’m a big fan of Autotest and it runs almost constantly on my machine. Autotest automatically reruns your tests whenever your files change. Instead of constantly flipping to another shell to rerun your tests, just let Autotest cheerfully do it for you in the background. It’s highly addictive.

The only problem with Autotest is that it is specific to Ruby. I do a mix of different kinds of programming including Ruby, PHP, Python, and C. I’d like my TDD to be accelerated for all of these languages.

Thanks to Geoffrey Grossenbach, last week I came across stakeout.rb from Mike Clark. This is a tiny, dead simple Ruby script that runs an arbitrary command when certain files change. This is a stripped-down Autotest for everybody else. I’m sure it has all kinds of other uses as well.

To get started testing with stakeout.rb, you’ll need Ruby installed. Any recent version is fine and you might already have it installed. Next, grab the stakeout.rb script and add the shebang line to the top (Unix-like OS assumed):
```
#!/usr/bin/env ruby -w

if ARGV.size < 2
  puts "Usage: stakeout.rb  [files to watch]+"
...
```
Make the file executable and put it somewhere in your PATH. You can test it out by typing stakeout.rb from an arbitrary directory and you should see the help message.

Next, change over to a project directory where you have some test files. Most of the projects that I am involved with tend to use some directory structure similar to this:
```
/project_name
  /lib
  /test
  ...
```
To test such a project, run stakeout.rb from the /project_name directory. Most PHP projects using PHPUnit tend to have an AllTests.php file or equivalent to run all the tests, so we’ll assume this for the example:
```
project_name$ stakeout.rb "php test/AllTests.php" **/*
```
The first argument is what command to run when the tests change. The second argument, and any subsequent arguments, are the files to watch for changes. These can use a Ruby globbing pattern. The pattern **/* will watch all files under project_name recursively, which includes lib/ and test/.

Once stakeout.rb is run, it will show no output but will sit and wait for changes. As soon as you change a watched file, stakeout.rb will automatically rerun you tests and will continue to do so until you exit with Control-C.
Wrapping PHP Functions for Testability Aug 1, 2007

Posted by Mike Naberezny in PHP,Testing

One of the problems that hampers the testability of PHP code is the coupling created by accessing all of the PHP global functions. This happens often because a large number of useful extensions are accessed only through global functions. Consider the following code snippet:
```
$res = ldap_connect($host, $port);
if (! $res) {
  // error logging
  return false;
}
```
There are two code paths shown above: the connection succeeding, and it failing. Both of them are very difficult to test because of the coupling to the global function ldap_connect() provided by the LDAP extension.

To make it succeed, you’d need an LDAP server. Causing it to fail is easier but it could take a very long time until the connection timeout occurs. Also, the code can’t be tested at all without the LDAP extension. All of these problems are unacceptable.

The solution is to use to the extension through an object instead of calling the extension function directly. This way, we can inject either the extension wrapper or a mock object for testing.

However, writing these wrappers and maintaining them can be a pain and this is often the rationale given for not using them. There’s an easy answer to this excuse:
```
class ExtensionProxy {
  protected $ext;

  public function __construct($ext) {
    $this->ext = $ext;
  }

  public function __call($method, $args) {
    return call_user_func_array("{$this->ext}_{$method}", $args);
  }
}
```
Since most PHP extensions prefix all of their functions with the name followed by an underscore, it’s easy to wrap them with something like the class above.

There’s some performance penalty from call_user_func_array() in the above example but you can always write out a class later if that ever actually becomes a problem. Meanwhile, it can get you going very going quickly.

Our connection example then simply becomes:
```
$ldap = new ExtensionProxy('ldap');

...

$res = $ldap->connect($host, $port);
if (! $res) {
  // error logging
  return false;
}
```
The difference in usage is trivial but this version is easily testable. It now depends only on an $ldap instance, which the class needing LDAP can receive in its constructor. To test, now just pass a mock object for $ldap.

The technique of putting lightweight wrappers around PHP extension functions has been around for a long time. For example, Horde has a small wrapper around the IMAP extension for testing.

The continued improvements in PHP 5 allow for simple tricks like the ExtensionProxy above, and advances in tools like PHPUnit are making tests increasingly convenient and practical.

Whatever methods you choose, there really is no excuse for untested (or untestable) PHP code these days. I consider anything without good tests to be broken and you should also.
New RubyOSA Website Apr 12, 2007

Posted by Mike Naberezny in Ruby

RubyOSA now has a new website! The new site features updated information from Laurent and I, including a new guide. Over the coming weeks, we’ll continue to expand and improve its content.

Please let us know what you think about it!

Update: This post was featured on Laurent Sansonetti’s diary.
OSA::ObjectSpecifierList#every Mar 26, 2007

Posted by Mike Naberezny in Ruby

For those of you who aren’t yet on the RubyOSA list (you should be), Laurent just committed a nice new feature to OSA::ObjectSpecifierList. It’s actually been there since last week, in the form of some method_missing hackery, but we finally decided to take that out and call it #every.

Sometimes you need to collect an attribute from every object in the specifier list. Normally, you’d do something like this:
```
names = OSA.app('iCal').calendars.collect { |c| c.name }
```
Symbol#to_proc fans use the convenient form:
```
names = OSA.app('iCal').calendars.collect(&:name)
```
Now, OSA::ObjectSpecifierList also has the #every method:
```
names = OSA.app('iCal').calendars.every(:name)
```
The difference is subtle but #every will fetch all of the attributes in a single Apple Event, something not possible when iterating over each item in the collection. For most purposes this is not important but it is a nice feature that could make a difference on larger collections.

This feature is currently in the RubyOSA trunk and will appear in the next stable release (coming soon).

Update: RubyOSA 4.0 has been released and includes this feature!
DRY up testing in Rails with Autotest Mar 23, 2007

Posted by Mike Naberezny in Ruby,Testing

As Rails developers, we’ve been trained hard to test early and test often. We are also acutely aware of the DRY principle (Don’t Repeat Yourself). However, these ideas don’t quite agree in Rails because in our test-code-test cycle, we’re constantly typing rake every time we need to run our tests.

Autotest will DRY up your testing by running your tests automatically whenever your files change. In this article, we’ll explore Autotest:
Installation

Autotest is a smart little program included in the ZenTest bundle of goodies. To install it, you’ll just need to install the gem for ZenTest.
```
gem install ZenTest
```
Depending on how your system is set up, you might need to run this as the root user or through sudo.

Starting Autotest

Running Autotest is as simple as running rake. First, change to the root directory of your Rails project. This is the directory that has Rakefile, app/, config/, etc. Next, run the autotest command:
```
$ autotest
```
Autotest will discover that it is running inside a Rails project and your tests will run just as they do with rake (or the wordier rake test).

After your tests run, Autotest will not exit back to the shell prompt. It will then sit and poll your files. When it notices files that change, it will run the tests for only the files that you’ve changed! It will do this continuously until you stop it.

Stopping Autotest

Pressing Control-C once will run your entire test suite again.

Pressing Control-C twice in quick succession will exit Autotest back to the shell prompt.

Autotest Plugins

Autotest includes a plugin mechanism that allows plugins to monitor different aspects of the testing lifecycle. Autotest includes a number of useful plugins out of the box.

In the next sections, we’ll see how to activate the plugins and what functionality they provide.

Coloring with RedGreen

One of the problems of testing under rake and autotest is that a lot of output can be generated and when looking at the results, you sometimes have to filter out the normal output to see the failures.

RedGreen is a simple Autotest plugin that solves this problem by coloring the summary lines of the test output either red or green to indicate whether the tests passed or failed:

Autotest automatically looks for a dotfile (.autotest) when it is started. This file may be in your Rails project directory or in your home directory where it will be used by all projects.

To install RedGreen or any other plugin, create the .autotest file with a simple require to load the plugin:
```
# .autotest

require 'autotest/redgreen'
```
That’s it! When you run autotest again, the plugin will be automatically loaded and your test output colored.

Notifications

Autotest also comes with the plugins growl, snarl, and kdenotify. Each are installed the same way as shown above, simply add the require line to your .autotest file. These allow Autotest to communicate each respective notification system.

Using one of these can be useful when running autotest in the background or in a minimized window. The screenshot above shows a Growl pop-up notification from Autotest under Mac OS X.

Note that for Autotest to send notifications to Growl, the growlnotify utility must be installed. This comes in the Extras/ directory of the Growl disk image.

Next Steps

Autotest isn’t limited to plugins shown here. There are a number of other useful plugins you can explore and more are added all the time. The plugins can be found in /path/to/your/gems/ZenTest-x.x.x/lib/autotest.

While Autotest can be an invaluable tool when testing Rails applications, it isn’t limited to Rails at all. Autotest can be used with an Ruby project that follows some simple conventions.

Visit the Autotest section of the ZenTest RDoc to learn about this, writing plugins, and more.

← Previous Entries

Next Entries →

Repoze: Retooling Zope Nov 1, 2007

Zend/PHP Conference 2007 Sep 23, 2007

New in Horde: Routes Sep 15, 2007

Keep the Flash and Test it, too. Sep 8, 2007

Better PHPUnit Group Annotations Sep 4, 2007

Faster TDD with Stakeout.rb Sep 4, 2007

Wrapping PHP Functions for Testability Aug 1, 2007

New RubyOSA Website Apr 12, 2007

OSA::ObjectSpecifierList#every Mar 26, 2007

DRY up testing in Rails with Autotest Mar 23, 2007

Installation

Starting Autotest

Stopping Autotest

Autotest Plugins

Coloring with RedGreen

Notifications

Next Steps

Search

Links

Contact

Categories