Mike Naberezny – Testing

PHP Temporary Streams Oct 17, 2008

Posted by Mike Naberezny in PHP,Testing

It’s been a while since David Sklar called out to let a thousand string concatenations bloom. That discussion produced some entertaining suggestions for putting strings together such as using preg_replace and calling out to MySQL with SELECT CONCAT.

Here’s an approach that uses filesystem functions. When combined with some lesser-known PHP streams functionality, it has several practical applications.

Opening Files for Reading and Writing

There are several modes for opening a file that will allow you to seek to arbitrary positions in the file and read or write at those positions. One of the most frequently used of these modes is w+, which the tmpfile function uses automatically.

We can repeatedly write, then rewind the pointer and read:
```
$f = tmpfile();
fwrite($f, 'foo');
fwrite($f, 'bar');

rewind($f);
$contents = stream_get_contents($f);  //=> "foobar"
fclose($f);
```
When writing to the filesystem, the above provides yet another inefficient solution to David’s exercise. Now let’s take it a bit further to see how this can be useful.

In-Memory Streams

PHP 5.1 introduced two new in-memory streams: php://memory and php://temp. The php://memory stream operates entirely in memory. The php://temp stream operates in memory until it reaches a given size, then transparently switches to the filesystem.

We can modify the above example to use the php://memory stream instead of hitting the filesystem:
```
$f = fopen('php://memory', 'w+');
fwrite($f, 'foo');
fwrite($f, 'bar');

rewind($f);
$contents = stream_get_contents($f);  //=> "foobar"
fclose($f);
```
Putting a string inside a fast temporary stream can be very useful. For example, we can then attach filters to that stream.

Testing

Temporary streams are also handy for testing. There are some rather elaborate virtual file system libraries out there but many times a stream is all you need.

Zend_Log has a log handler for streams that accepts either a filename or a stream resource. We can configure it with a php://memory stream for testing:
```
$f = fopen('php://memory', 'w+');
$writer = new Zend_Log_Writer_Stream($f);
$logger = new Zend_Log($writer);
```
Assuming your well-designed application has a convenient injection point for the logger instance, your test can pass it in before your test activates some action which should result in logging:
```
$logger->crit('critical message worth testing');
```
When the action has completed, the test can rewind $f and use it as a test spy.
```
rewind($f);
$contents = stream_get_contents($f);
$this->assertRegExp('/message worth testing/i', $contents);  // PHPUnit
```
Not surprisingly, my unit tests for Zend_Log use this same technique.

Application Usage

Not long ago, we built a custom document storage system for a client. One of its more interesting features is that it integrates with an internet fax service so users can select documents in the system and then fax them. For each fax, the software automatically generates a cover page.

To make the cover page, I first made a nice template using a drawing tool and then saved it in PDF format. I then used Zend_Pdf to write over the template with dynamic content. I first demonstrated this technique in this article.

Since the cover page is only used once (during transmission) and easy to regenerate, I don’t save the output to the filesystem. Instead, I create a php://temp stream. The instance method Zend_Pdf->render() writes the PDF output only to that stream, which is then rewound. Functions like stream_copy_to_stream or fpassthru can then be used to send the final output where it needs to go, and the whole process normally never needs to use the disk.
PHP Developer Best Practices Sep 16, 2008

Posted by Mike Naberezny in PHP,Testing

Matthew Weier O’Phinney and I gave a tutorial session at ZendCon 2008 this year titled “PHP Developer Best Practices”. The tutorial touched on source control, coding standards, testing, documentation, and more.

The slides are now available in PDF format.

We were located in Hall B of the Santa Clara Convention Center, which is a very large room that’s also used for the keynotes. Andi told us the room was selected based on the number of people who registered for our session. We initially had doubts but the attendance was greater than any previous year and the room worked out quite well. We were thankful that unlike last year, everyone was able to get a seat.

Thank you to all who attended. We enjoyed meeting many of you during the breaks and hope that you found our session helpful.
OSCON 2008 Slides Jul 25, 2008

Posted by Mike Naberezny in PHP,Python,Testing

Slides from my two talks at OSCON 2008 are now available:
- Integration Testing PHP Applications
- Supervisor as a Platform
Both talks were well attended had great audience participation. Thanks to everyone that attended and I hope you enjoyed them.

About seven people came up after the Integration Testing talk with good questions and feedback. I think that 45 minutes was enough to provide good starting points for the topics covered but it was clear that these users were engaged and ready to dig into it more. If I give this talk again, it is going to be in a tutorial format so we can get much deeper into the material and have some exercises.

After my Supervisor talk at the end of the day, I had the pleasure of going out to dinner with Roger Hoover and some other Supervisor users. It’s been very exciting to see Supervisor picking up traction over the past year. Supervisor is being used in several large architectures of companies whose names you know. If you’re using Supervisor and wouldn’t mind us telling others about it, please contact me.
Speaking at OSCON 2008 Apr 2, 2008

Posted by Mike Naberezny in PHP,Python,Testing

The OSCON 2008 website has published its talk schedule. I’ll be giving two talks at OSCON this year; one on the Python track and one on the PHP track.

Supervisor as a Platform

I will quickly introduce you to Supervisor and the immediate benefits of running your server processes under it. We will then dive into how applications written specifically for Supervisor can take advantage of it as a platform — writing your own event listeners to observe Supervisor and the process lifecycle, controlling with XML-RPC, and extending the Supervisor core with your own Python extensions.

This will be an expanded version of the talk given at PyCon 2008. Since PyCon, there’s been quite a few interesting developments in Supervisor like the ability to extend supervisorctl and progress made on configuration reloading. We’ll touch on these as well, so if you attended the PyCon talk there will still be new and interesting material in this talk for you.

Integration Testing PHP Applications

While more PHP developers are recognizing the importance and benefits of unit testing, the uptake of PHP developers using automated integration or acceptance testing is relatively slow. This testing is equally crucial to maintaining the integrity of applications.

I’m going to help get you started testing at the application level with practical tips and source code. We’ll look at how to structure your HTML markup so it’s more easily testable, making tests easier to write and maintain with CSS selectors, organizing your tests, and testing with or without a browser.

I’d also suggest you check out Sebastian Bergmann’s tutorial session on PHPUnit’s integration with Selenium RC.
Keep the Flash and Test it, too. Sep 8, 2007

Posted by Mike Naberezny in Ruby,Testing

The flash in Ruby on Rails is a special hash that is stored in the session. Its contents are available to the next request and cleared out immediately after.

This makes is very useful one-time events, such as an item being added to a shopping cart:
```
flash[:success] = "#{cart.items.size} items added to your cart."
```
The flash[:success] will be available for the next request only.

However, Ajax requests will also clear the flash. This can lead to the sometimes mysterious problem of the flash contents disappearing before their intended request because an intermediate Ajax request caused them to be cleared.

The solution is to explicitly preserve the flash contents in actions that could unintentionally clear it. This is done using flash.keep:
```
flash.keep
```
The current contents of the flash will then be preserved for the next request. You can also pass a specific key such as flash.keep(:foo).

On one application I work on, we make Ajax requests periodically on a timer. Putting flash.keep in these Ajax actions was a simple way to make sure they didn’t unexpectedly gobble the flash contents.

Of course, once adding flash.keep, we also want to add a functional test for it. You might try doing this:
```
def test_something_keeps_flash
  @request.flash[:foo] = 'bar'
  xhr :get, :something
  assert_response :success
  assert_equal 'bar', @response.flash[:foo]
end
```
You’d be close but wrong. That would seem like a natural way to do it but unfortunately, the flash can’t be populated that way due to how TestRequest was built. It doesn’t have a flash attribute and the session doesn’t have a FlashHash. However, that won’t stop us from writing our test.

We could try monkeypatching but I try to avoid clever things like that. This problem is easily remedied with a helper in test_helper.rb:
```
def make_flash_hash(with_contents = {})
  returning ActionController::Flash::FlashHash.new do |flash_hash|
    flash_hash.update with_contents
  end
end
```
Our test case now can now use the simple make_flash_hash helper to populate the request’s session with flash:
```
def test_something_keeps_flash
  @request.session['flash'] = make_flash_hash(:foo => 'bar')
  xhr :get, :something
  assert_response :success
  assert_equal 'bar', @response.flash[:foo]
end
```
In addition to using @request.session as shown above, you could also populate the session in the xhr method directly. I like the former for readability.

Whichever way you choose to do it, writing the functional test will ensure that you’ll continue to preserve the flash when the application changes later.
Better PHPUnit Group Annotations Sep 4, 2007

Posted by Mike Naberezny in PHP,Testing

Last week, Sebastian Bergmann wrote about the new support for TestNG-style grouping of tests in the upcoming PHPUnit 3.2. This feature allows individual test methods to be grouped with an @group annotation.

I typically organize my test case classes into high-level groups such as unit and functional. Method-level group annotations are inconvenient for us because we’d need to annotate every method of every test case class.

I was discussing this with Sebastian and not long after he had committed changesets 1293 and 1294 to the PHPUnit repository.

Now, @group annotations on the class inherit to all the methods of that class. When all the test methods of a class belong to the same group, just annotate the class:
```
/**
 * @group functional
 */
class FooTest extends PHPUnit_Framework_TestCase {
...
```
With that one annotation to the class, a command like this one will run all the tests in the class above and any other tests annotated with @group functional:
```
$ phpunit --group functional AllTests.php
```
That’s a great shortcut. It shouldn’t take long to put @group annotations on the test case classes of a project.

Having the ability to annotate individual test methods is still very useful because test methods (and even classes) may belong to multiple groups.

Sebastian pointed out a great use case for this in the form of testing bugs that span multiple test case classes. I was already commenting my tests with bug tracker numbers, now I’ve just converted them to @group annotations.

Whenever we receive a bug report from a client, we will create tests to reproduce the bug and then fix the implementation so the tests pass. This ensures that the bug can be reproduced and that we won’t repeat the same mistake again later.

Now, the test methods associated with a bug can be annotated like @group bug42. Running phpunit --group bug42 AllTests.php will run only the tests associated with bug #42, regardless of what files and groups those test methods span.

PHPUnit 3.2 is looking great.
Faster TDD with Stakeout.rb Sep 4, 2007

Posted by Mike Naberezny in PHP,Ruby,Testing

I’m a big fan of Autotest and it runs almost constantly on my machine. Autotest automatically reruns your tests whenever your files change. Instead of constantly flipping to another shell to rerun your tests, just let Autotest cheerfully do it for you in the background. It’s highly addictive.

The only problem with Autotest is that it is specific to Ruby. I do a mix of different kinds of programming including Ruby, PHP, Python, and C. I’d like my TDD to be accelerated for all of these languages.

Thanks to Geoffrey Grossenbach, last week I came across stakeout.rb from Mike Clark. This is a tiny, dead simple Ruby script that runs an arbitrary command when certain files change. This is a stripped-down Autotest for everybody else. I’m sure it has all kinds of other uses as well.

To get started testing with stakeout.rb, you’ll need Ruby installed. Any recent version is fine and you might already have it installed. Next, grab the stakeout.rb script and add the shebang line to the top (Unix-like OS assumed):
```
#!/usr/bin/env ruby -w

if ARGV.size < 2
  puts "Usage: stakeout.rb  [files to watch]+"
...
```
Make the file executable and put it somewhere in your PATH. You can test it out by typing stakeout.rb from an arbitrary directory and you should see the help message.

Next, change over to a project directory where you have some test files. Most of the projects that I am involved with tend to use some directory structure similar to this:
```
/project_name
  /lib
  /test
  ...
```
To test such a project, run stakeout.rb from the /project_name directory. Most PHP projects using PHPUnit tend to have an AllTests.php file or equivalent to run all the tests, so we’ll assume this for the example:
```
project_name$ stakeout.rb "php test/AllTests.php" **/*
```
The first argument is what command to run when the tests change. The second argument, and any subsequent arguments, are the files to watch for changes. These can use a Ruby globbing pattern. The pattern **/* will watch all files under project_name recursively, which includes lib/ and test/.

Once stakeout.rb is run, it will show no output but will sit and wait for changes. As soon as you change a watched file, stakeout.rb will automatically rerun you tests and will continue to do so until you exit with Control-C.
Wrapping PHP Functions for Testability Aug 1, 2007

Posted by Mike Naberezny in PHP,Testing

One of the problems that hampers the testability of PHP code is the coupling created by accessing all of the PHP global functions. This happens often because a large number of useful extensions are accessed only through global functions. Consider the following code snippet:
```
$res = ldap_connect($host, $port);
if (! $res) {
  // error logging
  return false;
}
```
There are two code paths shown above: the connection succeeding, and it failing. Both of them are very difficult to test because of the coupling to the global function ldap_connect() provided by the LDAP extension.

To make it succeed, you’d need an LDAP server. Causing it to fail is easier but it could take a very long time until the connection timeout occurs. Also, the code can’t be tested at all without the LDAP extension. All of these problems are unacceptable.

The solution is to use to the extension through an object instead of calling the extension function directly. This way, we can inject either the extension wrapper or a mock object for testing.

However, writing these wrappers and maintaining them can be a pain and this is often the rationale given for not using them. There’s an easy answer to this excuse:
```
class ExtensionProxy {
  protected $ext;

  public function __construct($ext) {
    $this->ext = $ext;
  }

  public function __call($method, $args) {
    return call_user_func_array("{$this->ext}_{$method}", $args);
  }
}
```
Since most PHP extensions prefix all of their functions with the name followed by an underscore, it’s easy to wrap them with something like the class above.

There’s some performance penalty from call_user_func_array() in the above example but you can always write out a class later if that ever actually becomes a problem. Meanwhile, it can get you going very going quickly.

Our connection example then simply becomes:
```
$ldap = new ExtensionProxy('ldap');

...

$res = $ldap->connect($host, $port);
if (! $res) {
  // error logging
  return false;
}
```
The difference in usage is trivial but this version is easily testable. It now depends only on an $ldap instance, which the class needing LDAP can receive in its constructor. To test, now just pass a mock object for $ldap.

The technique of putting lightweight wrappers around PHP extension functions has been around for a long time. For example, Horde has a small wrapper around the IMAP extension for testing.

The continued improvements in PHP 5 allow for simple tricks like the ExtensionProxy above, and advances in tools like PHPUnit are making tests increasingly convenient and practical.

Whatever methods you choose, there really is no excuse for untested (or untestable) PHP code these days. I consider anything without good tests to be broken and you should also.
DRY up testing in Rails with Autotest Mar 23, 2007

Posted by Mike Naberezny in Ruby,Testing

As Rails developers, we’ve been trained hard to test early and test often. We are also acutely aware of the DRY principle (Don’t Repeat Yourself). However, these ideas don’t quite agree in Rails because in our test-code-test cycle, we’re constantly typing rake every time we need to run our tests.

Autotest will DRY up your testing by running your tests automatically whenever your files change. In this article, we’ll explore Autotest:
Installation

Autotest is a smart little program included in the ZenTest bundle of goodies. To install it, you’ll just need to install the gem for ZenTest.
```
gem install ZenTest
```
Depending on how your system is set up, you might need to run this as the root user or through sudo.

Starting Autotest

Running Autotest is as simple as running rake. First, change to the root directory of your Rails project. This is the directory that has Rakefile, app/, config/, etc. Next, run the autotest command:
```
$ autotest
```
Autotest will discover that it is running inside a Rails project and your tests will run just as they do with rake (or the wordier rake test).

After your tests run, Autotest will not exit back to the shell prompt. It will then sit and poll your files. When it notices files that change, it will run the tests for only the files that you’ve changed! It will do this continuously until you stop it.

Stopping Autotest

Pressing Control-C once will run your entire test suite again.

Pressing Control-C twice in quick succession will exit Autotest back to the shell prompt.

Autotest Plugins

Autotest includes a plugin mechanism that allows plugins to monitor different aspects of the testing lifecycle. Autotest includes a number of useful plugins out of the box.

In the next sections, we’ll see how to activate the plugins and what functionality they provide.

Coloring with RedGreen

One of the problems of testing under rake and autotest is that a lot of output can be generated and when looking at the results, you sometimes have to filter out the normal output to see the failures.

RedGreen is a simple Autotest plugin that solves this problem by coloring the summary lines of the test output either red or green to indicate whether the tests passed or failed:

Autotest automatically looks for a dotfile (.autotest) when it is started. This file may be in your Rails project directory or in your home directory where it will be used by all projects.

To install RedGreen or any other plugin, create the .autotest file with a simple require to load the plugin:
```
# .autotest

require 'autotest/redgreen'
```
That’s it! When you run autotest again, the plugin will be automatically loaded and your test output colored.

Notifications

Autotest also comes with the plugins growl, snarl, and kdenotify. Each are installed the same way as shown above, simply add the require line to your .autotest file. These allow Autotest to communicate each respective notification system.

Using one of these can be useful when running autotest in the background or in a minimized window. The screenshot above shows a Growl pop-up notification from Autotest under Mac OS X.

Note that for Autotest to send notifications to Growl, the growlnotify utility must be installed. This comes in the Extras/ directory of the Growl disk image.

Next Steps

Autotest isn’t limited to plugins shown here. There are a number of other useful plugins you can explore and more are added all the time. The plugins can be found in /path/to/your/gems/ZenTest-x.x.x/lib/autotest.

While Autotest can be an invaluable tool when testing Rails applications, it isn’t limited to Rails at all. Autotest can be used with an Ruby project that follows some simple conventions.

Visit the Autotest section of the ZenTest RDoc to learn about this, writing plugins, and more.
Best Practices of PHP Development Oct 30, 2006

Posted by Mike Naberezny in PHP,Testing

Slides from my ZendCon 2006 session are now available:
- Best Practices of PHP Development
It is a new, three hour talk that I developed and presented together with Matthew Weier O’Phinney. This is the second opportunity that Matthew and I have had to team up for a session at ZendCon and it was a great time again.

Update: You might also like to check out the updated presentation that we gave at ZendCon 2008.