-
Commodore SuperPET Sep 1, 2008
I’ve been spending more of my free time recently restoring vintage computer hardware. I am interested in Commodore 8-bit equipment, from the PET/CBM line through to the 64/128 home computers. I think it’s important to preserve computer history to remember the machines that got us where we are today.
Since much of the hardware I restore is over twenty-five years old, at least half of it is not working when I receive it. I try to repair everything I can when it’s practical. My Flickr photos page has daily progress of my chip-level repairs on this equipment.
Recently, I received a Commodore SuperPET computer. This is a remarkable machine that was a collaboration between Commodore and the Computer Systems Group at the University of Waterloo in Ontario, Canada. The SuperPET is a standard Commodore PET 8032 computer with an internal expansion that adds a powerful Motorola 6809 microprocessor, an additional 64K of expansion RAM, a fast 6551-based RS232 serial port, and custom Waterloo software in ROM.
The SuperPET can operate in MOS 6502 mode, where it is a Commodore PET 8032 with the extra 64K expansion and 6551 ACIA. Curiously, this 64K expansion memory is not compatible with the 8096. A switch on the side puts the SuperPET into 6809 mode, where it can run a number of disk-based Waterloo programming languages including BASIC, Pascal, APL, Fortan, and COBOL. When in 6809 mode, a menu in ROM prompts the user to select a language which is then loaded from disk.
My SuperPET seemed to work when I got it, with the 6502 mode working perfectly and 6809 mode showing the power-on menu. However, after obtaining the disk-based software, none of the Waterloo languages would run after loading. After verifying the disks were good, I suspected the 64K expansion RAM since the rest of the machine seemed to be working. Using technical information from the PET Index on 6502.org, I wrote several memory test programs to exercise the expansion RAM.
The expansion RAM is comprised of thirty-two 4116 DRAM chips. A couple of these had become loose from their sockets and my test program found that one of them had failed. I got a replacement from the pick-up counter at Jameco and installed it. Now, my SuperPET passes my expansion memory test and also boots all of the Waterloo languages. It is now fully functional and I’m exploring the Waterloo software.
One of the most interesting features of the Waterloo languages is how files are accessed. Commodore disk drives attached to the SuperPET are accessed with a filename like
disk8/1.program-namewhich selects unit 8, drive 1. The SuperPET was designed to be attached to a mainframe computer, known as the “host”, through its serial port. Accessing a file likehost/program-namewould load it from the mainframe if it was running the specialHOSTCMprogram from Waterloo.The
HOSTCMprogram was available for VM/CMS and other mainframe operating systems. Beyond that, I’ve not been able to find out much information about it. I would like to figure out the protocol and write a program so that a modern PC could be used as a host computer for the SuperPET.If you have any information on the SuperPET or have Commodore hardware you’d like to donate, please contact me.
-
Speaking at PyWorks 2008 Aug 29, 2008
PyWorks is a new conference from the folks at Python Magazine. It is being co-hosted with a PHP conference, php|works, and attendees of either conference have access to talks on both the Python and PHP tracks. Being a user of both languages, I think this a great idea and I’m looking forward to this format. I’ll be giving two talks at PyWorks 2008:
Microcontroller Simulation in Python
This talk will present Py65, my open-source simulation of a small microcomputer system based on the MOS 6502. The 6502 is a very famous microprocessor that was used in early microcomputers like the Commodore 64, but its design has stood up for over 30 years. Cores based on the 6502 are now widely used in embedded devices, with WDC estimating annual volumes in the hundreds of millions of units.
Using Python and software tools such as Py65, low-level software for embedded systems can be developed and tested much faster. This talk will discuss the design and implementation of Py65, how it and tools like it can help, and will also touch on other Python libraries that can assist with embedded development. (PyWorks Abstract)
URL Mapping with Routes
Routes is a Python package that provides a solution to the problem, “how do I map URLs to my code?” Its solution is an interesting one, and is actually a re-implementation of a feature from Ruby on Rails. Routes itself has also now been ported to PHP 5 as part of the Horde project. Routes is used by the Pylons web framework and other frameworks in Python, and is relatively easy to use as a standalone package.
The Routes method of URL dispatch is based around pattern matching rather than object publishing. For those new to Routes, we’ll have an introduction to the basic Routes concepts and how it works. We’ll also dive into the Routes internals and follow some URLs through their recognition phase, learning about how Routes does it job along the way. Web developers and framework implementers alike will gain a better understanding of Routes and how to use it effectively. (PyWorks Abstract)
-
Speaking at ZendCon 2008 Aug 22, 2008
I will be speaking again this year at the Zend/PHP Conference. I have teamed up with Matthew Weier-O’Phinney to present PHP Developer Best Practices:During this tutorial session, we will cover a number of best practices you can institute in your organization or in your personal coding toolbox, including:
- Testing Strategies
- Coding Standards
- Documentation
- Version Control
- Deployment Methodologies
We will discuss the hows and whys of each topic, and strive to cover concrete examples that you can start implementing immediately, so that you and your developers can start working more efficiently.
It has become a tradition for Matthew and I to present a tutorial session together at ZendCon. We’ve done a new one every year since the first ZendCon. I always enjoy this conference and I’m looking forward to it again this year.
Update: Slides from the presentation are now available.
-
OSCON 2008 Slides Jul 25, 2008
Slides from my two talks at OSCON 2008 are now available:
Both talks were well attended had great audience participation. Thanks to everyone that attended and I hope you enjoyed them.
About seven people came up after the Integration Testing talk with good questions and feedback. I think that 45 minutes was enough to provide good starting points for the topics covered but it was clear that these users were engaged and ready to dig into it more. If I give this talk again, it is going to be in a tutorial format so we can get much deeper into the material and have some exercises.
After my Supervisor talk at the end of the day, I had the pleasure of going out to dinner with Roger Hoover and some other Supervisor users. It’s been very exciting to see Supervisor picking up traction over the past year. Supervisor is being used in several large architectures of companies whose names you know. If you’re using Supervisor and wouldn’t mind us telling others about it, please contact me.
-
Py65: 6502 Microprocessor Simulator Jul 1, 2008
I’ve started a new Python project called Py65. It aims to provide building blocks for modeling 65xx microcomputer systems in software, with a focus on homebuilt hardware. Using simulation, embedded systems software can be developed and tested much faster.
In its current form, Py65 supports the original NMOS 6502 microprocessor from MOS Technology. The venerable 6502 and variants of it powered such famous microcomputers as the Commodore 64, game systems like the Atari 2600 and Nintendo Entertainment System (NES), as well as thousands of consumer and embedded devices today.
I haven’t released a package for Py65 yet but I’ll be doing this after some organizational changes are complete. For now, you can get it from its Subversion repository linked from the Py65 page on Ohloh.
The Python interactive interpreter itself can be used as a monitor:
>>> from py65.mpu6502 import MPU >>> mpu = MPU() >>> mpu <6502: A=00, X=00, Y=00, Flags=20, SP=ff, PC=0000> >>> mpu.memory[0x0000:0x0001] = (0x69, 0x16) #=> $0000 ADC #$16 >>> mpu.step() <6502: A=16, X=00, Y=00, Flags=20, SP=ff, PC=0002>
The simulator currently supports all legal NMOS 6502 opcodes and I have written a large unit test suite to verify its core. Most of its tests look like this one:
def test_bne_zero_set_does_not_branch(self): mpu = MPU() mpu.flags |= cpu.ZERO mpu.memory[0x0000:0x0001] = (0xD0, 0x06) #=> BNE +6 mpu.step() self.assertEquals(0x0002, mpu.pc)
Py65’s own unit tests hint at what tests for your own assembly language programs could look like. I am especially interested in unit testing my own embedded software and I will be working on an assertion vocabulary and test helpers for this.
There are a lot of possibilities for what could be done with Py65. For now, I plan to include a system for mapping virtual devices into the microprocessor’s address space, add more hooks into the simulator, and include support for additional 65xx family hardware.
Py65 does not include an assembler. If you need one, I recommend André Fachat‘s excellent xa assembler which is currently maintained by Cameron Kaiser.
-
Horde/Routes 0.3 Released Jun 15, 2008
Horde/Routes is a URL mapping system for PHP 5. It is a direct port of the Routes, a Python library that is part of the Pylons project. Horde/Routes is a standalone library designed to be integrated into any MVC framework.
This release brings in some changes from Routes 1.8. The most noticeable change is that custom actions on RESTful routes are now delimited with the forward slash (
/) instead of the semicolon (;). This was done for parity with Ruby on Rails.I have also fixed some bugs, notably that the resource route generator failed to generate routes that recognized
PUTandDELETErequests on “formatted” resources (/messages/1.xml). This fix will be merged upstream to the Python version.I am also using the Python version and I met with Ben Bangert, the author of the Python version, at PyCon 2008. Each release of Horde/Routes has resulted in patches to the Python version. It’s nice how this small ecosystem has developed around the routes concept between these projects (Ruby on Rails, Pylons, and our work in Horde).
Since Horde/Routes 0.3, the default RESTful routes generated by Horde/Routes are fully compatible with the latest version of ActiveResource. We have a new project at work that is using Horde/Routes and ActiveResource together and it works well.
-
Deploying on Phusion Passenger May 13, 2008
A very large number of PHP developers, perhaps even the majority, are building smaller web applications. These applications receive only a moderate amount of traffic and usually have a single database server, often on the same machine.
Deployment, or moving your application to production for use by real customers, is largely an afterthought for these small PHP applications. In many cases, decent PHP code can just be installed on the server and it runs without much trouble.
One of the points we stress towards the back of our book is that deploying Rails applications can be more difficult than deploying their PHP counterparts. There are more moving parts and things to you’ll need to learn. Until you get the hang of it, deploying even small Rails applications can be frustrating.
A recent development has greatly improved this situation.
Introducing Phusion Passenger
PHP can be deployed using a variety of server configurations. Some of these can be just as frustrating as traditional Rails deployments. However, the majority of PHP applications are still deployed on Apache using
mod_php. PHP’s tight integration with Apache is simple, proven, and just works for many needs.Phusion is a small company in the Netherlands that recently released an open source product called Passenger (
mod_rails). Passenger aims to take the complexity out of deploying Rails applications by also integrating with Apache.Installing Passenger on the average Linux or Mac server is usually simple. It is installed as a gem and contains an automated installer program that compiles and installs the necessary components. Once installed, many Rails applications can be deployed under Apache simply by configuring a VirtualHost for each application. The installer even outputs an example for you to copy and paste.
Passenger gives a deployment experience closer to what we’ve come to appreciate with PHP. In the background, there’s still more moving parts, but Passenger automatically manages them. It spawns Rails application server processes, proxies to them, and largely eliminates the configuration and glue that other Rails deployment options leave up to you. Passenger has nice documentation for when some configuration is necessary.
At work, we develop applications in both PHP and Ruby. We’ve been testing Passenger for a few weeks on a dedicated server and recently deployed an application on it. We have been very impressed with its reliability, performance, and easy-of-use compared to previous Rails deployment options. You should give Passenger strong consideration, especially if you’re just starting out.
Finally, Rails on Shared Hosting
Traditionally, deploying Rails applications on shared hosts (usually with FastCGI) has been difficult and unreliable. Due to this, many Rails applications are deployed on small VPS plans from providers like Rimuhosting and Slicehost. These run well, typically using small Mongrel cluster behind a frontend proxy balancer. However, VPS plans are usually more expensive and more work to set up and maintain than shared hosting.
Passenger makes running Rails applications on shared hosting much better. Dreamhost, a major shared hosting provider, recently announced full support for Passenger. We tested a small Rails application on Dreamhost using it and we were impressed. It seemed to run without issues and with reasonable performance.
Deploying some Rails applications on shared hosting may soon become commonplace.
Thinking Ahead
Will better Rails deployment options or Rails on shared hosting mean it’s time to switch from PHP? No. It’s unlikely this blog will switch from WordPress anytime soon and Phusion’s own blog even runs on WordPress. There are many great PHP applications out there. More importantly, sometimes PHP will simply be a more appropriate solution.
PHP, Ruby, and Rails are all great tools you can choose from. You’ll still need to learn these tools, their relative strengths, and when it’s appropriate to apply each.
Passenger just means Rails deployment is getting simpler and more mature. It also means deploying PHP and Rails applications together on the same Apache instance is now much easier. These are more reasons to consider adding Rails to your toolbox. It’s certainly a great time to be building web applications.
-
Parsing Quoted Strings in Ruby Apr 28, 2008
Python has a nice module in the standard library called shlex that parses strings as a Unix shell would. Here’s a Python interpreter session demonstrating its usage:
>>> import shlex >>> shlex.split('foo "bar baz" qux') ['foo', 'bar baz', 'qux']It’s useful for creating your own mini-languages or external DSLs that need to parse quoted strings like the one shown above.
We recently built an inventory tracking application in Ruby that has a user interface for selecting search filters. To expose the same search capabilities as a web service, we created a simple query language. I was looking for an
shlexequivalent in Ruby.It turns out that the Ruby Standard Library has a module called Shellwords:
>> require 'shellwords' => true >> Shellwords::shellwords('foo "bar baz" qux') => ["foo", "bar baz", "qux"]Shellwordsis a little less capable thanshlexbut handles the most common use case just fine. It’s a convenient solution for a problem that comes up too often.Update: Ruby 1.8.7 has added the shortcut methods
Shellwords.splitandString#shellsplit. Very nice. -
Speaking at OSCON 2008 Apr 2, 2008
The OSCON 2008 website has published its talk schedule. I’ll be giving two talks at OSCON this year; one on the Python track and one on the PHP track.Supervisor as a Platform
I will quickly introduce you to Supervisor and the immediate benefits of running your server processes under it. We will then dive into how applications written specifically for Supervisor can take advantage of it as a platform — writing your own event listeners to observe Supervisor and the process lifecycle, controlling with XML-RPC, and extending the Supervisor core with your own Python extensions.
This will be an expanded version of the talk given at PyCon 2008. Since PyCon, there’s been quite a few interesting developments in Supervisor like the ability to extend
supervisorctland progress made on configuration reloading. We’ll touch on these as well, so if you attended the PyCon talk there will still be new and interesting material in this talk for you.Integration Testing PHP Applications
While more PHP developers are recognizing the importance and benefits of unit testing, the uptake of PHP developers using automated integration or acceptance testing is relatively slow. This testing is equally crucial to maintaining the integrity of applications.
I’m going to help get you started testing at the application level with practical tips and source code. We’ll look at how to structure your HTML markup so it’s more easily testable, making tests easier to write and maintain with CSS selectors, organizing your tests, and testing with or without a browser.
I’d also suggest you check out Sebastian Bergmann’s tutorial session on PHPUnit’s integration with Selenium RC.
-
Fail Early Mar 25, 2008
I’m pleased to have been able to contribute a recipe to Mike Clark’s new book, Advanced Rails Recipes. The concept presented in my recipe, “Fail Early”, is that you can use initializers to prevent your application from starting up under certain conditions.
Rails applications typically run under persistent application server processes, like
mongrelorthin. When a Rails application starts, it goes through a startup procedure that is performed only once. The startup includes reading the environment configuration files and running any initializers that have been set up. This can also be used as an opportunity to detect potentially dangerous situations and bail out.“Fail Early” uses the case of pending migrations to demonstrate. If the application is started while there are pending migrations for the production database, the results can wreck production data. Instead, an initializer detects this condition and exits by calling Ruby’s
Kernel.abort.Here’s another case where this idea is useful. It’s well-known that the Ruby-based MySQL driver included with Rails isn’t suitable for use in production. In fact, Rails will produce this warning in the log if it is in use:
WARNING: You’re using the Ruby-based MySQL library that ships with Rails. This library is not suited for production. Please install the C-based MySQL library instead (gem install mysql).
This can go unnoticed in the log. Instead, we can write a short initializer that detects this condition and aborts the application start if the production server is misconfigured.
config/initializers/check_mysql_driver.rb:
if RAILS_ENV == 'production' config = ActiveRecord::Base.configurations['production'] if config['adapter'] == 'mysql' ActiveRecord::Base.require_mysql if Mysql::VERSION.to_s.include?('-ruby') abort "Ruby-based MySQL driver is not suitable for production" end end endWhen the initializer above is run in a production environment that has the Ruby-based MySQL driver instead of the C-based one, startup will be aborted.
$ mongrel_rails start -e production ** Starting Mongrel listening at 0.0.0.0:3000 ** Starting Rails with production environment... Ruby-based MySQL driver is not suitable for production $
You can run many other safety checks like this at startup. Since they will be run only once and not per-request, your application incurs no performance penalty by doing so.
