Parsing Quoted Strings in Ruby

  • Posted by Mike Naberezny in Python,Ruby

    Python has a nice module in the standard library called shlex that parses strings as a Unix shell would. Here’s a Python interpreter session demonstrating its usage:

    >>> import shlex
    >>> shlex.split('foo "bar baz" qux')
    ['foo', 'bar baz', 'qux']

    It’s useful for creating your own mini-languages or external DSLs that need to parse quoted strings like the one shown above.

    We recently built an inventory tracking application in Ruby that has a user interface for selecting search filters. To expose the same search capabilities as a web service, we created a simple query language. I was looking for an shlex equivalent in Ruby.

    It turns out that the Ruby Standard Library has a module called Shellwords:

    >> require 'shellwords'
    => true
    >> Shellwords::shellwords('foo "bar baz" qux')
    => ["foo", "bar baz", "qux"]

    Shellwords is a little less capable than shlex but handles the most common use case just fine. It’s a convenient solution for a problem that comes up too often.

    Update: Ruby 1.8.7 has added the shortcut methods Shellwords.split and String#shellsplit. Very nice.