Apr
29
2006

Using Dot Notation for writing DSLs


"""
    This is all valid syntax in my shiny new 
    domain specific language, implemented in Python.

    >>> put("jason").into.element.named("username")
    >>> put("jason").into.element.labeled("username")      
    >>> put("jason").into.element.identified.by("username")

    ...or...    
    >>> put("jason").into.element(name="username")
    >>> put("jason").into.element(label="username")
    >>> put("jason").into.element(id="username")

    ...and don't forget...
    >>> get.button.named("submit")
    >>> click.it()


    For "advanced stuff", you could do this, too:
    >>> xpath_expr = "//table[@id='table1']//tr[4]/td[2]"
    >>> put("1234").into.element.identified.by(xpath_expr)

"""

# Here's the implementation:

# Built-in commands:
def put        (*args, **kwargs): return put
def get        (*args, **kwargs): return get
def click      (*args, **kwargs): return click

# Built-in page objects:
def element    (*args, **kwargs): pass
def field      (*args, **kwargs): pass   
def button     (*args, **kwargs): pass

# Properties of a page object:
def named      (*args, **kwargs): pass
def labeled    (*args, **kwargs): pass
def identified (*args, **kwargs): pass
def by         (*args, **kwargs): pass

# Prepositions treated as whitespace
def into       ()               : pass

# "Helper" nouns:
def it         ()               : pass


# Now let's define the "grammar"!
# Command grammar
put.__dict__        = { 'into'    :into,
                        'element' :element, 
                        'field'   :field,
                        'button'  :button}

get.__dict__        = put.__dict__                

click.__dict__      = { 'it':it } 


# Object grammar
element.__dict__    = { 'named'      :named,
                        'labeled'    :labeled,
                        'identified' :identified }
button.__dict__ = element.__dict__ 

identified.__dict__ = { 'by': by}

# Preposition grammar                            
into.__dict__       = { 'element': element}  


What does all this code mean?

I just defined a new domain specific language for scripting and testing web pages (like we need another one, I know!). And I did this without having to write a new parser or learning Backus Naur Form. If you step way back and squint your eyes, though, all that “dict = * ” business does look like BNF notation a little.

Separating tokens by dots instead of whitespace makes this pretty easy to define and parse. Also, by using dots, IDE’s are already tuned to give you a list of valid tokens in a drop-down menu when you type each “dot”. The big down side though is tripping on reserved words. For example, I would get a syntax error if I tried to write “my.dsl.is.the.coolest.in.the.world” because “is” and “in” are reserved words in Python. I don’t know if that will become a show-stopper down the road.

A slightly cool thing about my little DSL is that it is valid syntax in C#, C++, Java, Ruby, Python, JavaScript, Groovy, and Lua. (Dear LazyWeb, call me out here if I’m wrong on this point! And feel free to add to my list, too.) My DSL can now become a library for those other languages. When it comes time to add the “hard stuff” to my DSL, though, (e.g. loops, conditionals, objects, functions, etc.), I won’t have to because I automatically inherit the capabilities of the “parent” language.

The current code is just a shell, of course. Now I have to figure out how to translate it into stuff that actually does something. A simple matter of programming, right? :-)

I figured I’d post this, though, ’cause these days, Ruby fans are going ga-ga for DSLs. And they love to go on and on about how Ruby is just about the most wonderful and amazing language for creating DSLs that ever walked (or ever will walk) the face of the earth. As a Python (and not ashamed to admit it–JavaScript) developer, you can understand my knee-jerk reaction to say “But, But!!! Python can do that, too!” And with my fancy “Dot Notational Domain Specific Language Toolkit”, I now have some ammunition with which to fight back the onslaught of a million rabid Ruby/DSL fans. (Throw Lisp and Tcl in there, too, since those dudes throw around the phrase “DSL” alot, as well.)

Actually, I give major props to the Rails and Ruby communities for pushing DSLs mainstream. I didn’t really “get” LISP, Scheme, Tcl, and Rexx, etc. until recently when I realized that those languages were really tuned for creating DSLs. I also finally “got” that those languages sucked, too, because there’s an inherent “glass ceiling” to doing non-DSL development in those languages. By writing a DSL inside a general purpose language (like Python or Ruby), there’s no glass ceiling, only sky’s-the-limit goodness. At this point, I should stop, because Martin Fowler explains this aspect of DSLs way better in this post on his blog.

Now that we’re living in a post Ajax world, I need a catchy acronym for this technique. I’m tempted to call this “DDD” or “Triple D” as in — “DDD is all about using Dots for DSLs”. What do you think? Have you “Triple D’d” your app today?

  • Jason

P.S. After a little googling around, it looks like these guys “Do the ‘D’” quite well.

posted in django, python, rails, selenium by Jason Huggins

8 Comments to "Using Dot Notation for writing DSLs"

  1. Mike Williams wrote:

    Interesting stuff, Jason. It has similarities with the stuff Nat Pryce did around “Higher Order Messages” (in Ruby, though I suspect you could do similar in Python). See
    http://nat.truemesh.com/archives/000535.html

    I can see how the extra “words” could make a test more understandable – for non-programmers – but I suspect all the dots would confuse the hell out of them :-(

  2. Sam Newman wrote:

    Shucks – seems this is really flavour of the month right now – (http://www.magpiebrain.com/blog/2006/04/24/more-squiggle/)

  3. Jason Huggins wrote:

    Sam, very interesting! I hadn’t heard about squigle (or squiggle 2) until your reply here.

    So, do we have a quorum to write a manifesto or something, yet? :-)

  4. Sam Newman wrote:

    Write a manifesto? Heck, I’m just copying JMock-style use of Java as a DSL…

    Squiggle wasn’t part of some great plan – I just started using J Walnes’ old version (setter based) for a DB testing framework and reaslised it would be much more natural to use with a QueryBuilder-style API.

    As to whether or not people who used Squiggle 2 would then start writing/using business-oriented DSLs I really don’t know.

  5. Jason Huggins wrote:

    Sam, I was kidding about the manifesto bit. ;-)

  6. Efrem Lipkin wrote:

    Huh?

    To quote: “I didn’t really ‘get’ LISP, Scheme, Tcl, and Rexx, etc. until recently when I realized that those languages were really tuned for creating DSLs. I also finally “got” that those languages sucked, too, because there’s an inherent ‘lass ceiling’ to doing non-DSL development in those languages.”

    This is true of Tcl and maybe Rexx, but Lisp and Scheme (espeically with the Common Lisp Object System) are significantly more powerful than Python and Ruby and could be said to fully contain their semantics. Even Ruby’s term ‘Mixin’ comes from an ancient Lisp object system called Flavors. The problem with Lisp is that it is too powerful and too extensible for many programmers to handle. It used to have perforance problems, but I think a modern Lisp would execute at least as fast as Java.

  7. B Mahoney wrote:

    If your domain language has any characters that do not parse well in the world of Python/Ruby, e.g., domain language uses a ‘-’ which is interpreted as the minus sign, you will be entering kludgeville. (Override the sub Yuck.) Or domain has a “::” ?
    So, you may need to tweak your original domain. Just be warned.

    You do really need to think this out, else you will think of some great domain language syntax (great, as in just what is expected in the domain) which is simply impossible with the dot syntax.

    Aside from all that, the dot notation gives you some easy domain languages while letting you use all the tools of Python.

    If you have that under control, perhaps some “property” statements would make
    your class code more meaningful than just the put / get statements.

    That Java-based Squiggle language syntax is sort of sad by comparison with the Python syntax. And wouldn’t you want to be to interpret users DSL chunks of code with an exec statment? Is there any Java equivalent to Python ‘exec’ ?

  8. amix wrote:

    Really cool stuff ;)

    I will also try something similar – This gives me some fresh ideas ;-]

 
Powered by Wordpress and MySQL. Theme by openark.org