Embedding DSLs in Java using JRuby

Embedding DSLs in Java using JRuby

By Mario Aquino, OCI Principal Software Engineer

January 2007

Introduction

Software in the hands of a knowledgeable user should be empowering. Applications can act as the very extension of their user's will; performing tasks, solving problems, and generally being helpful. However, no matter how well-intentioned a program may be, the user interaction can either be fluid and intuitive or haphazard and clumsy. The best user interfaces are the simple ones, where the user is neither constrained nor overwhelmed by the choices to be made to properly express a need. One trick is to provide a way for the user to communicate with the system using a language the user is comfortable with, shifting the burden of learning how to respond from the user to the system. This article will describe how to introduce a mini-language of commands for interacting with a program (rather than components like radio buttons and combo boxes) using Java, the new Java 6 Scripting API, and JRuby.

JRuby

JRuby is an open-source Java implementation of the Ruby programming language. Like Java, Ruby is an object-oriented programming language (though, technically Java is not a fully OO language because primitive values like int and float are not objects, unlike in Ruby where everything is an object). Another difference between Ruby and Java is that Java is a compiled, statically-typed language while Ruby is a dynamically-typed, runtime-interpreted language (what some refer to as a "scripting" language).

The Java 6 Scripting API

Among the useful features of the recently released Java 6 Standard Edition is a new API for integrating "scripting" languages with Java applications. This API is akin to the Bean Scripting Framework, in that it allows objects to be passed to and back from non-Java language contexts running in the VM.

To interact with a scripting resource, a Java class can call the Java 6 Scripting API, which loads a scripting engine for the desired language. The engine interfaces with the language implementation.

Java 6 Scripting API

There is a long list of supported languages for the Java 6 scripting API including: Beanshell, Groovy, JavaScript, Python (via Jython), and Ruby (via JRuby). Engines for all languages supported by the Java 6 Scripting API can be downloaded from https://scripting.java.net.

Using the API is very simple. Here is some sample code to demonstrate:

  1. import java.io.InputStream;
  2. import java.io.InputStreamReader;
  3.  
  4. import javax.script.ScriptEngine;
  5. import javax.script.ScriptEngineManager;
  6. import javax.script.ScriptException;
  7.  
  8. public class ScriptingDemo {
  9.  
  10.     public static void main(String[] args) throws ScriptException {
  11.         ScriptEngineManager manager = new ScriptEngineManager();
  12.         ScriptEngine engine = manager.getEngineByName("ruby");
  13.         engine.put("message", "Hello, world!");
  14.         InputStream resource = ScriptingDemo.class.getClassLoader().getResourceAsStream("simple_demo.rb");
  15.         engine.eval(new InputStreamReader(resource));
  16.     }
  17. }

The Java code creates a ScriptEngineManager and uses it to retrieve an engine for the Ruby programming language. The script engine JAR file for Ruby as well as the JRuby JAR file must be in the classpath for this to work. The Java code then adds a key/value pair to the engine to be retrieved by the Ruby source file. Finally, the Ruby source file is loaded and executed.

Here is the Ruby code:

  1. require 'java'
  2.  
  3. def show(message)
  4.   # Passing in nil for dialog parent so it will show by itself
  5.   javax.swing.JOptionPane.showMessageDialog(nil, message)
  6. end
  7.  
  8. show($message)

This code uses a facility provided by JRuby to allow Ruby code to execute Java code. It passes the message (the value of the $message global variable set in the ScriptingDemo Java class) to the show() method, which pops up a JOptionPane.

It's worth noting that there is no parameter type checking going on in the Ruby code; the show()method accepts a parameter and passes it on to the JOptionPane.

Domain-Specific Languages (DSLs)

DSLs are a very powerful idea. A lot has been written about DSLs (see the References section for links) so this article will limit the scope of discussion to the use of DSLs as a means for facilitating user interaction. They represent a way to drive an application using terminology and concepts that originate from the problem domain. You may use DSLs everyday and not even realize it. For example, the text interface to http://local.google.com that allows you to find points of interest near you by typing "pizza near 12140 Woodcrest Executive Drive" is an example of a domain-specific language. Though there can be programming symbols and technical syntax introduced in DSLs, the most "user friendly" ones are simple and straight-forward but still very powerful.

DSLs ought to be free text, though they should have a well-defined syntax (the "L" in DSL is language, after all, and all languages have a syntax). The challenge for adding a DSL to a Java application is how interpret and then respond to the user's instruction. One approach is to write (or generate) a lexical analyzer to parse a user's input, map keywords to methods, call the methods, and then return the results to the user. Another approach is to write Ruby code that can respond to the users input directly, treating the text as a series of method calls.

DSL for Rich Text Editing

A simple rich text editor has a place to put text and controls for marking up the content. For this article, the controls for marking up the text will be a single text input field and a button to execute a command expression. The full source code for the sample editor is available for download. As well, the application itself can be run directly using Java WebStart, though you will have to have the Java 6 runtime installed (see the References section for links).

The editor needs to be able to highlight or clear the markup (remove the highlighting) of text content. The DSL should support expressions like "highlight each sentence" or "highlight the last word of the first sentence" or "clear every sentence" or variations of those combinations. With these simple expressions, we can take advantage of the Ruby parser to convert the plain text instructions from the user into method calls to retrieve the user's text and then apply markup to it.

The expression "highlight the first sentence" can be thought of as a series of chained method calls: highlight( the( first( sentence() ) ) ). Here, the sentence() method returns its value to the first()method, which uses the value returned by sentence() and does something with it, then returns some value to its caller (the the() method). One of the things we are taking advantage of here is Ruby's convenient syntax which makes the presence of parenthesis denoting method calls optional. All of the methods for determining the boundaries of what the user intends to highlight are at the end of the expression, leaving the action keyword (for highlighting or clearing the markup) at the beginning.

Java Editor

The Java code for this editor is trivial. Its responsibilities are creating two JTextPanes and arranging them in the frame. Both of these are passed to a Ruby script that will parse the input text and apply whatever markup instructions have been supplied by the user. The script is invoked in response to the "Evaluate" button being pressed.

The following is the portion of the Ruby code (from a file called swing_ui.rb) that handles calls to markup text from the uppermost textpane:

  1. require 'java'
  2. require 'text_dsl'
  3. include_class 'java.awt.Color'
  4. include_class 'java.awt.event.ActionEvent'
  5.  
  6. class UI < Text
  7.  
  8.   def initialize(component)
  9.     # replace double spaces with single spaces to simplify text parsing
  10.     text = component.text.gsub(/  /, ' ')
  11.     super(text)
  12.     @component = component
  13.     @component.text = text
  14.   end
  15.  
  16.   # expects an array of arrays of integer values representing the
  17.   # ranges of text content needing to be highlighted in Yellow
  18.   # background
  19.   def highlight(ranges)
  20.     markup(ranges, 'Yellow', Color::yellow)
  21.   end
  22.  
  23.   # expects an array of arrays of integer values representing the
  24.   # ranges of text content needing have their background color set
  25.   # to White
  26.   def clear(ranges)
  27.     markup(ranges, 'White', Color::white)
  28.   end
  29.  
  30.   private
  31.  
  32.   # ranges is expected to be an array of arrays of integer values,
  33.   # where the inner-most array has two values: the starting point
  34.   # of the text that needs to be marked up and the length of text
  35.   # to apply the markup to
  36.   # label should be a string and color is expected to be a
  37.   # java.awt.Color object
  38.   def markup(ranges, label, color)
  39.     background = com.ociweb.BackgroundAction.new(label, color)
  40.     ranges.each do |range|
  41.       @component.select(range[0], range[0] + range[1])
  42.       event = ActionEvent.new(@component, ActionEvent::ACTION_PERFORMED, 'apply markup')
  43.       background.actionPerformed(event)
  44.     end
  45.     @component.select(0,0)
  46.   end
  47. end
  48.  
  49. UI.new($textpane).instance_eval($dslpane.text) unless $textpane.text.empty? or $dslpane.text.empty?

From the JRuby point of view, this code calls require 'java' to make all the Java integration facilities available to the script. The require keyword is somewhat similar to the import keyword in Java; it is a signal to load resources from a file whose name follows the keyword. Below the 'require' directives, the code uses a JRuby mechanism, include_class, to make Color and ActionEvent Java classes "visible" for the script to use. Both the highlight() and clear()methods defer to a common method (markup()) to issue requests for marking up the background of the ranges of text supplied as a parameter to the call.

The last line of the file passes references to the main text content textpane and the DSL textpane (which it expects to find as the global variables $textpane and $dslpane) to an instance of the Ruby class defined in this file. The call to instance_eval() is where the magic happens; this method accepts a string and attempts to execute it as Ruby code. This is a very powerful capability that Ruby (and many other dynamic languages) can provide: the capacity to define and execute arbitrary code at runtime.

The remainder of the Ruby code comes from a Ruby file (text_dsl.rb) which provides the methods that parse the input content and setup arrays of integers representing the boundaries of text to which the user intends to apply markup.

  1. class DSL
  2.  
  3.   # Adds methods to the current class that return their arguments
  4.   # on to their callers. Useful for adding non-processing methods
  5.   # to the DSL that improve the readability of the syntax
  6.   # This idea is described in Jay Fields blog (http://jayfields.blogspot.com)
  7.   def self.bubble(*methods)
  8.     methods.each do |method|
  9.       define_method(method) { |args| args }
  10.     end
  11.   end
  12. end
  13.  
  14. class Text < DSL
  15.   # Adding these methods allows the DSL to be more English-like.
  16.   bubble :each, :every, :of, :the
  17.  
  18.   def initialize(text)
  19.     @text = text
  20.   end
  21.  
  22.   # This returns an array containing an element for each first item
  23.   # (word or sentence) that is found.
  24.   # Each element is an array containing two integers.
  25.   # The first is the index where the item begins
  26.   # and the second is the length of the item.
  27.   def first(sentences=nil)
  28.     end_point :first, sentences
  29.   end
  30.  
  31.   # This returns an array containing an element for each last item
  32.   # (word or sentence) that is found.
  33.   # Each element is an array containing two integers.
  34.   # The first is the index where the item begins
  35.   # and the second is the length of the item.
  36.   def last(sentences=nil)
  37.     end_point :last, sentences
  38.   end
  39.  
  40.   # called in response to a keyword not defined
  41.   # as a method for this class
  42.   def method_missing(method_id, *args)
  43.     raise "don't understand #{method_id}"
  44.   end
  45.  
  46.   # This returns an array containing an element
  47.   # for each sentence that is found.
  48.   # Each element is an array containing two integers.
  49.   # The first is the index where the item begins
  50.   # and the second is the length of the item.
  51.   # Returns an array of sentence strings.
  52.   def sentence
  53.     raise "invalid use of sentence" unless @last_token.nil?
  54.  
  55.     @last_token = @last_unit = :sentence
  56.  
  57.     arr = []
  58.     cursor = 0
  59.  
  60.     @text.split(/\.|\?|!/).each do |sentence|
  61.       sentence.lstrip! # removes leading whitespace
  62.       len = sentence.length + 1 # +1 for punctuation
  63.       arr << [cursor, len]
  64.       cursor += len
  65.       cursor += 1 # for space between sentences
  66.     end
  67.  
  68.     arr
  69.   end
  70.  
  71.   # The parameter is an array of arrays.
  72.   # The inner elements can be integer pairs (index and length)
  73.   # or words in a sentence.
  74.   # Returns an array with an element for each sentence in the text.
  75.   # Each element is an array of the words in the corresponding sentence.
  76.   def word(arg=nil)
  77.     raise "invalid use of word" unless arg
  78.     unless [nil, :sentence].include?(@last_unit)
  79.       raise "invalid use of word"
  80.     end
  81.  
  82.     @last_token = @last_unit = :word
  83.  
  84.     all_words = []
  85.  
  86.     # Start cursor at index of first word in first sentence passed in.
  87.     cursor = arg[0][0]
  88.  
  89.     arg.each do |sentence|
  90.       if sentence.instance_of? Array then
  91.         index, length = *sentence
  92.         sentence = @text[index, length]
  93.       end
  94.  
  95.       all_words << split_by_words(cursor, sentence)
  96.       cursor += sentence.length
  97.       cursor += 1 # for space between sentences
  98.     end 
  99.  
  100.     all_words
  101.   end
  102.  
  103.   private
  104.  
  105.   def split_by_words(cursor, sentence)
  106.     words = sentence.split(' ')
  107.  
  108.     # Remove punctuation from last word in sentence.
  109.     words[-1] = words[-1][0..-2]
  110.  
  111.     arr = []
  112.  
  113.     words.each do |word|
  114.       arr << [cursor, word.length]
  115.       cursor += word.length
  116.       cursor += 1 # for space between words
  117.     end
  118.  
  119.     arr
  120.   end
  121.  
  122.   def end_point(position, sentences=nil)
  123.     raise "invalid use of #{position}" unless sentences
  124.     unless [:sentence, :word].include?(@last_token)
  125.       raise "invalid use of #{position}"
  126.     end
  127.  
  128.     @last_token = position
  129.  
  130.     case @last_unit
  131.     when :sentence
  132.       [sentences.send(position)]
  133.     when :word
  134.       words = []
  135.       sentences.each { |sentence| words << sentence.send(position) }
  136.       words
  137.     else
  138.       nil # should never happen
  139.     end
  140.   end
  141. end

Summary

According to the imagination of science fiction writers, in the future you will be able to express your need to the "Computer" by raising your voice and barking out orders. If we are ever going to get there, software developers will have to shift away from things like buttons and checkboxes toward more elementary means of human-computer interaction. This article has demonstrated how to introduce an interpretive textual interface to execute the system's capabilities. While input for the sample application came from the keyboard, it wouldn't be difficult to bolt on a speech-to-text engine to convert voice requests into DSL expression input. That sounds pretty good, doesn't it?

Mario Aquino would like to thank Mark Volkmann for all his help in getting the sample application for this article working. He also thanks Jeff Brown for reviewing the article and Matz for creating a great language!

References