Getting Groovy with Google Home

Getting Groovy with Google Home 

By Ryan Vanderwerf, OCI Software Engineer

SEPTEMBER 2017

Introduction

Google Home is Google’s entry into the smart home personal assistant market. Google has made the strongest competitor to Amazon’s Alexa to date. It features an excellent built-in system of conversation interaction that is arguably more advanced than Amazon’s. With built-in smart home and Chromecast integration, it’s a pretty powerful device that can help automate your home or office.

In this guide, you are going to learn how to create a Grails application to host a Google Action for your Google Home device to interact with. We will build, deploy, and set up everything necessary to facilitate this on Google App Engine Flexible and Google APIs.

We will be using the early access unofficial Java SDK to build actions. This is because the only official SDK is built with node. The first part will mostly consist of the Grails guide currently at http://guides.grails.org.

The second part of the article covers making actions that interface to Google’s API.AI service and Grails. We will use a Grails plugin to help us accomplish this through their Webhooks and Data Service API.

Please keep in mind that these APIs worked at the time of this writing – however, Google is famous for changing things at any time that may break functionality and force you, the developer, to update things to keep up! Keep in touch on Twitter or the Grails or Groovy slack communities if you encounter any of these issues, and I’ll do my best to keep things current or help you out.

What You Need

  • Some time on your hands
  • A decent text editor or IDE such as IntelliJ
  • A Google account
  • A Google Cloud account
  • curl command to help debugging
  • Java JDK 1.8 or better
  • Recent Grails SDK (install via SDKman or manually)

How to Complete the Samples

To get started do the following:

or

The Grails guides repositories contain two folders:

  1. initial. Initial project. Often a simple Grails app with some additional code to give you a head-start.
  2. complete. A completed example. The result of working through the steps presented by the guide and applying those changes to the initial folder.

To complete the guide, go to the initial folder

  • cd into grails-guides/grails-google-home/initial

and follow the instructions in the next sections.

Note: You can go right to the completed example if you cd into grails-guides/grails-google-home/complete

The initial project is a Grails application built with the web profile where we removed the asset-pipeline and hibernate dependencies.

Although you can go right to the completed example, in order to deploy the app you need to complete several configuration steps in Google Cloud Settings.

You need to edit the file src/main/resources/action.json to point to your Google App Engine Flexible deployment urls.

Google Cloud Settings

Sign up for Google Cloud Platform and create a new project:

Install Cloud SDK for your operating system.
After you have installed the SDK, run the init command in your terminal:

$ gcloud init

It will prompt you to select the Google account and the project that you want to use.

Install gactions CLI

We will use gactions CLI to test our app.

Go to gactions CLI and download it:

Make the gactions binary executable (chmod +x on linux and OSX).

After you have installed gactions, run the init command in your terminal:

$ gactions init --force

Other commands you can run: preview, update, deploy, test, and list.

Configure Google Actions

Enable Google Actions API

Go to Google API Manager Console and enable Google Actions API.

Import your Project to the Google Actions Console

Go to the new Google Actions Console and import the project your previously created:

Configure your project ID in action.json

action.json is a metadata file which we use to inform the Google Cloud Actions Console about the actions supported by our application.

Here you can also control:

  • sample queries to help Google understand which action to call
  • intent to call

src/main/resources/action.json

  1. {
  2.  "actions": [
  3.    {
  4.      "description": "Default Welcome Intent",
  5.      "name": "MAIN",
  6.      "fulfillment": {
  7.        "conversationName": "color-finder-echo"
  8.      },
  9.      "intent": {
  10.        "name": "actions.intent.MAIN"
  11.      }
  12.  
  13.    },
  14.    {
  15.      "description": "Deep link that finds brighter colors",
  16.      "name":"color.intent",
  17.      "fulfillment": {
  18.        "conversationName": "color.intent"
  19.      },
  20.      "intent": {
  21.        "name": "color.intent",
  22.        "parameters": [{
  23.          "name": "color",
  24.          "type": "SchemaOrg_Color"
  25.        }],
  26.        "trigger": {
  27.          "queryPatterns": [
  28.            "find a brighter color for $SchemaOrg_Color:color"
  29.          ]
  30.        }
  31.      }
  32.    }
  33.  ],
  34.  "conversations": {
  35.    "color-finder-echo": {
  36.      "name": "color-finder-echo",
  37.      "url": "https://grails-color-finder.appspot.com/assistantAction/index"
  38.    },
  39.    "color.intent": {
  40.      "name": "color.intent",
  41.      "url": "https://grails-color-finder.appspot.com/assistantAction/color"
  42.    }
  43.  }
  44. }

As shown above, our app supports two actions. We explain those actions with more detail in the next sections of this guide.

We will need to modify the httpExecution items to match your deployed application endpoints.

src/main/resources/action.json

"url": "https://PROJECT_ID.appspot.com/assistantAction/index"

Add Actions to your assistant app

The first time you select a project in the Google Cloud Actions Console, you will need to supply the action package JSON via the gactions tool like so:

gactions update --action_package src/main/resources/action.json --project PROJECT_ID

Follow the instructions prompted by the above command. It involves an authorization process.

At the end you will see an output similar to:

Your app for the Assistant for project PROJECT_ID was successfully updated with your actions. Visit the Actions on Google console to finish registering your app and submit it for review at https://console.actions.google.com/project/PROJECT_ID/overview

Add App Information

After you actions package has been validated, you will be able to enter directory information such as images, privacy policy, and descriptions. Here you can also pick the voice of the speaker in your skill.

The previous screenshots display two buttons. Test and Submit. We will click theTestin the previous screenshots once we have deployed our app to Google App Engine Flexible and we are ready to test our actions.

Google App Engine 

We are going to deploy the Grails application developed in this guide to the Google App Engine Flexible Environment:

App Engine allows developers to focus on doing what they do best: writing code. Based on Google Compute Engine, the App Engine flexible environment automatically scales your app up and down while balancing the load. Microservices, authorization, SQL and NoSQL databases, traffic splitting, logging, versioning, security scanning, and content delivery networks are all supported natively.

Run the command:

$ gcloud app create

to initialize an App Engine application within the current Google Cloud project.

Writing The Application

Handle Request from Google in Grails

We name our app color finder. We want to achieve the scenario pictured below:

First, we are going to add a dependency to Google Actions Java SDK

/build.gradle

compile 'com.frogermcs.gactions:gactions:0.1.1'

The illustrated conversation is managed with the handlers. We instantiate the handlers with factories. Add the next classes to your app:

/src/main/groovy/demo/MainRequestHandlerFactory.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.api.RequestHandler
  4. import com.frogermcs.gactions.api.request.RootRequest
  5. import groovy.transform.CompileStatic
  6.  
  7. @CompileStatic
  8. class MainRequestHandlerFactory extends RequestHandler.Factory {
  9.     @Override
  10.     RequestHandler create(RootRequest rootRequest) {
  11.         new MainRequestHandler(rootRequest)
  12.     }
  13. }

/src/main/groovy/demo/MainRequestHandler.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.ResponseBuilder
  4. import com.frogermcs.gactions.api.RequestHandler
  5. import com.frogermcs.gactions.api.request.RootRequest
  6. import com.frogermcs.gactions.api.response.RootResponse
  7. import groovy.transform.CompileStatic
  8.  
  9. @CompileStatic
  10. class MainRequestHandler extends RequestHandler {
  11.     protected MainRequestHandler(RootRequest rootRequest) {
  12.         super(rootRequest)
  13.     }
  14.  
  15.     @Override
  16.     RootResponse getResponse() {
  17.         ResponseBuilder.askResponse('Hey, it works! Now tell something so I could repeat it.')
  18.     }
  19. }

/src/main/groovy/demo/MainRequestHandler.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.ResponseBuilder
  4. import com.frogermcs.gactions.api.RequestHandler
  5. import com.frogermcs.gactions.api.request.RootRequest
  6. import com.frogermcs.gactions.api.response.RootResponse
  7. import groovy.transform.CompileStatic
  8.  
  9. @CompileStatic
  10. class MainRequestHandler extends RequestHandler {
  11.     protected MainRequestHandler(RootRequest rootRequest) {
  12.         super(rootRequest)
  13.     }
  14.  
  15.     @Override
  16.     RootResponse getResponse() {
  17.         ResponseBuilder.askResponse('Hey, it works! Now tell something so I could repeat it.')
  18.     }
  19. }

/src/main/groovy/demo/TextRequestHandlerFactory.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.api.RequestHandler
  4. import com.frogermcs.gactions.api.request.RootRequest
  5. import groovy.transform.CompileStatic
  6.  
  7. @CompileStatic
  8. class TextRequestHandlerFactory extends RequestHandler.Factory {
  9.     @Override
  10.     RequestHandler create(RootRequest rootRequest) {
  11.         new TextRequestHandler(rootRequest)
  12.     }
  13. }

/src/main/groovy/demo/TextRequestHandler.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.ResponseBuilder
  4. import com.frogermcs.gactions.api.RequestHandler
  5. import com.frogermcs.gactions.api.request.RootRequest
  6. import com.frogermcs.gactions.api.response.RootResponse
  7. import groovy.transform.CompileStatic
  8.  
  9. @CompileStatic
  10. class TextRequestHandler extends RequestHandler {
  11.     protected TextRequestHandler(RootRequest rootRequest) {
  12.         super(rootRequest)
  13.     }
  14.  
  15.     @Override
  16.     RootResponse getResponse() {
  17.         ResponseBuilder.tellResponse("You just told: ${rootRequest.inputs.get(0).raw_inputs.get(0).query}")
  18.     }
  19. }

We are going to handle the request that comes from Google in a Grails Controller:

A controller fulfills the C in the Model View Controller (MVC) pattern and is responsible for handling web requests. In Grails, a controller is a class with a name that ends in the convention "Controller" and lives in the grails-app/controllers directory.

The Grails Controller, in turn, will delegate to Actions SDK.

/grails-app/controllers/demo/AssistantActionController.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.AssistantActions
  4. import com.frogermcs.gactions.api.StandardIntents
  5. import com.frogermcs.gactions.api.request.RootRequest
  6. import com.google.gson.Gson
  7. import com.google.gson.stream.JsonReader
  8. import javax.servlet.http.HttpServletRequest
  9. import groovy.transform.CompileStatic
  10.  
  11. @CompileStatic
  12. class AssistantActionController {
  13.     def index() {
  14.         AssistantActions assistantActions =
  15.                 new AssistantActions.Builder(new GrailsResponseHandler(response))
  16.                         .addRequestHandlerFactory(StandardIntents.MAIN, new MainRequestHandlerFactory())  
  17.                         .addRequestHandlerFactory(StandardIntents.TEXT, new TextRequestHandlerFactory())
  18.                         .build()
  19.         RootRequest rootRequest = parseActionRequest(request)
  20.         assistantActions.handleRequest(rootRequest) 
  21.         null 
  22.     }
  23.     private RootRequest parseActionRequest(HttpServletRequest request) throws IOException {
  24.         JsonReader jsonReader = new JsonReader(request.reader)
  25.         new Gson().fromJson(jsonReader, RootRequest)
  26.     }

We can chain handlers.

Grails action delegates to Actions SDK to handle requests that come from Google.

We do not want to return a grails view. The handlers manage the response.

Add GrailsResponseHandler to your project. As illustrated above, it allows us to use a controller to process the incoming requests.

/src/main/groovy/demo/GrailsResponseHandler.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.ResponseHandler
  4. import com.frogermcs.gactions.api.response.RootResponse
  5. import com.google.gson.Gson
  6. import groovy.transform.CompileStatic
  7. import javax.servlet.http.HttpServletResponse
  8. import groovy.util.logging.Slf4j
  9.  
  10. /**
  11.  * This is a handler for the SDK to make it work with Grails properly
  12.  */
  13. @CompileStatic
  14. @Slf4j
  15. class GrailsResponseHandler implements ResponseHandler {
  16.     private final HttpServletResponse httpServletResponse
  17.     private final Gson gson
  18.  
  19.     GrailsResponseHandler(HttpServletResponse httpServletResponse) {
  20.         this(httpServletResponse, new Gson())
  21.     }
  22.  
  23.     GrailsResponseHandler(HttpServletResponse httpServletResponse, Gson gson) {
  24.         this.httpServletResponse = httpServletResponse
  25.         this.gson = gson
  26.     }
  27.  
  28.     @Override
  29.     void onPrepareContentType(String contentType) {
  30.         httpServletResponse.setContentType(contentType)
  31.     }
  32.  
  33.     @Override
  34.     void onPrepareResponseHeaders(Map<String, String> headers) {
  35.         for (String headerName : headers.keySet()) {
  36.             httpServletResponse.addHeader(headerName, headers.get(headerName))
  37.         }
  38.     }
  39.  
  40.     @Override
  41.     void onResponse(RootResponse rootResponse) {
  42.         try {
  43.             gson.toJson(rootResponse, httpServletResponse.writer)
  44.             httpServletResponse.flushBuffer()
  45.         } catch (IOException e) {
  46.             log.error('Error writing response', e)
  47.         }
  48.     }
  49.  
  50.     String getResponse(RootResponse rootResponse) {
  51.  
  52.         try {
  53.             gson.toJson(rootResponse)
  54.         } catch (IOException e) {
  55.             log.error('Error getting response', e)
  56.         }
  57.     }
  58. }

ColorHandler

To honor our app’s name we want to handle the next scenario as well:

scenarioB

Similar to the previous example, we need a handler and a factory to instantiate the handler.

This handler demonstrates passing a verbal parameter into the intent. When the user asks for a brighter color for X, it will try to find a brighter color with a name and say it to the user.

/src/main/groovy/demo/ColorRequestHandlerFactory.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.api.RequestHandler
  4. import com.frogermcs.gactions.api.request.RootRequest
  5. import groovy.transform.CompileStatic
  6.  
  7. @CompileStatic
  8. class ColorRequestHandlerFactory extends RequestHandler.Factory {
  9.     @Override
  10.     RequestHandler create(RootRequest rootRequest) {
  11.         new ColorRequestHandler(rootRequest)
  12.     }
  13.  
  14. }
  15. /src/main/groovy/demo/ColorRequestHandler.groovy
  16. package demo
  17.  
  18. import com.frogermcs.gactions.ResponseBuilder
  19. import com.frogermcs.gactions.api.RequestHandler
  20. import com.frogermcs.gactions.api.request.RootRequest
  21. import com.frogermcs.gactions.api.response.RootResponse
  22. import demo.util.ColorUtils
  23. import java.awt.Color
  24. import java.lang.reflect.Field
  25. import groovy.transform.CompileStatic
  26. import groovy.util.logging.Slf4j
  27.  
  28. /**
  29.  * take a color name input and returns a brighter color name in return if possible
  30.  */
  31. @CompileStatic
  32. @Slf4j
  33. class ColorRequestHandler extends RequestHandler {
  34.  
  35.     protected ColorRequestHandler(RootRequest rootRequest) {
  36.         super(rootRequest)
  37.     }
  38.  
  39.     @Override
  40.     RootResponse getResponse() {
  41.         log.debug("Inputs=${rootRequest.inputs.toListString()}")
  42.         String color = rootRequest.inputs[0].arguments[0].raw_text.toLowerCase()
  43.  
  44.         Color parsedColor = null
  45.         try {
  46.             Field field = Class.forName('java.awt.Color').getField(color)
  47.             parsedColor = (Color)field.get(null)
  48.         } catch (NoSuchFieldException ne) {
  49.             return colorNotFound(color)
  50.         }
  51.         if (parsedColor) {
  52.             ColorUtils colorUtils = new ColorUtils()
  53.             String brighter = colorUtils.findBrighterNameForColor(parsedColor).toLowerCase()
  54.             String answer = "Sorry I can't find a brighter color for ${color}."
  55.             if (brighter != color) {
  56.                 answer = "The brighter color for ${color} is ${brighter} "
  57.             }
  58.             return ResponseBuilder.tellResponse(answer)
  59.         }
  60.         colorNotFound(color)
  61.     }
  62.  
  63.     private RootResponse colorNotFound(String color) {
  64.         ResponseBuilder.tellResponse("Sorry I don't understand the color ${color}.")
  65.     }
  66. }

We are going to handle the request that comes from Google in another action of our controller:

/grails-app/controllers/demo/AssistantActionController.groovy

  1. package demo
  2.  
  3. import com.frogermcs.gactions.AssistantActions
  4. import com.frogermcs.gactions.api.StandardIntents
  5. import com.frogermcs.gactions.api.request.RootRequest
  6. import com.google.gson.Gson
  7. import com.google.gson.stream.JsonReader
  8. import javax.servlet.http.HttpServletRequest
  9. import groovy.transform.CompileStatic
  10.  
  11. @CompileStatic
  12. class AssistantActionController {
  13.     def color() {
  14.         AssistantActions assistantActions =
  15.                 new AssistantActions.Builder(new GrailsResponseHandler(response))
  16.                         .addRequestHandlerFactory('color.intent', new ColorRequestHandlerFactory()) 
  17.                         .build()
  18.         RootRequest rootRequest = parseActionRequest(request)  
  19.         assistantActions.handleRequest(rootRequest)
  20.         null  
  21.     }
  22.     private RootRequest parseActionRequest(HttpServletRequest request) throws IOException {
  23.         JsonReader jsonReader = new JsonReader(request.reader)
  24.         new Gson().fromJson(jsonReader, RootRequest)
  25.     }

Instantiate the handler with the help of the factory

Grails action delegates to Actions SDK to handle requests that come from Google.

We do not want to return a Grails view. The handlers manage the response.

Let’s use a simple utility class to make our action do something interesting. This will help us map a given java.awt.Color to give us an English name for it.

/src/main/groovy/demo/util/ColorUtils.java

  1. package demo.util;
  2.  
  3. import java.awt.*;
  4. import java.util.ArrayList;
  5.  
  6. /**
  7.  * Java Code to get a color name from rgb/hex value/awt color
  8.  *
  9.  * The part of looking up a color name from the rgb values is edited from
  10.  * https://gist.github.com/nightlark/6482130#file-gistfile1-java (that has some errors) by Ryan Mast (nightlark)
  11.  *
  12.  * @author Xiaoxiao Li
  13.  *
  14.  */
  15. public class ColorUtils {
  16.  
  17.     /**
  18.      * Initialize the color list that we have.
  19.      */
  20.     private ArrayList<ColorName> initColorList() {
  21.         ArrayList<ColorName> colorList = new ArrayList<ColorName>();
  22.         colorList.add(new ColorName("AliceBlue", 0xF0, 0xF8, 0xFF));
  23.         colorList.add(new ColorName("AntiqueWhite", 0xFA, 0xEB, 0xD7));
  24.         colorList.add(new ColorName("Aqua", 0x00, 0xFF, 0xFF));
  25.         colorList.add(new ColorName("Aquamarine", 0x7F, 0xFF, 0xD4));
  26.         colorList.add(new ColorName("Azure", 0xF0, 0xFF, 0xFF));
  27.         colorList.add(new ColorName("Beige", 0xF5, 0xF5, 0xDC));
  28.         colorList.add(new ColorName("Bisque", 0xFF, 0xE4, 0xC4));
  29.         colorList.add(new ColorName("Black", 0x00, 0x00, 0x00));
  30.         colorList.add(new ColorName("BlanchedAlmond", 0xFF, 0xEB, 0xCD));
  31.         colorList.add(new ColorName("Blue", 0x00, 0x00, 0xFF));
  32.         colorList.add(new ColorName("BlueViolet", 0x8A, 0x2B, 0xE2));
  33.         colorList.add(new ColorName("Brown", 0xA5, 0x2A, 0x2A));
  34.         colorList.add(new ColorName("BurlyWood", 0xDE, 0xB8, 0x87));
  35.         colorList.add(new ColorName("CadetBlue", 0x5F, 0x9E, 0xA0));
  36.         colorList.add(new ColorName("Chartreuse", 0x7F, 0xFF, 0x00));
  37.         colorList.add(new ColorName("Chocolate", 0xD2, 0x69, 0x1E));
  38.         colorList.add(new ColorName("Coral", 0xFF, 0x7F, 0x50));
  39.         colorList.add(new ColorName("CornflowerBlue", 0x64, 0x95, 0xED));
  40.         colorList.add(new ColorName("Cornsilk", 0xFF, 0xF8, 0xDC));
  41.         colorList.add(new ColorName("Crimson", 0xDC, 0x14, 0x3C));
  42.         colorList.add(new ColorName("Cyan", 0x00, 0xFF, 0xFF));
  43.         colorList.add(new ColorName("DarkBlue", 0x00, 0x00, 0x8B));
  44.         colorList.add(new ColorName("DarkCyan", 0x00, 0x8B, 0x8B));
  45.         colorList.add(new ColorName("DarkGoldenRod", 0xB8, 0x86, 0x0B));
  46.         colorList.add(new ColorName("DarkGray", 0xA9, 0xA9, 0xA9));
  47.         colorList.add(new ColorName("DarkGreen", 0x00, 0x64, 0x00));
  48.         colorList.add(new ColorName("DarkKhaki", 0xBD, 0xB7, 0x6B));
  49.         colorList.add(new ColorName("DarkMagenta", 0x8B, 0x00, 0x8B));
  50.         colorList.add(new ColorName("DarkOliveGreen", 0x55, 0x6B, 0x2F));
  51.         colorList.add(new ColorName("DarkOrange", 0xFF, 0x8C, 0x00));
  52.         colorList.add(new ColorName("DarkOrchid", 0x99, 0x32, 0xCC));
  53.         colorList.add(new ColorName("DarkRed", 0x8B, 0x00, 0x00));
  54.         colorList.add(new ColorName("DarkSalmon", 0xE9, 0x96, 0x7A));
  55.         colorList.add(new ColorName("DarkSeaGreen", 0x8F, 0xBC, 0x8F));
  56.         colorList.add(new ColorName("DarkSlateBlue", 0x48, 0x3D, 0x8B));
  57.         colorList.add(new ColorName("DarkSlateGray", 0x2F, 0x4F, 0x4F));
  58.         colorList.add(new ColorName("DarkTurquoise", 0x00, 0xCE, 0xD1));
  59.         colorList.add(new ColorName("DarkViolet", 0x94, 0x00, 0xD3));
  60.         colorList.add(new ColorName("DeepPink", 0xFF, 0x14, 0x93));
  61.         colorList.add(new ColorName("DeepSkyBlue", 0x00, 0xBF, 0xFF));
  62.         colorList.add(new ColorName("DimGray", 0x69, 0x69, 0x69));
  63.         colorList.add(new ColorName("DodgerBlue", 0x1E, 0x90, 0xFF));
  64.         colorList.add(new ColorName("FireBrick", 0xB2, 0x22, 0x22));
  65.         colorList.add(new ColorName("FloralWhite", 0xFF, 0xFA, 0xF0));
  66.         colorList.add(new ColorName("ForestGreen", 0x22, 0x8B, 0x22));
  67.         colorList.add(new ColorName("Fuchsia", 0xFF, 0x00, 0xFF));
  68.         colorList.add(new ColorName("Gainsboro", 0xDC, 0xDC, 0xDC));
  69.         colorList.add(new ColorName("GhostWhite", 0xF8, 0xF8, 0xFF));
  70.         colorList.add(new ColorName("Gold", 0xFF, 0xD7, 0x00));
  71.         colorList.add(new ColorName("GoldenRod", 0xDA, 0xA5, 0x20));
  72.         colorList.add(new ColorName("Gray", 0x80, 0x80, 0x80));
  73.         colorList.add(new ColorName("Green", 0x00, 0x80, 0x00));
  74.         colorList.add(new ColorName("GreenYellow", 0xAD, 0xFF, 0x2F));
  75.         colorList.add(new ColorName("HoneyDew", 0xF0, 0xFF, 0xF0));
  76.         colorList.add(new ColorName("HotPink", 0xFF, 0x69, 0xB4));
  77.         colorList.add(new ColorName("IndianRed", 0xCD, 0x5C, 0x5C));
  78.         colorList.add(new ColorName("Indigo", 0x4B, 0x00, 0x82));
  79.         colorList.add(new ColorName("Ivory", 0xFF, 0xFF, 0xF0));
  80.         colorList.add(new ColorName("Khaki", 0xF0, 0xE6, 0x8C));
  81.         colorList.add(new ColorName("Lavender", 0xE6, 0xE6, 0xFA));
  82.         colorList.add(new ColorName("LavenderBlush", 0xFF, 0xF0, 0xF5));
  83.         colorList.add(new ColorName("LawnGreen", 0x7C, 0xFC, 0x00));
  84.         colorList.add(new ColorName("LemonChiffon", 0xFF, 0xFA, 0xCD));
  85.         colorList.add(new ColorName("LightBlue", 0xAD, 0xD8, 0xE6));
  86.         colorList.add(new ColorName("LightCoral", 0xF0, 0x80, 0x80));
  87.         colorList.add(new ColorName("LightCyan", 0xE0, 0xFF, 0xFF));
  88.         colorList.add(new ColorName("LightGoldenRodYellow", 0xFA, 0xFA, 0xD2));
  89.         colorList.add(new ColorName("LightGray", 0xD3, 0xD3, 0xD3));
  90.         colorList.add(new ColorName("LightGreen", 0x90, 0xEE, 0x90));
  91.         colorList.add(new ColorName("LightPink", 0xFF, 0xB6, 0xC1));
  92.         colorList.add(new ColorName("LightSalmon", 0xFF, 0xA0, 0x7A));
  93.         colorList.add(new ColorName("LightSeaGreen", 0x20, 0xB2, 0xAA));
  94.         colorList.add(new ColorName("LightSkyBlue", 0x87, 0xCE, 0xFA));
  95.         colorList.add(new ColorName("LightSlateGray", 0x77, 0x88, 0x99));
  96.         colorList.add(new ColorName("LightSteelBlue", 0xB0, 0xC4, 0xDE));
  97.         colorList.add(new ColorName("LightYellow", 0xFF, 0xFF, 0xE0));
  98.         colorList.add(new ColorName("Lime", 0x00, 0xFF, 0x00));
  99.         colorList.add(new ColorName("LimeGreen", 0x32, 0xCD, 0x32));
  100.         colorList.add(new ColorName("Linen", 0xFA, 0xF0, 0xE6));
  101.         colorList.add(new ColorName("Magenta", 0xFF, 0x00, 0xFF));
  102.         colorList.add(new ColorName("Maroon", 0x80, 0x00, 0x00));
  103.         colorList.add(new ColorName("MediumAquaMarine", 0x66, 0xCD, 0xAA));
  104.         colorList.add(new ColorName("MediumBlue", 0x00, 0x00, 0xCD));
  105.         colorList.add(new ColorName("MediumOrchid", 0xBA, 0x55, 0xD3));
  106.         colorList.add(new ColorName("MediumPurple", 0x93, 0x70, 0xDB));
  107.         colorList.add(new ColorName("MediumSeaGreen", 0x3C, 0xB3, 0x71));
  108.         colorList.add(new ColorName("MediumSlateBlue", 0x7B, 0x68, 0xEE));
  109.         colorList.add(new ColorName("MediumSpringGreen", 0x00, 0xFA, 0x9A));
  110.         colorList.add(new ColorName("MediumTurquoise", 0x48, 0xD1, 0xCC));
  111.         colorList.add(new ColorName("MediumVioletRed", 0xC7, 0x15, 0x85));
  112.         colorList.add(new ColorName("MidnightBlue", 0x19, 0x19, 0x70));
  113.         colorList.add(new ColorName("MintCream", 0xF5, 0xFF, 0xFA));
  114.         colorList.add(new ColorName("MistyRose", 0xFF, 0xE4, 0xE1));
  115.         colorList.add(new ColorName("Moccasin", 0xFF, 0xE4, 0xB5));
  116.         colorList.add(new ColorName("NavajoWhite", 0xFF, 0xDE, 0xAD));
  117.         colorList.add(new ColorName("Navy", 0x00, 0x00, 0x80));
  118.         colorList.add(new ColorName("OldLace", 0xFD, 0xF5, 0xE6));
  119.         colorList.add(new ColorName("Olive", 0x80, 0x80, 0x00));
  120.         colorList.add(new ColorName("OliveDrab", 0x6B, 0x8E, 0x23));
  121.         colorList.add(new ColorName("Orange", 0xFF, 0xA5, 0x00));
  122.         colorList.add(new ColorName("OrangeRed", 0xFF, 0x45, 0x00));
  123.         colorList.add(new ColorName("Orchid", 0xDA, 0x70, 0xD6));
  124.         colorList.add(new ColorName("PaleGoldenRod", 0xEE, 0xE8, 0xAA));
  125.         colorList.add(new ColorName("PaleGreen", 0x98, 0xFB, 0x98));
  126.         colorList.add(new ColorName("PaleTurquoise", 0xAF, 0xEE, 0xEE));
  127.         colorList.add(new ColorName("PaleVioletRed", 0xDB, 0x70, 0x93));
  128.         colorList.add(new ColorName("PapayaWhip", 0xFF, 0xEF, 0xD5));
  129.         colorList.add(new ColorName("PeachPuff", 0xFF, 0xDA, 0xB9));
  130.         colorList.add(new ColorName("Peru", 0xCD, 0x85, 0x3F));
  131.         colorList.add(new ColorName("Pink", 0xFF, 0xC0, 0xCB));
  132.         colorList.add(new ColorName("Plum", 0xDD, 0xA0, 0xDD));
  133.         colorList.add(new ColorName("PowderBlue", 0xB0, 0xE0, 0xE6));
  134.         colorList.add(new ColorName("Purple", 0x80, 0x00, 0x80));
  135.         colorList.add(new ColorName("Red", 0xFF, 0x00, 0x00));
  136.         colorList.add(new ColorName("RosyBrown", 0xBC, 0x8F, 0x8F));
  137.         colorList.add(new ColorName("RoyalBlue", 0x41, 0x69, 0xE1));
  138.         colorList.add(new ColorName("SaddleBrown", 0x8B, 0x45, 0x13));
  139.         colorList.add(new ColorName("Salmon", 0xFA, 0x80, 0x72));
  140.         colorList.add(new ColorName("SandyBrown", 0xF4, 0xA4, 0x60));
  141.         colorList.add(new ColorName("SeaGreen", 0x2E, 0x8B, 0x57));
  142.         colorList.add(new ColorName("SeaShell", 0xFF, 0xF5, 0xEE));
  143.         colorList.add(new ColorName("Sienna", 0xA0, 0x52, 0x2D));
  144.         colorList.add(new ColorName("Silver", 0xC0, 0xC0, 0xC0));
  145.         colorList.add(new ColorName("SkyBlue", 0x87, 0xCE, 0xEB));
  146.         colorList.add(new ColorName("SlateBlue", 0x6A, 0x5A, 0xCD));
  147.         colorList.add(new ColorName("SlateGray", 0x70, 0x80, 0x90));
  148.         colorList.add(new ColorName("Snow", 0xFF, 0xFA, 0xFA));
  149.         colorList.add(new ColorName("SpringGreen", 0x00, 0xFF, 0x7F));
  150.         colorList.add(new ColorName("SteelBlue", 0x46, 0x82, 0xB4));
  151.         colorList.add(new ColorName("Tan", 0xD2, 0xB4, 0x8C));
  152.         colorList.add(new ColorName("Teal", 0x00, 0x80, 0x80));
  153.         colorList.add(new ColorName("Thistle", 0xD8, 0xBF, 0xD8));
  154.         colorList.add(new ColorName("Tomato", 0xFF, 0x63, 0x47));
  155.         colorList.add(new ColorName("Turquoise", 0x40, 0xE0, 0xD0));
  156.         colorList.add(new ColorName("Violet", 0xEE, 0x82, 0xEE));
  157.         colorList.add(new ColorName("Wheat", 0xF5, 0xDE, 0xB3));
  158.         colorList.add(new ColorName("White", 0xFF, 0xFF, 0xFF));
  159.         colorList.add(new ColorName("WhiteSmoke", 0xF5, 0xF5, 0xF5));
  160.         colorList.add(new ColorName("Yellow", 0xFF, 0xFF, 0x00));
  161.         colorList.add(new ColorName("YellowGreen", 0x9A, 0xCD, 0x32));
  162.         return colorList;
  163.     }
  164.  
  165.     /**
  166.      * Get the closest color name from our list
  167.      *
  168.      * @param r
  169.      * @param g
  170.      * @param b
  171.      * @return
  172.      */
  173.     public String getColorNameFromRgb(int r, int g, int b) {
  174.         ArrayList<ColorName> colorList = initColorList();
  175.         ColorName closestMatch = null;
  176.         int minMSE = Integer.MAX_VALUE;
  177.         int mse;
  178.         for (ColorName c : colorList) {
  179.             mse = c.computeMSE(r, g, b);
  180.             if (mse < minMSE) {
  181.                 minMSE = mse;
  182.                 closestMatch = c;
  183.             }
  184.         }
  185.  
  186.         if (closestMatch != null) {
  187.             return closestMatch.getName();
  188.         } else {
  189.             return "No matched color name.";
  190.         }
  191.     }
  192.  
  193.     /**
  194.      * Convert hexColor to rgb, then call getColorNameFromRgb(r, g, b)
  195.      *
  196.      * @param hexColor
  197.      * @return
  198.      */
  199.     public String getColorNameFromHex(int hexColor) {
  200.         int r = (hexColor & 0xFF0000) >> 16;
  201.         int g = (hexColor & 0xFF00) >> 8;
  202.         int b = (hexColor & 0xFF);
  203.         return getColorNameFromRgb(r, g, b);
  204.     }
  205.  
  206.     public int colorToHex(Color c) {
  207.         return Integer.decode("0x"
  208.                 + Integer.toHexString(c.getRGB()).substring(2));
  209.     }
  210.  
  211.     public String getColorNameFromColor(Color color) {
  212.         return getColorNameFromRgb(color.getRed(), color.getGreen(),
  213.                 color.getBlue());
  214.     }
  215.  
  216.     public String findBrighterNameForColor(Color color) {
  217.         String name = getColorNameFromColor(color);
  218.         String nextColorName = name;
  219.         int colorCount = 0;
  220.         while (name.equals(nextColorName) && colorCount < 100) {
  221.             color = color.brighter();
  222.             nextColorName = getColorNameFromColor(color);
  223.             colorCount++;
  224.         }
  225.         return nextColorName;
  226.  
  227.  
  228.  
  229.     }
  230.  
  231.     public String findDarkerNameForColor(Color color) {
  232.         String name = getColorNameFromColor(color);
  233.         String nextColorName = name;
  234.         int colorCount = 0;
  235.         while (name.equals(nextColorName) && colorCount < 100) {
  236.             color = color.darker();
  237.             nextColorName = getColorNameFromColor(color);
  238.             colorCount++;
  239.         }
  240.         return nextColorName;
  241.  
  242.  
  243.  
  244.     }
  245.  
  246.     /**
  247.      * SubClass of ColorUtils. In order to lookup color name
  248.      *
  249.      * @author Xiaoxiao Li
  250.      *
  251.      */
  252.     public class ColorName {
  253.         public int r, g, b;
  254.         public String name;
  255.  
  256.         public ColorName(String name, int r, int g, int b) {
  257.             this.r = r;
  258.             this.g = g;
  259.             this.b = b;
  260.             this.name = name;
  261.         }
  262.  
  263.         public int computeMSE(int pixR, int pixG, int pixB) {
  264.             return (int) (((pixR - r) * (pixR - r) + (pixG - g) * (pixG - g) + (pixB - b)
  265.                     * (pixB - b)) / 3);
  266.         }
  267.  
  268.         public int getR() {
  269.             return r;
  270.         }
  271.  
  272.         public int getG() {
  273.             return g;
  274.         }
  275.  
  276.         public int getB() {
  277.             return b;
  278.         }
  279.  
  280.         public String getName() {
  281.             return name;
  282.         }
  283.     }
  284. }

The previous class is a Java Class. Grails allows us to mix Groovy and Java code seamlessly.

Unofficial Google Actions Java SDK

The SDK is at https://github.com/frogermcs/Google-Actions-Java-SDK however all we need to do is edit our build.gradle to include the library.

/build.gradle

compile 'com.frogermcs.gactions:gactions:0.1.1'

Google App Engine Gradle Plugin

To deploy to App Engine, we are going to use the Google App Engine Gradle Plugin.

We need to modify build.gradle, add the plugin as a buildscript dependency and apply the plugin:

build.gradle

  1. buildscript {
  2.     repositories {
  3.         mavenLocal()
  4.         maven { url "https://repo.grails.org/grails/core" }
  5.     }
  6.     dependencies {
  7.         classpath "org.grails:grails-gradle-plugin:$grailsVersion"
  8.         classpath 'com.google.cloud.tools:appengine-gradle-plugin:1.1.1'
  9.     }
  10. }
  11. ...
  12. apply plugin: 'com.google.cloud.tools.appengine'

Application Deployment Configuration

To deploy to Google App Engine, we need to add the file src/main/appengine/app.yaml

It describes the application's deployment configuration:

/src/main/appengine/app.yaml

  1. runtime: java
  2. env: flex
  3.  
  4. runtime_config:
  5.     jdk: openjdk8
  6.     server: jetty9
  7.  
  8. health_check:
  9.     enable_health_check: False
  10.  
  11. resources:
  12.     cpu: 1
  13.     memory_gb: 2.3
  14.  
  15. manual_scaling:
  16.   instances: 1

Here, app.yaml specifies the runtime used by the app, and sets env: flex, specifying that the app uses the flexible environment.

The minimal app.yaml application configuration file shown above is sufficient for a simple Grails application. Depending on the size, complexity, and features that your application uses, you may need to change and extend this basic configuration file. For more information on what can be configured via app.yaml, please see the Configuring Your App with app.yaml guide.

For more information on how the Java runtime works, see Java 8 / Jetty 9.3 Runtime.

SpringBoot Jetty

As shown in the previous app engine configuration file, we are using Jetty.

Following SpringBoot’s documentation, we need to do the following changes to deploy to Jetty instead of Tomcat.

Replace tomcat:

app/build.gradle

provided 'org.springframework.boot:spring-boot-starter-tomcat'

with jetty:

/build.gradle

provided "org.springframework.boot:spring-boot-starter-jetty"

We need to exclude the tomcat-juli dependency as well. Add:

/build.gradle

  1. configurations {
  2.     compile.exclude module: "tomcat-juli"
  3. }

Deploying the App

To deploy the app to Google App Engine run:

$ ./gradlew appengineDeploy

Initial deployment may take a while. When finished, you will be able to access your app.

Test

Test and interact with your Google Action

Once your app is deployed in Google App Engine Flexible, you can test it.

Go back to the Google Actions Console and click test.

Try to test out your actions with the following phrases:

"Ask Grails Color Finder to find a brighter color for magenta"

You should see the response:

"Sure, here is the test version of Grails Color Finder The brighter color for magenta is fuchsia"

Let’s try another:

Talk to Grails Color Finder

You’ll see:

Sure, here is the test version of Grails Color Finder Hey, it works! Now tell something so I could repeat it

Now respond with something:

Grails is awesome

You should see it echo back what you entered:
You just told: Grails is awesome

Now you can make a more rich skill, and when you are ready to go to the Google API dashboard, fill in the directory listing and submit it for testing.

Next, let’s look at a more advanced example here that we use at our booth demo of a Star Wars Quiz port from an Alexa skill: https://github.com/vanderfox/grails-google-actions-demo

Let’s start here: https://github.com/vanderfox/grails-google-actions-demo/blob/master/grails-app/controllers/grails/google/actions/demo/AssistantActionController.groovy

Here is the code for the action in the controller:

  1. def startStarWarsQuiz() {
  2.  
  3.        AssistantActions assistantActions =
  4.                new AssistantActions.Builder(new GrailsResponseHandler(response))
  5.                        .addRequestHandlerFactory("start.quiz.intent", new StarWarsQuizHandlerFactory())
  6.                        .addRequestHandlerFactory(StandardIntents.TEXT, new StarWarsQuizHandlerFactory())
  7.  And                        .build()
  8.        RootRequest rootRequest = parseActionRequest(request)
  9.        // log to file for debugging
  10.        Writer writer = new FileWriter("/tmp/google-action-request-startstarwarzquiz-${System.currentTimeMillis()}-debug.json")
  11.        Gson gson = new GsonBuilder().create()
  12.        gson.toJson(rootRequest, writer)
  13.        writer.flush()
  14.        writer.close()
  15.        assistantActions.handleRequest(rootRequest)
  16.        null // we dont want to return a grails view here the handlers do this
  17.  
  18.  
  19.  
  20. }

This simple action created a Handler called StarWarsQuizRequestHandler and StarWarsQuizRequestHandlerFactory to do everything.

Let’s look here at StarWarsQuizRequestHandler:

https://github.com/vanderfox/grails-google-actions-demo/blob/master/src/main/groovy/com/vanderfox/gactions/StarWarsQuizRequestHandler.groovy

  1.     @Override
  2.     public RootResponse getResponse() {
  3.         log.debug("Inputs=${rootRequest.inputs.toListString()}")
  4.         Conversation conversation = getConversationService().getConversation(rootRequest.conversation.conversation_id)
  5.         log.debug("Conversation id=${rootRequest.conversation.conversation_id}")
  6.         if (!conversation) {
  7.             log.debug("No current conversation - starting a new one")
  8.             conversation =  conversationService.startConversation(rootRequest.conversation.conversation_id,[:])
  9.         }
  10.         Map conversationMap = conversationService.getConversationMap(conversation.conversationValueMap)
  11.         String intent = rootRequest.inputs.get(0).intent
  12.         switch (intent) {
  13.             case "start.quiz.intent":
  14.                 conversationMap.put(CURRENT_QUESTION,1)
  15.                 return startQuiz(rootRequest,conversation,conversationMap)
  16.             break
  17.             case StandardIntents.TEXT: // actions sdk on google's end requires all response types to be hard coded to text (lame!)
  18.                 return answerQuestion(rootRequest,conversation,conversationMap)
  19.                 break
  20.             case "ask.question":
  21.                 return askNextQuestion("",rootRequest,conversation,conversationMap)
  22.             default:
  23.                 break
  24.         }
  25.     }
  26. Add Comment

The switch statement decides what we are doing based on our action.json schema intent name passed in. So here is the catch -  you can map an intent from the start of the action running, but if the user responds with an answer, you can only use the StandardIntents.TEXT intent or it will not work. From there it’s up to you to decide what action to take. If you don’t know this up front you will get frustrated. It gets worse, because you can’t make a TYPE to that intent, and it’s up to you to parse a natural language response. So if you look in the class when you answer a question with 1,2,3, or 4,  you have to make your own mapping to the English version of the numeric word to the answer from the database.  This really limits what you can do with user responses. This is why Google is steering everyone to use the newer API.AI interface - don’t worry though we have some examples and a Grails plugin to help you with that too! We will talk about that later.

So that leaves us with 3 main methods to handle everything for the quiz - startQuiz, answerQuestion, and askNextQuestion.

So here is the natural order of the flow that happens when the user takes the quiz. We start with startQuiz which gives a initial greeting message and asks the first question. Then the TEXT intent maps to answerQuestion. After you answer the question correctly or not, we ask the next question with askNextQuestion. When looking at the code you can see we are pulling the data from not Google but Amazon’s DynamoDB. This isn’t a requirement though; you can use any relational database or many noSQL stores like Google’s BigTable service. Feel free to adapt this code to use whatever data store you want to use to store your quiz data.

Now let’s move on to API.AI support to make actions.

API.AI Support

What is API.AI? This was a platform agnostic service built for conversational AI that Google acquired shortly after releasing Google Home. Because Amazon’s Alexa had such a head start over Google they needed something to get them on par as quick as possible and API.AI was their answer. It has a nice web UI to build conversations in a way without a lot of coding - but they thought ahead and created SDK’s for several platforms like Java. We use that Java SDK for enabling Webooks and Data Services to build a Grails plugin to make this easy.

Grails Plugin

The Grails plugin uses a simple Trait to enhance a Controller object to plug in to make Webhooks for actions linked to your API.AI project (and Data Services, we’ll get to that later).

First, we need to add the repository to your Gradle project to access the plugin:

  1. repositories {
  2.       maven {
  3.         url  "http://dl.bintray.com/vanderfox/grails-api-ai"
  4.     }
  5. }

Next, we include the plugin itself in the dependencies{} block:

'org.grails.plugins:grails-api-ai:0.1.2'

Now let’s check out an example Controller called WeatherWebhookController here: https://github.com/vanderfox/grails-google-actions-demo/blob/master/grails-app/controllers/grails/google/actions/demo/WeatherWebhookController.groovy

To use the plugin simply implement the Trait:

class WeatherWebhookController implements AiWebhookController {

This will add an index{} action to handle the plumbing to handle talking to the the API.AI service. If you generate a controller that has an index{} action simply delete it and add override doWebhook:

@Override
    void doWebhook(AIWebhookServlet.AIWebhookRequest input, Fulfillment output) {}

Here is where you will put your logic to handle the conversational response to API.AI. Here is the rest of the code that talks to Yahoo weather service and formulates a response:

  1. @Override
  2.     void doWebhook(AIWebhookServlet.AIWebhookRequest input, Fulfillment output) {
  3.  
  4.         String baseurl = 'https://query.yahooapis.com/v1/public/yql?q='
  5.         String city = input.result.parameters.'geo-city'
  6.         String yqlQuery = 'select * from weather.forecast where woeid in ' +
  7.                 "(select woeid from geo.places(1) where text=\'${city}\')"
  8.         String yqlUrl = baseurl + URLEncoder.encode(yqlQuery, 'UTF-8') + '&format=json'
  9.         String yqlResponse = yqlUrl.toURL().text
  10.         JSONElement channel = JSON.parse(yqlResponse).query.results.channel
  11.         JSONElement forecast = channel.item.forecast[0] // this is day 1 of a 7 day forecast
  12.         def weather = "The weather in ${channel.location.city} for " +
  13.                 "${forecast.date} is a high of ${forecast.high} and a low " +
  14.                 "of ${forecast.low} and it will be ${forecast.text}" // units are in F
  15.         output.setSpeech(weather)
  16.         output.setDisplayText(weather)
  17.         //output.contextOut = [] // these are not needed because conversation ends at this point
  18.         //output.data = [:]      // these are not needed because conversation ends at this point
  19.         output.source = 'grails-yahoo-weather'
  20.  
  21.     }

Now let’s go the API.AI web UI to see the other part of what we need to fill in.

Let's add an intent that looks like this:

Note the highlighted areas with city names. The web UI will recognize the city name as a parameter type called ‘geo-city’. This will be passed into your webhook so you can find the city the user wants the weather for.

Next, let’s click on the fulfillment tab on the left. Here we will enter the URL of our Grails app controller and action that responds to the webhook. You can also add a response it will say if it can’t contact your webhook.

Now let’s test out under ‘Try now’. Type in ‘weather in London’ and take a look at the response:

Now we have a fully working webhook!

Data Services

Let’s look at one more thing the plugin helps you with on API.AI and it has a sort of reverse use case.  Let’s talk about that for a second - say you want to integrate some sort of AI ability to your application and leverage actions you’ve assembled in API.AI. You can pass in text as a query to an action programmatically and return a decision it made based on input. For example, maybe you want to give suggestions in your application about the weather in a location while they are booking a hotel on a given date. Maybe you want to find out what recipes you can make with given ingredients while someone is shopping. The possibilities are endless! Here’s how we do it - let’s hook into our Yahoo weather action and webhook to show you how it works.

AiDataController

  1. import ai.api.AIServiceException
  2. import ai.api.model.AIResponse
  3. import org.grails.apiai.AiServiceController
  4.  
  5.  
  6. class AiDataController implements AiServiceController {
  7.  
  8.  
  9. def index() {
  10.     try {
  11.         AIResponse aiResponse = request(request.getParameter("query"), request.getSession())
  12.         response.setContentType("text/plain")
  13.  
  14. response.getWriter().append(aiResponse.getResult().getFulfillment().getSpeech())
  15.     } catch (AIServiceException e) {
  16.         log.error("Error talking to remote service: ${e.message}",e)
  17.     }
  18.  
  19.  
  20. }
  21. }

Conclusion

That’s all you have to do to implement the trait AiServiceController. Now you are ready to make lots of advanced Google Home Actions! Don’t forget these also work via mobile and the latest versions of Android Wear. Feel free to contact me if you are stuck and need some help on Twitter or Grails or Groovy Slack communities.

I'd like to give special thanks to Sergio del Amo for helping me out with publishing the Google Home Grails Guide which much of this content came from! Also to Lee Fox for helping me test out, code and debug this research! Also, thanks to Matthew Moss and Sergio for the Google compute guide which some of this material on bootstrapping Grails with Google comes from.

Software Engineering Tech Trends (SETT) is a regular publication featuring emerging trends in software engineering.

Related Articles

Building Alexa Skills with Grails

Grails Angular Scaffolding

The Power of Groovy DSLs

Web App, Step by Step – Grails Edition