Developing and distributing an Alexa Skill (Part 1)

Despite my protests, my wife went ahead and bought an Alexa Echo for the house. I didn’t like giving Amazon access to so much personal data, and still don’t, but now, having used the device for a month or so, it has become helpful.

After you get over the unsavory way in which Alexa tries to sell you amazon stuff, and the frustration of a stuttered or hesitant sentence not being recognized, the utility of having a live mic that can process inquiries in real time is undoubtedly useful, and it’s also fun (my son likes asking it what a blue whale sounds like).

Alexa is at its best when used simply. Single queries, not conversations, quips, not chapters, reading, not writing.

Amazon has made the development pretty easy, but unfortunately, it is fragmented across multiple offerings which was the main issue I had approaching the platform.

The utility I wanted early on was the ability to ask for verses from the New American Bible, and not the King James version which is missing several books. You can take a look at the outcome here.

And what follows is the path I took to develop my Alexa Skill that can do lookup a verse in the New American Bible (Revised Edition)…

Creating a new skill

First, go to https://developer.amazon.com and create an account.

Once logged-in, visit the Developer Console. There you will see the Alexa option in the navbar, and choose Alexa Skills Kit.

amazon-dev-console_navbar-alexa

Under the Alexa Skills listing, click the Create Skill button:

amazon-dev-console_alexa-skills_create-skill.jpg

In the next menu you can choose a sample app to start with. I would definitely recommend choosing the Fact Skill as it has the minimal point of entry. As far as I can tell, the biggest difference between them besides the source code that it creates with, is the Intent settings that are also created.

alexa-skills_choose-template

This will land you on the Skill’s dashboard page:

alexa-skills_skill-dashboard

On the left side navigation, you will see the elements that compose your skill, from a meta-data standpoint. All of these boil down the JSON that appears in the JSON Editor, which comes in handy when you need to edit some of the Utterances, or if you want to quickly copy a Skill’s settings into another skill.

Also in the sidebar (not shown in screen above), is the Endpoint configuration link. This is the URL where Amazon’s servers will look for your app’s code to handle the launch request.

Where does the skill app code go?

The skill has to point to a URL which is hosted either by AWS or on your own server. That URL needs to handle specific requests.

I used the AWS Lambda function option, which the Amazon developer site encourages, for a variety of reasons; most notably is that the free-tier offers a lot of usage for no cost. However, you can point it to a different URL where your server is hosted, or an ngrok.io endpoint to do local development. I explored this, but ended up going with the Lambda option, because it removed some of the unknowns, and I wanted to get up and running with least amount of effort.

System and custom Intents

When you develop a skill, you will need to handle a few system level Intents. These are like system events that occur depending on how your app is used. They can get pretty advanced, such as doing media playback, but that’s out of scope for us here.

The Fact Skill creates most of the Intents you need to handle, and includes one custom Intent to drive the core function of the app: to speak a random fact.

In addition to your own custom Intents, the Intents you will need to handle in your app are

  • AMAZON.CancelIntent
  • AMAZON.HelpIntent
  • AMAZON.StopIntent
  • LaunchRequest
  • SessionEndedRequest

Notice that the last two listed do not appear in the sidebar of the out-of-the-box Intents for the Fact Skill template. But they will need to be handled within the app.

Invocation of the skill

The invocation is how Alexa will launch your app. This is not so straightforward.

Alexa will launch your app given certain launch phrases such as “open”, “ask”, “start”, “launch”, etc, and the Invocation Name is the phrase that will map to your Skill. So if your skill is named “My Awesome Skill”, your invocation name could be simply “my awesome skill”, or you can make it different from the Skill name, such as “the awesomest skill,” whereby the full invocation would be

Alexa, open the awesomest skill.
alexa-skills_invocation-setting.jpg

Skill Utterances and Slots

When you want to try your skill, you need to launch it first. In the case of the out-of-the-box Fact Skill, the app doesn’t have need for an interaction, it simply blurts out one of a bunch of space facts.

When you develop, you should prompt the user for input when they launch the app. This input will then be matched against the Utterances you’ve added to the Skill. This will then be fed into the Intent that the matched Utterance maps to, and will be available as an event “slot” parameter.

If you click on the GetNewFactIntent, you are taken to a page where you can both rename the Intent (or you can just delete it create a new one), and then populate the Utterances for it.

alexa-skills_utterances_rename

In our case, we are going to create a couple ways to state how awesome something is.

As you type in the new Sample Utterance field, you can use the curly brackets to create a slot, which is the variable that will be populated by the user when he interacts with your skill. The utterance that gets matched and populated depends on the formation of the spoken sentence.

Let’s create a one to ten utterance with the following slot

{OneOutOfTen} out of ten
alexa-skills_utterances.jpg

I set the slot type to AMAZON.NUMBER so that it will be populated with an integer value.

I’m gonna make two ways of assessing awesomeness of a thing, the first is a number from 1 to 10 using the AMAZON.NUMBER slot type, the second is a custom slot type which I will call AwesomeType, and it will allow me to make a list of phrases which map to 1 to 10 numbers. Finally, I will create a slot for the thing which is being assessed using the AMAZON.SearchQuery which will use Alexa’s voice recognition to resolve a string phrase.

First, to create the custom slot type, click the add button next to the Slot Types menu item in the left sidebar.

Fill in a name for the type (no spaces):

alexa-skills_slot-type-create

Next you can add values in one of two ways:

  1. by entering them one at a time, or
  2. by using the bulk import, which receives a CSV file.

I prefer the second option, whereby the developer can load in a tab delimited file (without headers). The first column is the value which Alexa will use speech recognition to match, the second is the ID which is what will be sent to your app’s function (the Name is also sent in), and any following column is a synonym which Alexa will use alternately to map to the same ID (or value if no ID is provided). So the CSV would look like this:

Value1  ID1     Synonym1        Synonym2        (etc)

With the field populated, my slot type looks like this:

alexa-skills_slot-type_values

So Alexa will look for one of these values to populate the AwesomeType of one of my Utterances, which are as follows:

alexa-skills_utterances-list.jpg

So

  • {Thing} will use Amazon speech recognition to deliver a string phrase
  • {OneOutOfTen} will resolve to an integer value
  • {AwesomeType} will resolve to one of my field options which will include an ID that I can parse into an integer

So I have one Custom Intent and 5 built-in ones, one of which is optional; AMAZON.FallbackIntent, which can be used in the event that Alexa cannot understand what you said. It’s probably a good idea to handle this with at least a “Sorry, I couldn’t understand that kind of awesomeness” message.

Make sure to save the model using the Save Model button at the top of the page.

Creating an AWS Lambda function

Save the model, and then hop to https://aws.amazon.com and sign up for an account. Once you’ve done that, you can create a Lambda function which will be used to host your app.

Under the AWS Console, click on the services menu on the top navbar and select Lambda from the drop down menu:

aws-services_menu.jpg

On the listing for functions, click Create Function. Then on the Create Function page, select Blueprints, and then enter “alexa” in the search filter and choose alexa-skill-kit-sdk-factskill from the results, and click Configure:

aws-services_lambda_create-function

On the next screen you will enter the Basic Information of the Lambda; the name of the Lambda function (no spaces), and also choose a Role (which has the permissions for which the function to be called). I just click through which looks like this:

aws-services_lambda_role-access

Allow the IAM Role, and then you will return to the Basic Information screen:

aws-services_lambda_create-function_basic-information

Syncing the Lamba function with the Alexa Skill

After this, you will be taken to the Lambda editing page, where you will see printed at the top, the ARN code of the Lambda function. You will need to copy this and enter it into the Endpoint screen of your Alexa Skill later:

aws-services_lambda_add-trigger

On this edit page, you will need to add a Alexa Skills Kit trigger from the Add Triggers menu. You will see that when you do this, you will be prompted to configure the trigger using the Skill ID of your Alexa Skill:

aws-services_lambda_configure-trigger

So you need to have open both the Alexa Developer Console and the AWS Lambda Editor open to do this next part.

  1. In the Alexa Developer Console, click the Endpoint option in the leftside navigation, and copy the Skill ID you see listed and
  2. Paste it in the AWS Lambda Editor‘s trigger configuration field, and click Add there. Then Save it using the Save button near the ARN code at the top of the page.
  3. In the AWS Lambda Editor, copy the ARN code that appears at the top of the page, and
  4. Paste it in the Alexa Developer Console‘s Endpoint configuration form’s Default Region field, then save it using the Save Endpoint button that appears at the top of the page. You should see a green Skill Manifest Saved Successfully toast message appear.

Note on Default Region: I’ve been using East 1 (N. Virginia). I don’t know if choosing another region has any impact but I have read some users complaining about region issues, possibly between resources, not sure, but I stuck to East 1.

Back to the AWS Lambda Editor, you now need to code your app…!

I will tackle this in Part 2.

Every issue, quirk, and glitch I ran into while developing a SharePoint Add-In

My job can be heavy with the SharePoint Online, and I do like to tinker around on it using the REST API and Script Editor web parts, and you can do cool stuff just with those.

But as soon as you start specking out workflows, it’s probably time to elevate to Apps or Add-Ins.

I can’t figure out what SharePoint wants to call them, but when you go to develop it in Visual Studio, you’ll be picking the SharePoint add-in project:

sharepoint-addin-project.jpg

I set out to create a polling add-in because I already had it working as a REST API script web part, so I figured, why not package the thing?

But the simplicity with which you can build out your app via Visual Studio’s bountiful UI is almost wholly negated by the weird ticks will will encounter as you develop.

I will try to catalog them here…

Extra, un-asked for Features

The first issue I ran into was new Feature nodes would be added depending on what I deleted from and added to the project. Visual Studio would seem to lose track of the Feature1 that it created by default, and begin to add new items to a second or even 3rd Feature node.

When that happens, delete the extra feature nodes, and take a look at Feature1 to be sure it is including all the relevant items of your project (which is probably all of them).

sharepoint-addin-quirk-extra-features

Retraction, Retraction, what’s your function?

When you re-deploy your add-in, it sometimes needs to retract the one that’s already there, and many issues arise.

One is, retraction can dodder on indefinitely, causing you to need to cancel the operation (the option should appear under the Build menu). But then you will need to rerun retraction, which is fine, except when SharePoint thinks the add-in is retracted.

A couple times I’ve had to go onto the deployment site, and delete the add-in, but even when cleared from the recycle bin and second stage recycle bin, SharePoint still believes it’s there!

One solution I found is to tick up the version (after you’re certain the add-in is removed), this way, SharePoint will not insist the app is there with the same version, because you changed the version:

pollster-app-manifest

(h/t this thread)

Confused schema

It’s not enough to have lists with separate schema settings, if their Type field is the same, SharePoint may confuse the schema, and you will see two lists with the same field set, which is jarring, really. Even if you completely delete the list, and recreate it, remapping every association it had in the project, SharePoint may find the schemas of the second list confusing.

One user found the solution was to set the type of the second list to 1000, and tick up as you add even more lists with possibly confusion inducing schemas.

(h/t this thread)

List Field Default Values

It’s not that there’s an issue with setting default values for your list fields, its just that when the UI is so comprehensive as it is in Visual Studio, it can be perplexing to need to do something on the code-level, especially setting a default value of a field which is standard procedure.

To do this, you have to edit the schema of your list, and add a sub-element <Default> within the <Field> element:

sharepoint-addin-quirk-default-list-field-values.jpg

(h/t to this thread)

Hiding list fields from the New, Edit, and View forms

Not every field of your list should be shown on every form, unless you want to give the user that sense of vague formlessness so common in modern art today. But to hide the field, you have to specify a series of values in the list Schema.xml.

Go tho the <Fields> element, and find the fields within that you want to show or hide, by adding the following (True/False) properties:

  • ShowInNewForm
  • ShowInEditForm
  • ShowInViewForms

 

HOW DO I GET THE STUPID SITE PATH TO MY ADD-IN?!

Alas, there are many answers to this on various developer forums, and none of them work.

Basically, your add-in is going to situate itself in an iframe that loads content which is outside of the site it gets added to, so it’s not really aware of some key globals such as _spPageContextInfo, or SP.ClientContext.get_current(), and you can’t add scripts to fix the problem because ultimately…it’s in a separate place.

NOTE: If you need to talk to info that resides on the destination site, your going to need to use SharePoint’s cross-domain ajax functionality, which was beyond my purpose, but you can read about it here.

If you’re simply looking to using SharePoint REST API within your web part’s quarantined site/section/ward, you need to pass in that URL via the web part parameters. SharePoint has a bunch of dynamic parameters called tokens, listed in the second table (Tokens that can be used inside a URL) here, but you have to add them.

sharepoint-addin-quirk-passing-the-app-path.jpg

Once you add them, you can build the REST API calls off of them, especially the {AppWebUrl} parameter. I suppose you could do some text processing on the document.URL string to get this value, but that seems risky, and SharePoint ought to tell you where it put your add-in.

If you’re interested in making calls to the site you’re adding to, you’ll want to make use of the {HostUrl} parameter. This may require some permissions setup in the AppManifest, but that’s further than I went.

Showing a list on your Add-In’s default page

My add-in makes use of a couple of lists, one of which I want to show on that default page, which I figure should be the sort of administrative front-end of the app and provide the user with some web parts to let them perform basic content addition, but don’t expect SharePoint to just give one to you.

You have to edit the elements.xml of the Pages node in the project, and add the following within the Default.aspx page listing:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Module Name="Pages">
    <File Path="Pages\Default.aspx" Url="Pages/Default.aspx" ReplaceContent="TRUE">
      <AllUsersWebPart WebPartZoneID="HomePage1" WebPartOrder="1">
        <![CDATA[
          <webParts>
            <webPart xmlns="http://schemas.microsoft.com/WebPart/v3">
              <metaData>
                <type name="Microsoft.SharePoint.WebPartPages.XsltListViewWebPart, Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
              </metaData>
              <data>
                <properties>
                  <property name="ListUrl">Lists/Polls</property>
                  <property name="IsIncluded">True</property>
                  <property name="NoDefaultStyle">True</property>
                  <property name="Title">Polls</property>
                  <property name="PageType">PAGE_NORMALVIEW</property>
                  <property name="Default">False</property>
                  <property name="ViewContentTypeId">0x</property>
                </properties>
              </data>
            </webPart>
          </webParts>
        ]]>
      </AllUsersWebPart>
    </File>
 </Elements>

Now, in the Default.aspx page itself, you can add markup that refers to this element data you just added:

sharepoint-addin-quirk-default-page-webpart

Wait, what’s wrong? You mean you want to see more than just the title field of your list in that web part?

I’m not 100% on this part, but this change worked for me:

Edit the <ViewFields> tag in the Schema.xml of your list where you find a <View> element with BaseViewId=”0″, Type=”HTML”…

sharepoint-addin-view-fields.jpg

Taking Add-In to the Market

I wanted to share, and possible earn some coin, off my frustration developing this supposedly simplistic add-in, but in doing so, I hit the following wierdness:

You’ve got to select at least one Supported Locale in the AppManifest, which was fine with me, I just wanted English to start, but if you just use the UI, it won’t add the setting properly. You need to go into the AppManifest.xml code-view and edit the xml so that it’s en-US (xx-XX).

sharepoint-addin-quirk-locale.jpg

Beyond that, the screenshots simply must be 1366px X 768px, naturally! Which seems a little unfair to little Add-Ins like mine which I had to pad-out with blank space because, seriously, it must be this size.

JSONP with Mendix REST Services

Mendix is an exciting platform because it simplifies something that tends to get overly-complicated: web-facing, record-based systems.

Because of Mendix’s platform ideology of rapid development, it isolates the developer from the guts of the java code it generates and compiles.

The problem is that many of the apps (modules) available from the App Store are not as flexible or feature rich as they might be, and some have the classic problem of backwards compatibility with a rapid-release IDE (in this case, the Mendix Business Modeler).

When Mendix started offering REST services, I took interest because it’s one of the better use cases for the platform, and I found that the REST Service module delivered both consume and publish services, and the setup was intuitive and worked well…with one exception: No JSONP support.

JSONP is a very simple modification to JSON output that simply wraps the JSON object of the response in a function name that you pass to it in a parameter usually named callback.

So we go from

[{"field1":"stringvalue", "field2":100}]

to

passedCallbackFunctionName([{"field1":"stringvalue", "field2":100}]);

Then, on the receiving side, the function named in the query parameters is either automatically invoked by libraries like jQuery (because it detects a parameter named callback in the request url), or manually by calling eval() on the response body.

This can be fixed, but requires an adventure into the java code layer that Mendix generally keeps out of sight.

To setup REST in Mendix, first download the Rest Services module from the App Store:

mendix-rest-services-module

With this downloaded, you will see the following content in the tree view of the module in the Project Explorer:

mendix-rest-module-tree

You want to create a microflow to handle the initialization of your REST microflow endpoints, and set that in the Project Settings After startup property:

mendix-project-settings-after-startup.jpg

 

My Initialization microflow looks like this:

mendix-rest-init-microflow

The first activity takes no parameters. The second needs and third need settings similar to the following:

mendix-rest-create-microflow-service-action-settings

So in my module named Search, I am exposing a microflow named “incidents”; giving it a wildcard for security role, which means anyone can access it; being unimaginative with my description; and specifying HttpMethod of RestServices.HttpMethod.GET enum value.

I do this again for another endpoint that serves up records from a entity table of requests.

I’m not going to go into the mechanics of the microflows I’m publishing, except to note that they produce JSON based on the fields of a non-persistable entity, which I populate via a persistable entity at the time of endpoint invocation. For this simple ticket system app, my domain model looks like this:

mendix-rest-domain-entities

The search args will be automatically populated, based on the parameters in the query string.

For example, when you hit the running endpoint and specify contenttype=json, you get this result:

http://localhost:8080/rest/incidents?firstname=James&lastname=Gilbert&limit=10&offset=0&status=Open&q=&contenttype=json

mendix-rest-query-results

So the service publishes under the path

http://<your-app>/rest/<microflow>

…But as you can see, this is not JSONP.

 

To have Mendix Rest Services offer JSONP support, you only need to make two tiny edits to the files RestServiceRequest.java, and RestServices.java which are added to your Mendix project when you add the Rest Services module.

To find this, go to the Project menu, and choose Show Project Directory in Explorer

mendix-show-project-directory.jpg

And navigate to

<Your Project Path>\javasource\restservices

mendix-rest-path

There’s RestServices.java, and RestServiceRequest.java is under the publish folder.

Add the following to RestServices.java:

mendix-rest-RestServices-java

And in RestServiceRequest.java, here are the changes:

mendix-rest-RestServiceRequest-edit1mendix-rest-RestServiceRequest-edit2mendix-rest-RestServiceRequest-edit3mendix-rest-RestServiceRequest-edit4

Here’s a zip of all the changes: Link

When you have your service compiling and running locally, you can deploy it to the Mendix Free App Cloud for further testing. NOTE: the service will go to sleep after a period without requests, so as tempting as it is, it’s probably a bad idea to leave it on the free app cloud.

So now you can make ajax calls your Mendix REST service endpoints from jQuery without worrying about Cross-Domain requests, which ajax now blocks.

Here’s a sample of one such request:

var mendixurlbase = "https://<your-hosted-app>/rest/incidents?callback=?";
$.getJSON( mendixurl, { 
    firstname: "James",
    lastname: "Gilbert",
    limit: 10,
    offset: 0,
    status: "Open",
    contenttype: "json"
}) .done(function( data ) {
    console.log(data);
});

Creating and publishing an extension for Visual Studio Code

As far as text editors go, I like Notepad++, it does the job, and there’s little tolerance for frustration with text editors. But good extensions tend to win me over (Firebug for Firefox kept me off Chrome for years).

So I’ve got to hand to to Visual Studio Code; they made easy the extensions area of text editor functionality which is usual esoteric and above the average coders ability.

Start with the documentation, which is extensive, and the Getting Started guide is comprehensive: https://code.visualstudio.com/docs/extensions/example-hello-world

To start, you’ll need to install yo from npm:

npm install -g yo generator-code

Then create a folder for your extension, and cd to it. Run yo from that folder:

your folder path>yo code

…This launches the utility which sets up a basic hello world extension.

You don’t need to setup a publisher prior to running the utility, but I did by following this guide:

https://code.visualstudio.com/docs/extensions/publish-extension#_publishing-extensions

The key take-away is that you will be creating a permission which will have a Personal Access Token (they refer to it in some steps as PAT). When you create this you have to copy it somewhere safe!

vsteam-personal-access-token

With the token copied, go to command line and run:

command line>vsce create-publisher <your-publisher-handle>
Publisher human-friendly name: <your-display-name>
E-mail: <your@email.com>
Personal Access Token: <paste-the-token>

Successfully created publisher ‘<your-publisher-handle>’.

Now with a publisher name and token, you can run yo and fill in the info:

vscode-yo-utility-create-new-pub

This is going to create a bunch of files, but the main ones are package.json, which you will need to edit, and at the very least add:

 "icon": "icon.png",
 "repository": {
   "type": "git",
   "url": "https://github.com/jamespgilbert/comment-labels.git"
 }

NOTE: the url is pointing to my own github account.

The icon can reside in your folder, and it needs to be a 128×128 px PNG file, such as:

icon

You also need to alter the README.md file. Mine is pretty spare:

# comment-labels README

This extension allows you to create big comment label blocks for easy visual separation of code.

## Using

On a blank line in the editor, type the text you want to make a comment label for, and then run Comment Label from the command palette.

![Usage](https://raw.githubusercontent.com/jamespgilbert/comment-labels/master/demo.GIF)

## Known Issues

The command does not support some characters such as slashes and other characters that do not render well in the ascii text format.

## Release Notes

### 1.0.0

Initial release of Comment Label extension.

Notice the reference to the image, I am refering to an image that’s hosted by github because I checked it into my repository. You’ll need to provide the image from a hosted location. It doesn’t need to be github, none of your project needs to be on github, but if you’re using git with your project, you might as well, right?

If you use github, you got to reference the file using the raw.githubusercontent.com url, which you can get by browsing to the file on github and copying the location of the image, then pasting that into the browser and going to the location, which will reformulate the url.

In short: it’s not going to be the same address as the one that you can browse to in github.

git-hub-image

Packaging the extension

In order publish, you need to package your extension using another utility called vsce. You can install it via Node.js command line:

>npm install -g vsce

You need to create a package file (.vsix), which you can do with the following command:

C:\your-path\your-extension-folder>vsce package

This will create a .vsix package file with the version number specified in your package.json.

Publishing your extension

Then login with vsce so your extension can be uploaded:

C:\your-path\your-extension-folder>vsce login jamespgilbert
Publisher ‘jamespgilbert’ is already known
Do you want to overwrite its PAT? [y/N] y
Personal Access Token for publisher ‘jamespgilbert’: <paste-your-token>

Authentication successful. Found publisher ‘James P Gilbert’.

Now, to publish the extension to the Extension Marketplace, run

C:\your-path\your-extension-folder>vsce publish -p <your-personal-access-token>

Note: you can’t publish a version that is the same or earlier than a previously published one.

If you want to replace the current version of your extension without upping the version, you’ve got to unpublish it:

C:\your-path\your-extension-folder>vsce unpublish jamespgilbert.comment-labels
This will FOREVER delete ‘jamespgilbert.comment-labels’! Are you sure? [y/N] y
Successfully deleted jamespgilbert.comment-labels!

Then you can go ahead and publish the same version as the now-deleted extension.

Behold the published extension

Once published, the extension should show up very soon if not immediately afterwards in the extensions tab in VS Code:

vscode-extensions-tab-with-my-extension

You can see it here too: https://marketplace.visualstudio.com/items?itemName=jamespgilbert.comment-labels

I created animated demo gif using Snagit 13, which allows you to record your computer use and edit out the awkward pauses.