Alexa Skill Developer #1 Support Issue for Me

I took a long hiatus from building skills and am just getting back into it.  I made some dumb mistakes getting stuff to work.  But recently I realized it wasn't always my dumb mistakes.  I ran into an issue twice where all I had to do was "reboot" my Alexa skill.  Equivalent of the the favorite help desk suggestion of turn it off and back on again; unplug it and plug it back in.

Flipping the Test is enabled for this skill was required a couple of times that I remember.

1. I updated my invocation phrase and rebuilt my models, but things did not work right until I toggled this switch.
2. After having my certification fail, I tried testing my interactions again and my skill didn't respond.  Toggling the switch fixed that also.

So, when all else fails... flip this switch off and on again.

Alexa Zipcode Lookup

I have been playing around with building another Alexa skill.  I mostly wanted to explore some different techniques to use in a skill.

This new skill will lookup zipcodes for you.  It does it in a few different ways:

• If you tell it a zipcode, it will tell you what city that zipcode is for
• If you tell it a city and state, it will tell you what zipcodes are associated with that city and state
• It will return a card to your Alexa app with a map of the location

The skill uses a few different services.  Zippopotam.us is the service that I use to do zipcode and reverse zipcode lookups.  And I make use of Google Maps API to assist with the spelling of city names, when the Alexa Voice Service doesn't recognize the spelling.  Also, the maps are provided by Google Maps API.

I am going to write about the different parts of this skill across a few different posts.  I hope some of these techniques will help you with your next skill.

The code for this skill is up on Github...  https://github.com/chrispauly/alexa-zipcode-lookup 

NPM to the Rescue for your Alexa Skill

I started writing Alexa Skills in AWS Lambda using the example blueprint and online code editor.

Recently, using the new code samples that Amazon has on Github, I've graduated to writing my code locally in VS Code, and then zipping up the files and uploading them into my Lambda function.

Now, I'm taking it one step further, and instead of writing all the code, I am leveraging NPM to provide functions for tasks that would be tedious and/or difficult to write myself.

I wasn't sure how difficult it would be to start using NPM, like what config settings I would need or how to use them in my skills.  I found out it is as easy as 1-2-3.

1. Download NodeJS and install it on your OS.
2. Open a Command Prompt and go into the ./src folder of your skill (the folder where index.js lives).  Run the command npm install [package name]
3. Refer to the examples provided for your package and incorporate that in your code.

That's it!

My main concerns were having to worry about directory paths or unfamiliar javascript configuration files.  But I had none of those issues.  I just zipped up my code and uploaded it into my Lambda function and everything worked!

I stumbled upon how easy this process was when I needed to write a function to translate digits into words.  When I wanted to ask for a city name in my service, I found that certain city names did not come in as expected.  For example, my hometown, Two Rivers, WI, when the CityName slot was populated by AVS with my spoken words, it came into the skill service as "2 Rivers".  I needed to use the actual city name in my skill service.  It required the actual spelling of the city name, so I needed to translate the number 2 into the word Two.

I found that there are other cities with numbers in their name, so I decided that I should build a more generic function that could translate numbers into words.  But before I wrote that function, I Google'd for javascript functions that do that.  NPM had a number of packages that would work for me, but I settled on number-to-words.  I followed the 1-2-3 steps above and was so happy it worked without any extra effort.

Alexa Meet LUIS

LUIS (Language Understanding Intelligent Service) is a research project that uses a Bing Cognitive AI service to match spoken dialog with an intent.  When creating a LUIS app, you'll create an Intent Schema and Sample Utterances similar to that in an Alexa skill.

Once you have a LUIS application built, you need to train it to create its internal models that it uses to predict which intent best matches the phrase sent to it.  It responds with a JSON object that includes your intent names and the scores that represent how well that phrase matched an existing intent.

I wanted to leverage the power of LUIS to determine what intent has the highest probability to match a spoken phrase, when I didn't include that phrase as a sample utterance in my Alexa skill.

This required me to create a catchall intent.  The schema for this intent is exactly like I described in a previous blog article about matching all the spoken words...  Polly Want a Cracker? A Simple Alexa Skill that Echoes Your Words

Once I created a skill that had a number of intents, along with my catchall intent, I began testing phrases that were not included in my sample utterances.  I soon realized that the AVS already performs LUIS-like language understanding.  It was extremely difficult to ever have the Alexa skill choose my catchall intent.

The one way I could see using LUIS now, would be to build the Polly Want a Cracker skill.  Then use LUIS to define my intents and slots (which are called entities in LUIS).  LUIS has a great service that provides active learning in your app.  LUIS stores the new phrases it hears and lets you review them and label new entities.  This is not available in Alexa right now, so a skill developer has a difficult time understanding what phrases people are trying in their skill that may not be working.

Polly Want a Cracker? A Simple Alexa Skill that Echoes Your Words

Doesn't it bug you that your skill service doesn't receive the understood text that was spoken.  When you don't launch a custom skill, the card that is included in your Alexa app displays the understood spoken text.

Why doesn't Amazon pass spoken text to a custom skill?  I think this keeps Alexa more secure.  If an Alexa owner added a skill that was easy to accidentally launch, then Alexa would be sending that service spoken dialog at times when the owner did not intend to send data.  Also, in a loud room, it is possible background dialog could be sent to your skill.  If someone launches the skill in the same room where someone else is on the phone reciting their credit card number, that is data someone would not want sent to a random skill developer.

However, if you had a need to receive the understood spoken text in your skill, it is possible.  You can use a built-in slot type that only exists for compatibility with earlier ASK skills, AMAZON.LITERAL.  Here is an example of how that would work below:

Intent Schema

{
  "intents": [
     {
      "intent": "Repeater",
      "slots": [
        {
          "name": "A",
          "type": "AMAZON.LITERAL"
        },
        {
          "name": "B",
          "type": "AMAZON.LITERAL"
        },

        --- CLIPCLIPCLIP ---
        --- And so on... ---
        --- CLIPCLIPCLIP ---

        {
          "name": "Y",
          "type": "AMAZON.LITERAL"
        },
        {
          "name": "Z",
          "type": "AMAZON.LITERAL"
        }
      ]
    }
  ]
}

Sample Utterances

Repeater {a|A} {a|B} {a|C} {a|D} {a|E} {a|F} {a|G} {a|H} {a|I} {a|J} {a|K} {a|L} {a|M} {a|N} {a|O} {a|P} {a|Q} {a|R} {a|S} {a|T} {a|U} {a|V} {a|W} {a|X} {a|Y} {a|Z}

I only have one intent that I use to match a phrase.  You'll need to add the missing slots that I clipped.  My sample utterance matches the intent with all AMAZON.LITERALs.  The one thing I noticed while testing this, is if I didn't include enough slots for the spoken words, the AVS wouldn't match my intent.  So, I created a bunch of slots, to catch a decently long phrase.

Then I use some javascript math to loop over all the slots and rebuild the phrase that was heard by Alexa.

exports.handler = function (event, context) {
    console.log(JSON.stringify(event.request));

    if(event.request.type === "LaunchRequest")
        context.succeed(buildResponse("Say something", {}, false));
    else if(event.request.type === "IntentRequest") {
        var output = "";
        for(var i=0; i <= "Z".charCodeAt(0)-"A".charCodeAt(0); i++ ) {
            var slot = event.request.intent.slots[String.fromCharCode("A".charCodeAt(0) + i)];
            if(slot && slot.value)
                output += " " + slot.value;
        }
        context.succeed(buildResponse(output, {}, false));
    }
    else
        context.succeed(buildResponse("", {}, true));
};

function buildResponse(output, attributes, shouldEndSession) {
    return {
        version: "1.0",
        sessionAttributes: attributes,
        response: {
            outputSpeech: {
                type: "PlainText",
                text: output
            },
            reprompt: {
                outputSpeech: {
                    type: "PlainText",
                    text: output
                }
            },
            shouldEndSession: shouldEndSession
        }
    };
}

That should do it for you.  You will now get the complete spoken phrase in your response.

How might this be useful?  It actually won't be that useful generally, since you'd need to do your own language understanding AI to match the spoken phrase in your skill logic.  And generally, if you have other intents in your schema, AVS tries really hard to match the spoken dialog with one of those before it would use this one.

Monitoring time while using your skill

I wanted to explore making a game using the Alexa.  I thought it would be useful to know how long the game had been running.  Here is an example of how I am able to track the time elapsed since launching a skill.

Intent Schema

{

  "intents": [

     {

      "intent": "TimeIntent"

     },

    {

      "intent": "AMAZON.StopIntent"

    }

    ]

}

Sample Utterances

TimeIntent How much time has passed

I only have one intent that I use to ask for the elapsed time.  And the StopIntent just helps me quit the skill.  All of the work happens in the Lambda function.

var tickingClockMP3 = 

    "https://alexafiles.blob.core.windows.net/demo/ticking-clock.mp3";

exports.handler = function (event, context) {

    var attributes = {};

    

    if (event.session.new) {

        attributes = { startTime: event.request.timestamp };

    }

    else {

        attributes = 

        { 

            startTime: event.session.attributes.startTime,

            timeDiff: (new Date(event.request.timestamp) - 

                       new Date(event.session.attributes.startTime)) / 1000

        };

    }

    

    try {

        if (event.request.type === "SessionEndedRequest") {

                    onSessionEnded(event.request, event.session);

                    context.succeed();

        }

        else if(event.request.type === "LaunchRequest")

            context.succeed(buildTickerResponse(attributes, false));

        else if(event.request.type === "IntentRequest") {

            var output = "" + (attributes.timeDiff || 0) 

                       + " seconds have past.

";

            context.succeed(buildTickerResponse(attributes, false, output));

        }

        else

            context.succeed(buildTickerResponse(attributes, false));

    } catch (e) {

        context.fail("Exception: " + e);

    }

};

function buildTickerResponse(attributes, shouldEndSession, output) {

    return {

        version: "1.0",

        sessionAttributes: attributes,

        response: {

            outputSpeech: {

                type: "SSML",

                ssml: output || ""

            },

            reprompt: {

                outputSpeech: {

                    type: "SSML",

                    ssml: "At anytime you can say... " + 

                          "Alexa, how much time has past?

"

                }

            },

            shouldEndSession: false

        }

    };

}

A few things to note here.  

1. When creating MP3s, make sure you follow the rules about where they can be hosted and the allowed format.  I used FFMPEG to fix it up: ffmpeg -i input.mp3 -b:a 48k -ar 16000 output.mp3.
2. While the SSML is playing the audio, the only way to get Alexa to hear you again, you have to say her name.
3. I store the startTime as an attribute.  I have to copy that value over to the new attributes object I send back each time.

That's it!  You can now keep track of how much time has past since your skill started.

Use Microsoft Azure to Host Alexa Card Images

Alexa Skills Kit includes the ability to add an image on your card.  So, I uploaded a file to my website and added the url to the response.card.image.smallImageUrl value.  Sadly, all I got was an empty grey box in the card.  It turns out that I needed to setup CORS.

An easy option is to use AWS's S3 storage.  But, I am more familiar with Microsoft Azure.  And I have free credits to use each month, so I wanted to see what it would take to use Azure's storage to host these images.

It turns out that it's not too difficult to setup.  And even if you don't have free credits, it will cost you pennies each year to host thousands of images used in your skills.

This blog will step you through

1. Creating an Azure Storage Account resource
2. Uploading files to the blob container
3. Enabling CORS on your Azure Storage Account

Create a Blob Storage

In your Azure portal click '+ New'


Search for Storage Account.
Click Create.
Give your storage a name.  This will be part of the URI you'll use to access your files.
There are a lot of other options in here that you can choose depending on your needs.  Each choice will change the cost of your storage.  But based on the small files and limited amount of data I'd be storing in here (less than 1GB), I can't see how it would cost me more than pennies every month, no matter which options I choose.

Now add a container.  Make sure to change the Access type to Blob.  This will ensure that your files are accessible to the Alexa app.

Access Keys

You'll need your access keys to finish up a couple of the final steps.  These are available under the settings of your storage account.  Copy your key1 because you will need it later.

(the keys are for my old storage account, so I didn't bother blurring them out)

Upload files

The easiest tool to upload files into your container is available in Microsoft Azure Tool's for Visual Studio.  In the Cloud Explorer, drill down into your resources (Storage Accounts -> [BlobStorageName] -> Blob Containers -> [ContainerName]).

The first time you open your container, you may need to provide your storage account's Access Key.

You can now open up the Blob Container Editor.  This tool allows you to upload blobs/files into your container.  Each file will have a URL associated with it that can be accessed publicly, since this was setup as a blob container.

Test out your uploads in a web browser.  You can right-click on your uploads and copy the url.

You can now upload MP3s to this container and use them in SSML responses.

If you would like to use images in your cards, you'll need one more change.  The echo.amazon.com website and Echo app requires that the hosting site has CORS enabled.  This lets the web browser know that it's ok to download content from your website, even though it's displaying a webpage hosted at Amazon.

Enable CORS

There is MSDN documentation about how to enable CORS on your Azure Storage containers.  I tried to follow the RESTful PUT request to enable this, but first had issues with the format of my authorization header.  I found another article that describes how to format the auth header, but then I ran into another issue. I was getting a generic error about my XML content being malformed.  I gave up and decided to spend time on a different approach, the Azure C# SDK.

I created a new console project and added the nuget package, WindowsAzure.Storage.  Then, with a few lines of code, I enabled the CORS rule necessary for using images on cards.  Copy the code below into the Main() function.  And then update it to include your AccountName and AccountKey.

CloudStorageAccount StorageAccount 
  = CloudStorageAccount.Parse(@"DefaultEndpointsProtocol=https;AccountName=alexademo;AccountKey=CKBnwLxjJqZ0n6zvoe...");

CloudBlobClient BlobClient = StorageAccount.CreateCloudBlobClient();

ServiceProperties blobServiceProperties = BlobClient.GetServiceProperties();
blobServiceProperties.Cors = new CorsProperties();
blobServiceProperties.Cors.CorsRules.Add(new CorsRule()
{
    AllowedMethods = CorsHttpMethods.Get,
    AllowedOrigins = new List() { "http://ask-ifr-download.s3.amazonaws.com" }
});
            
BlobClient.SetServiceProperties(blobServiceProperties);

Run the code once and your storage container will now return images to the Alexa app.

Test it out

Your cards will now be able to access your images (be sure to update your skill code to point to the new azure blob stored images).  If you have any problems, make sure you are following the other requirements around the images

• Correct format: JPEG or PNG
• Allowed size: Less than 2MB

Enjoy!

Add Some "Character" to Your Alexa Cards

Your Alexa skill can respond to a request with both a text string that the Echo will recite and some text that will display on a card in the Alexa app.  This card has very limited formatting options.  When the Alexa Skills Kit was initially announced, the only option was plain text on the Simple card.  You could include newlines, but in general, no other formatting.

Now there is an additional card type, the Standard type, that allows you to include an image.  However, in practice, there are a lot of restrictions for where the image can be hosted, the image format, and the image maximum file size.

An easy alternative way to make your card stand out is to include Unicode characters.  This technique works with all of the current card types.

Go to a website that shows the Unicode character that you want to use and copy the printable character.  Paste that into the text of your JSON response for your skill.  I use AWS Lambda to host my skill service and the Unicode character shows up in the online text editor.

When your skill sends the card to your Alexa app, it will display those characters inline with your text.  Here's an example of using the Unicode card suit characters in a skill I published:

Or how about adding some emoji's onto your card to personalize it... you can do that too!  Go to the Emoji Unicode Data website, and copy an emoji character from the Brow. column.  Paste that into your JSON response object.

Enjoy!

Additional details:

Unicode Characters
Emoji website

Home Cards for the Amazon Alexa App

Last Thing to Do Before Publishing An Alexa Skill Hosted In Lambda

If you develop a skill for Amazon'z Echo and use the AWS Lambda service to host your skill, there is something you should do before you submit your skill for certification.

Publish a New Version and update your skill to use that version.

You'll notice that the ARN changes for this new version.  This is the ARN that you'll want to update your Skill to use.  The reason that you should use this version, is because it is Read-Only and matches the code that you've spent all the time developing and debugging.

I also suggest giving this version an alias.  You'll notice that assigning an alias also changes the ARN, so I recommend using that alias ARN in your skill.

The main reason for a versioned ARN is to make it easier to work on the next version, without accidentally breaking your current live version.

You may notice that if you find a bug after you publish your skill this way, there is no way to change that code since it is in a read only version.  You can only update the $LATEST version.  But that isn't the one that your Alexa skill is referencing.  But, if you aliased your version and pointed your skill to use the alias named ARN, then you can:

1. Publish a new version of the fixed $LATEST code
2. Remove the alias from the old version
3. Assign the new version that alias name
4. Set the Event Source to Alexa Skills Kit (that setting doesn't seem to copy with new versions/aliases)

Now your skill is referencing the new bug-fixed version.