Question

Thanks for taking the time to review my query. I’m writing a program in C# WPF .NET 4 using Expression Blend 4 and Visual Studio 2010. I am using System.Speech.Recognition to listen for commands from the user and to perform a task. My question is;

Once the SpeechRecognized Event has been raised, and you perform a task, how can you pause the task, ask the user to voice another command, and resume or abandon the previous task?

My current train of thought has focused around changing the loaded Grammar and accept a new command, and pass that back to the previous operation (if that makes sense).

To give you an idea of what I am trying to do, I am listening for a command such as “Computer, what is the system status”. I would like to play an audible response back to the user asking for another command such as (Enter Authorization Code), and once the user does that, it will either perform the response when a certain command(code) is returned (such as, “Gamma 47 Echo”), reject the command(code) or time-out and reload the previous Grammar (I am using external XML Grammar files).

I have uploaded the code that I have to this point to PasteBin http://pastebin.com/X3vJJJrM. I am new to programming, with some experience with Java. Any help would be greatly appreciated.

Answer 1

I believe you are complicating the logic too much. I may be over-simplifying it though… 🙂

You can include all your grammar in a single file. As you listen for your initial command, set a flag in your code that will then listen for followups.

Here is a (simplified) grammar set:

<grammar version="1.0" xml:lang="en-US" root="rootRule" tag-format="semantics/1.0" xmlns="http://www.w3.org/2001/06/grammar">
<rule id="rootRule">
  <one-of>
    <item>
      <tag>out.SysCommand = "STATUS";</tag>
      Computer what is the system status
    </item>
    <item>
      <tag>out.SysCommand = "PASSWORD";</tag>
      Gamma
      <one-of>
        <item>Four</item>
        <item>Forty</item>
      </one-of>
        Seven Echo
    </item>
  </one-of>
</root>
</grammar>

Your callback might look something like the below:

private void OnSpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
  if (e.Result.Confidence >= ConfidenceThreshold)
  {
    if (IsListening)
    {
      if (e.Result.Semantics["SysCommand"] != null)
      {
        switch (e.Result.Semantics["SysCommand"].Value.ToString())
        {
          case "PASSWORD":
            // The was waiting for this, now you can act on it
            break;
          default:
            // something else was said, reset!
            break;
        }
      }
    }
    else if (e.Result.Semantics["SysCommand"] != null)
    {
      if (e.Result.Semantics["SysCommand"].Value.ToString() == "STATUS")
      {
        // do stuff that prompts user for password
        IsListening = true;
      }
    }
  }
}

Your callback looks for an “IsListening” flag. If it isn’t set, it checks if you asked for status. If you did, it sets the flag and now waits for the password.

You can streamline the above code. I copied it and did some quick edits out of something similar I did. In my case, I do the following:

1. Listen for the system’s name (“Floyd”)
2. Set “IsListening” flag
3. Display a prompt on the screen (visibility bound to IsListening flag) showing possible followup commands
4. Listen for those commands and act on them appropriately

You could load and unload new grammars as needed but, unless your grammar file gets huge, the overhead of those loads/unloads will start to quickly catch up to just looking through a few extra grammar rules in a single set.