Tuesday 17 January 2017

Get your Raspberry PI to Read Out your Text Messages using IFTTT

Continuing the theme of talking Raspberry Pi's from my recent posts (Get your RPi to Speak the News and Turn your RPi into a Amazon Echo like device), using the brilliant service IFTTT (If This Then That), I've created a way the my Raspberry Pi reads out a notification whenever I receive a SMS on my (android) mobile phone.

If you don't know what IFTTT is, wikipedia calls it :

IFTTT is a free web-based service that allows users to create chains of simple conditional statements, called "applets", which are triggered based on changes to other web services such as Gmail, Facebook, Instagram, and Pinterest. IFTTT is an abbreviation of "If This Then That".

The below shows the sequence of events of how information regarding a SMS received on your phone ends up on the Raspberry Pi.


In IFTTT, you create an "applet",  which fires a trigger when an SMS arrives on your phone, it then sends this info to the central IFTTT service which then performs an action you select. This action could be to turn on light using something like WeMo or Phillips Hue, or adding a row in a Google Sheet, or even sending a Gmail. But in this case the action is to send a webrequest to my raspberry pi.  The web request is handled by some PHP code using JSON , which reads the info send and then executes

Please note I am using the Android SMS trigger on IFTTTT. There doesn't seem to be an equivalent for IPhones

So what do you need to set this up:

1. Setup your Pi as a Webserver

The below 3 steps are covered on the site: http://www.penguintutor.com/linux/light-webserver
  • Raspberry Pi running a web server such as lighttpd
  • Have a fixed IP or use a dynamic dns service such as noip.com to you have a URL which connects to your Raspberry Pi . You may also need to setup your home router to forward incoming http requests to your Raspberry Pi
  • Install PHP to right server side code to handle IFTTT actions and/or to run other web applications you may want to write. 
2. Setup Script to execute Text to Speech 
3. Setup PHP page to handle request 

  • Create a file readSMS.php in your /var/www  directory (this is the default for lighttpd) with the following content :




if ($_SERVER['REQUEST_METHOD'] == 'POST')
{

   $data = file_get_contents("php://input");

 //Removes all 3 types of line breaks
 $data = str_replace("\r", " ", $data);
 $data = str_replace("\n", " ", $data);  

 $result = json_decode($data, true);

 $myfile = fopen("readSMStest.txt", "w") or die("Unable to open file!");
 fwrite($myfile, $data);


 $strText = "SMS received from " . $result['contact'];
 fwrite($myfile,"\n" .$strText);
 fclose($myfile);

 $strCommand = "sudo /home/pi/Documents/speech.sh" . " " .  $strText ; 

 exec($strCommand, $command_output);

 foreach($command_output as $line) :

  echo $line  ;

 endforeach;

}

?>

  • This code reads the input from an HTTP Post (we will setup IFTTT later to send an http post request) , decodes the JSON sent by IFTTT, and extracts the contact name of who sent the SMS to your phone. It then executes the speech.sh script which you should have set up from the previous step. 
  • You may need to change the $strCommand string to the location of where you saved the speech.sh file
  • The script also for debugging purposed writes a text file readSMStest.txt to the same folder
  • The above can be downloaded from:

4. Create Applet in IFTTT

  • Sign up to IFTTT and download the app to your phone 
  • In the IFTTT site click on My Applets then on the New Applet button
  • You will see the below Applet Maker, click on the "+ this" to setup the trigger (which is any SMS received)


  • Start typing "android" and then click on "Android SMS"
  • Now click on "Any new SMS received"

  • Now click on "+that" to setup the Action
  • Start typing "maker" and select the "Maker" action service. 
  • Click on "Make a web request"
  • Fill in the web request as below:
    • Change the URL to the address of your webserver. 
    • The JSON Body Text is below which is easier to copy and paste. This sends the info about your SMS message to your php page using JSON
    • {"contact":"{{ContactName}}" , "message":"{{Text}}" , "Occured":"{{OccurredAt}}", "FromNumber":"{{FromNumber}}"}
      

  • Hit Save and you're done 
  • You can rename the Applet to whatever you want, and can toggle whether you want logging

5.Test it out

Now all you need to do is test it out. Get somebody to send you a text or send one to yourself and hopefully you should hear your Raspberry PI (remember to turn on the speakers and volume is up) say "SMS received from" and then the name from your address book

6.Customise it

Now you can tailor this to do whatever you want. Maybe read out the full message, or use one for the other triggers. Or maybe get your php page to perform some other action like turning on lights


Hope the above is helpul. Any problems , please leave something in the comments.




Wednesday 11 January 2017

Building an Amazon Echo Like Device with a Raspberry Pi and Google Cloud Speech Api

In my previous post I showed how I wrote a python script to read out the latest news headlines using Googles text to speech api.  As I commented in that post, voice recognition and talking devices seem to be the in thing with the release of the Amazon Echo and Google Home.

In this post I show how I created a python script to record sound on your raspberry pi, invoke the google cloud speech api to interpret what was said, and then perform a command on your raspberry pi - so a bit like a basic Amazon Echo.

Setting up your mic

Before I get into the python code, you need a mic setup. As the Raspberry Pi does not have a soundcard you will need a USB mic or a webcam which has an inbuilt mic. I went for the latter and used a basic webcam from logitech.

Once you have your mic plugged in, follow the instructions in the "Step 1: Checking Your Microphone" in: https://diyhacking.com/best-voice-recognition-software-for-raspberry-pi/

Install prerequisites

There is one python library you need which is pycURL, which is used to send data to the Google Cloud Speech Api. Follow the instructions here: http://pycurl.io/docs/latest/install.html

You will also need to install SoX which is an opensource tool to analyse sound files. This is used in the script to detect whether any sound is on the recorded audio, before trying to send it to the google api.

You can install this by running:

 sudo apt-get install sox

One more thing to install, flac . Flac is used to record your sound file in a lossless format which is required by the google api:

You can install this by running:



 sudo apt-get install flac

Setup Google Cloud Speech Api

To do the voice to text processing I am using the speech api which is part of Google Cloud. It is in beta at the moment and offering a free trial.

Follow the instructions on the their site to get your api key which will be needed in the script:

The current downside I've found with this api is the latency. It's currently taking 5-6 seconds for a response to process a 2 second audio file. The google help files yes the response time should be similar to the length of audio being processed. 

Python Script

Now to the actual python code. 

All the files required can be downloaded from here:


The main file to look at is speechAnalyser.py.

This script does the following:

1. If no audio is playing (you don't want to record if you're playing something on your speakers), records sound from your microphone for 2 seconds
2. Uses SoX to check if any sound is on the file and is above a certain amplitude - this helps to not bother processing when there is silence or just background noises
3. If there is sound at a sufficient amplitude, then send the audio to the google api with a JSON message. As said earlier the google api takes a 5-6 seconds and returns a JSON message with the words detected.
4. If the trigger word in this case "Jarvis" is said during these two seconds, a beep sound is played.
5 Records another 3 seconds to listen for a user speaking a commandand sends to the google api like step 3
6.Checks if keyword found in returned text and executes the appropriate command. For example if "news" is mentioned it invokes the GetNews script which I described in my previous post.
7. Loops back to Step 1. 

Remeber to change the line below where it says with the key which was provided when you set up the Google Cloud Speech api


key = ''
stt_url = 'https://speech.googleapis.com/v1beta1/speech:syncrecognize?key=' + ke

Also you should customise your commands in the following section of code:

def listenForCommand(): 
 
 command  = transcribe(3)
 
 print time.strftime("%Y-%m-%d %H:%M:%S ")  + "Command: " + command 

 success=True 

 if command.lower().find("light")>-1  and  command.lower().find("on")>-1   :
  subprocess.call(["/usr/local/bin/tdtool", "-n 1"])
   
 elif command.lower().find("light")>-1  and  command.lower().find("off")>-1   :
  subprocess.call(["/usr/local/bin/tdtool", "-f 1"])
 elif command.lower().find("news")>-1 :
                os.system('python getNews.py')

  elif command.lower().find("weather")>-1 :
                os.system('python getWeather.py')
 
 elif command.lower().find("pray")>-1 :
                os.system('python sayPrayerTimers.py')
 
        elif command.lower().find("time")>-1 :
                subprocess.call(["/home/pi/Documents/speech.sh", time.strftime("%H:%M") ])
 
 elif command.lower().find("tube")>-1 :
                 os.system('python getTubeStatus.py')
 else:
  subprocess.call(["aplay", "i-dont-understand.wav"])
  success=False

 return success 

The other interesting part of the script to look at is, where it sends the data over to the Google Cloud Speech Api.

It creates a JSON message, and then encodes the audio in base64.

Within the outgoing JSON message, there is a phrases section, where I've included my trigger word "Jarvis", which makes it more likely the speech engine recognises this

The final bit then gets the text from the response.


#Send sound  to Google Cloud Speech Api to interpret
 #----------------------------------------------------
 
 print time.strftime("%Y-%m-%d %H:%M:%S ")  + "Sending to google api"


   # send the file to google speech api
 c = pycurl.Curl()
 c.setopt(pycurl.VERBOSE, 0)
 c.setopt(pycurl.URL, stt_url)
 fout = StringIO.StringIO()
 c.setopt(pycurl.WRITEFUNCTION, fout.write)
 
 c.setopt(pycurl.POST, 1)
 c.setopt(pycurl.HTTPHEADER, ['Content-Type: application/json'])

 with open(filename, 'rb') as speech:
  # Base64 encode the binary audio file for inclusion in the JSON
         # request.
         speech_content = base64.b64encode(speech.read())

 jsonContentTemplate = """{
    'config': {
         'encoding':'FLAC',
         'sampleRate': 16000,
         'languageCode': 'en-GB',
   'speechContext': {
        'phrases': [
         'jarvis'
      ],
     },
    },
    'audio': {
        'content':'XXX'
    }
 }"""


 jsonContent = jsonContentTemplate.replace("XXX",speech_content)

 #print jsonContent

 start = time.time()

 c.setopt(pycurl.POSTFIELDS, jsonContent)
 c.perform()


 #Extract text from returned message from Google
 #----------------------------------------------
 response_data = fout.getvalue()


 end = time.time()
 #print "Time to run:" 
 #print(end - start)


 #print response_data

 c.close()
 
 start_loc = response_data.find("transcript")
     temp_str = response_data[start_loc + 14:]
 #print "temp_str: " + temp_str
     end_loc = temp_str.find("\""+",")
     final_result = temp_str[:end_loc]
 #print "final_result: " + final_result
     return final_result






I have to give a big shout out to the following sites which gave me ideas on how to write this script:

https://diyhacking.com/best-voice-recognition-software-for-raspberry-pi/ - This contains the instructions on how to setup a microphoen on the raspberry pi
https://github.com/StevenHickson/PiAUISuite - Full Application which does what the above script does but is configurable. But not sure if it still works with the new Google Speech Api