Searching the UK Parliament API from Slack Slash Commands Using a Python Microservice via Hook.io Webhooks

Several years ago now, I remember being excited by the idea of webhooks which provided a simple callback mechanism for executing remote microservice commands on the web via an HTTP request. For whatever reason, webhooks never really became part of my everyday toolkit, but with Amazon Lambda functions coming to my attention again recently as Google experimented with a rival service, I’ve started looking at them again.

To get back into the swing of how these things work, I thought I’d tried to put together a Python simple script that could run a search query against a data collection from the UK Parliament API; the request would be triggered from a slash command in Slack.

On the Slack side, you need to define a couple of custom integrations:

• a Slash Command that will define the name of the command that you want to handle and provide a callback URL that should be accessed whenever the slash command is issued;
• an Incoming Webhook that  provides a callback URL on Slack that can handle a response from the microservice accessed via the slash command callback.

The slash command is declared and the URL of the service to be accessed needs to be specified. To begin with, you may not have this URL, so it can be left blank to start with, though you’ll need to add it in when you get your callback service address. When the callback URL is requested, a token is passed along with any extra text from the slash command string. The callback service can check this token against a local copy of the token to check that the request has come from a known source.

The incoming webhook creates an endpoint that the service called from the slash command can itself callback to, providing a response to the slash command message.

To handle the slash command, I’m going to develop a simple microservice on hook.io using Python 2.7. To begin with, I’ll define a couple of hidden variables that I can access as variables from my callback script. These are a copy of the token that will be issued as part of the slash command request (so I can verify that the service request has come from a known source) and the Slack incoming webhook address.

I can access these environment variables in my script as elements in the Hook['env'] dict. The data package from Slack can be accessed via the Hook['params'] dict.

The service definition begins with the declaration of the service name, which will provide a stub for the hook.io callback URL. The automatically generated Home URL is the one that needs to be provided as the callback URL in the Slack slash command configuration.

The code for the service can be specified locally, or pulled in from a (public) gist.

In the service I’ve defined, I make a request over the Parliamentary research briefings dataset (others are available) and return a list of keyword matching briefings as well as links to the briefing document homepage.

import json, re
import urllib2
from urlparse import urlparse
from urllib import urlopen, urlencode

class UKParliamentReader():

    """
    Chat to the UK Parliament API
    """

    def __init__(self):
        """ Need to think more about the structure of this... """
        pass

    def qpatch(self,query):
        t=[]
        for a in query.split('or'):
            t.append('({})'.format(a.strip().replace(' ',' AND ')))
        return ' OR '.join(t)

    def search_one(self,query, typ='Research Papers',page=0,ps=100):
        url='http://lda.data.parliament.uk/researchbriefings.json'
        urlargs={'_view':typ,'_pageSize':ps,'_search':self.qpatch(query),'_page':page}
        url='{}?{}'.format(url, urlencode(urlargs))
        data =json.loads(urlopen(url).read())
        response=[]
        for i in data['result']['items']:
            response.append("{} [http://researchbriefings.parliament.uk/ResearchBriefing/Summary/{}]".format( i['title'],i['identifier']['_value']))
        return response

    def search_all(self,query, typ='Research Papers',ps=100):
        return

    def responder(self,hook):
        ukparl_token=hook['env']['ukparl_token']
        ukparl_url=hook['env']['ukparl_url']
        r=self.search_one(Hook['params']['text'])
        r2="; \n".join(r)
        payload={"channel": "#slashtest", "username": "parlibot",
                 "text":"I know about the following Parliamentary research papers:\n\n {}".format(r2)}
        req = urllib2.Request(ukparl_url)
        req.add_header('Content-Type', 'application/json')
        response = urllib2.urlopen(req, json.dumps(payload))

u=UKParliamentReader()
if Hook['params']['token'] == Hook['env']['ukparl_token']:
    u.responder(Hook)

With everything now set up, I can make use of the slash command:

Next up, I’ll see if I can work out a similar recipe for using Amazon AWS Lambda functions…

See also: Chatting With ONS Data Via a Simple Slack Bot

NOTE: this recipe was inspired by the following example of using Hook.io to create a Javascript powered slash command handler: Making custom Slack slash commands with hook.io.