You get a Keyword! And you get a Keyword!

This was originally published on the Clarify blog on 10 June 2015.

Our audio and video search requires you to know something about your media. After all, it’s hard to find something if you don’t know what to look for… until now.

With our new Keywords Insight [Clarify keywords docs], our API performs its normal tuned automatic speech recognition to learn about your audio and video. From those results, it calculates the relevant keywords and allows you to retrieve them with just a couple requests.

To date, this is the most important insight we’ve released in that it informs you about your media. In fact, we already have developers using the results of the Keywords Insight to gain information about the media and perform an informed search.

Using the Keywords Insight is easy.

Just as you always would, add your media to the system:

curl --data "media_url=http://media.clarify.io/audio/books/dorothyandthewizardinoz_01_baum_64kb.mp3" \
     --data "notify_url=http://example.org/sample-receiver" \
     --data "name=Dorothy and the Wizard of Oz" https://api.clarify.io/v1/bundles \
     --X POST --header "Authorization: Bearer myapikey" | jq '.'
# The jq portion is optional and just used to pretty print the resulting json

load.sh

and you’ll receive 201 Created response code along with a payload like this:

{
    "id":"abcde12345",
    "_class":"Ref",
    "_links":{
        "self":{
            "href":"/v1/bundles/abcde12345"
        },
        "curies":[
            {
                "href":"/docs/rels/{rel}",
                "name":"clarify",
                "templated":true
            }
        ],
        "clarify:metadata":{
            "href":"/v1/bundles/abcde12345/metadata"
        },
        "clarify:tracks":{
            "href":"/v1/bundles/abcde12345/tracks"
        },
        "clarify:insights":{
            "href":"/v1/bundles/abcde12345/insights"
        }
    }
}

bundle_callback.json

The most important part of this payload is the href of the clarify:insights key. Once processing is complete, you can make a request to that URL as shown:

curl https://api.clarify.io/v1/bundles/abcde12345/insights \
     --header "Authorization: Bearer myapikey"  | jq '.'
# The jq portion is optional and just used to pretty print the resulting json

get-insights.sh

The response is the list of insights available for that bundle:

{
    "bundle_id": "abcde12345",
    "created": "2015-03-04T05:03:04.292Z",
    "updated": "2015-05-16T20:39:37.508Z",
    "_class": "Insights",
    "_links": {
        "curies": [
            {
                "href": "/docs/insights/{rel}",
                "name": "insight",
                "templated": true
            }
        ],
        "insight:spoken_keywords": {
            "href": "/v1/bundles/abcde12345/insights/54321edcba"
        },
        "insight:classification": {
            "href": "/v1/bundles/abcde12345/insights/edcba56789"
        },
        "parent": {
            "href": "/v1/bundles/abcde12345"
        },
        "self": {
            "href": "/v1/bundles/abcde12345/insights"
        }
    }
}

insights.sh

In this case, we’ll just worry about the insight:spoken_keywords URI. We make a GET request to that URI just as any other:

curl https://api.clarify.io/v1/bundles/abcde12345/insights/54321edcba \
    --header "Authorization: Bearer myapikey"  | jq '.'
# The jq portion is optional and just used to pretty print the resulting json

keywords.sh

The result is the full keyword list:

{
    "bundle_id": "abcde12345",
    "created": "2015-05-16T20:39:37.507Z",
    "id": "54321edcba",
    "name": "spoken_keywords",
    "status": "ready",
    "track_data": [
        {
            "keywords": [
                {
                    "count": 11,
                    "term": "horse",
                    "weight": 0.917
                }, {
                    "count": 9,
                    "term": "uncle",
                    "weight": 0.75
                }, {
                    "count": 7,
                    "term": "dorothy",
                    "weight": 0.583
                }, {
                    "count": 5,
                    "term": "little",
                    "weight": 0.417
                }, {
                    "count": 4,
                    "term": "train",
                    "weight": 0.333
                }
            ]
        }
    ],
    "updated": "2015-05-16T20:39:37.508Z",
    "_class": "SpokenKeywordsInsight",
    "_links": {
        "clarify:bundle": {
            "href": "/v1/bundles/abcde12345"
        },
        "curies": [
            {
                "href": "/docs/rels/{rel}",
                "name": "clarify",
                "templated": true
            }
        ],
        "parent": {
            "href": "/v1/bundles/abcde12345/insights"
        },
        "self": {
            "href": "/v1/bundles/abcde12345/insights/54321edcba"
        }
    }
}

keywords.json

From here, we can process the keywords just like any other JSON array. You may want to use them to perform informed searches, display word clouds, or tag your media. At this point, it’s up to you.