Showing posts with label Subtitles. Show all posts
Showing posts with label Subtitles. Show all posts

Friday, October 31, 2014

XmlRpcMessageService - A Google Apps Script Library

This post is part of a series: Trakt.TV & Subtitles - A Google Apps Script Project

Overview

As part of my project I worked on fetching subtitles availability from several providers. One of the providers is OpenSubtitles.org. This site has an API which is implemented based on XML-RPC. From Wikipedia on XML-RPC: "XML-RPC is a remote procedure call (RPC) protocol which uses XML to encode its calls and HTTP as a transport mechanism". This protocol is quite verbose and is not that JavaScript friendly. Because of these reasons I decided to implement a library to ease the use of XML-RPC in JavaScript. This library enables the user to describe the request and get the response in plain JSON. The bottom line is that using this library, executing a method call in JavaScript, for example the "LogIn" method to OpenSubtitles.org, can be as easy as:
var methodCall = {
   methodName : "LogIn",
   params : [ "", "", "en", "OS Test User Agent" ]
}
var url = "http://api.opensubtitles.org/xml-rpc";
var methodResponse;
var onSuccess = function(mr) { methodResponse = mr; };
XmlRpcMessageService.execMethodCall(url, methodCall, onSuccess);

The First Translation - Verbose JSON

First I defined a quite verbose JSON model which translates to and from an XML-RPC in a straight forward conversion. Examples are better than words-

A simple value is described in XML-RPC:
<value><string>David</string></value>
In JSON:
{ type : "string", value : "David" }

An array in XML-RPC:
<value><array>
   <data>
      <value><string>David</string></value>
      <value><string>John</string></value>
   </data>
</array></value>
In JSON:
{ type : "array", value : [
      { type : "string", value : "David" },
      { type : "string", value : "John" }
   ] }

A struct value in XML-RPC:
<value><struct>
   <member>
      <name>qwerty</name>
      <value><double>1234</double></value>
   </member>
   ...
</struct></value>
In JSON:
{ type : "struct", value : [
      name : "qwerty",
      value : { type : "double", value : "1234" }
   ] 
}
So basically, you can see that a similar schema is kept for the different value types.
A method call in XML-RCP contains the name of the method and the values for the parameter:
<methodCall>
   <methodName>foo</methodName>
   <params>
      <param>...value...</param>
      ...
   </params>
</methodCall>
In JSON:
{
   methodName : "foo",
   params : [
      { type : "...", value : "..." },
      ...
   ]
}
The result of this transformation is still verbose, more or less the same as the XML representation. The main advantage of the JSON representation is that it is much easier to use in JavaScript since there is no need for string manipulation, XML encoding, etc.

The Second Transformation - "Semantic" JSON

(Maybe the name "semantic" is not that good, but I'll stick with it for now)
After implementing the first translation, which can be used in all scenarios, I decided to go a little bit further. I thought - instead of persisting the name of the value type, persist the value in its designated type. So for the next transformation I decided on the following mapping:

JSON - Verbose
JSON - Semantic
double value object
{ type: "double", value: 7 }
number
7
string value object
{ type: "string", value: "Hello" }
string
"Hello"
boolean value object
{ type: "boolean", value: true }
boolean
true
array value object
{ type: "array", value: [ ... ] }
array
[ ... ]
struct value object
{ type: "struct", value: [ { name: "bar", value: ... } ] }
object
{ bar: ... }

The method call itself stays the same, the values become more simple:
{
   methodName : "foo",
   params : [ "Hello", 7, true, { bar : 100 } ]
}

Of course this transformation can be applied on the response as well, which is simple as:
{ params : [ ... ] }

Conclusion

The idea was to make the use of XML-RPC web services easier in JavaScript in general, and in Google Apps Script specifically. This implementation has two-step transformation. The first step is as verbose as the original XML-RPC, but can be used easily in JavaScript. The second step makes it much less verbose but does not support, for now, some of the value types defined in XML-RPC specification (e.g. integer, base64, etc).


The code is available in GitHub:
https://github.com/yinonavraham/GoogleAppsScripts/tree/master/XmlRpcMessageService

Full Example for Comparison


XML-RPC
<methodCall>
   <methodName>foo</methodName>
   <params>
      <param>
         <value><string>Hello</string></value>
      </param>
      <param>
         <value><double>7</double></value>
      </param>
      <param>
         <value><boolean>1</boolean></value>
      </param>
      <param>
         <value><array>
            <data>
               <value><string>a</string></value>
               <value><string>b</string></value>
            </data>
         </array></value>
      </param>
      <param>
         <value><struct>
            <member>
               <name>bar</name>
               <value><double>100</double></value>
            </member>
         </struct></value>
      </param>
   </params>
</methodCall>

JSON - Verbose
{ 
   methodName : "foo", 
   params : [ 
      { type: "string", value : "Hello" }, 
      { type: "double", value : 7 },
      { type: "boolean", value: true },
      { type: "array", value: [ 
            { type: "string", value: "a" }, 
            { type: "string", value: "b" } 
         ] },
      { type: "struct", value: [ 
            { name: "bar", value: { type: "double", value: 100 } } 
         ] } 
   ]  
}

JSON - Semantic
{ 
   methodName : "foo", 
   params : [ "Hello", 7, true, [ "a", "b" ], { bar : 100 } ] 
}

Friday, October 17, 2014

Trakt.TV & Subtitles - A Google Apps Script Project

Before I Get to the Technical Stuff

My wife and I like watching TV series, but we like choosing what and when, not what the satellite/cables providers dictate. This is why we mostly use XBMC for that. XBMC is great - you get all the new episodes from a variety of providers, quickly and with good quality. Since we're not native English speakers we also like having subtitles. XBMC is great for that as well.

A while ago, before we started using XBMC, I used to go once in a while browsing for new episodes, subtitles, etc. It was frustrating... XBMC solved most of it. I also found Trakt.TV a great tool for keeping track of what we haven't watched yet.

The only part that was left is keeping track of the subtitles availability. For that I had two options: start watching the episode, then try to find matching subtitles. The other option was to look for the subtitles in the providers' websites before that. Both options aren't nice. There are some series that we don't really mind watching without subtitles, but some we wont watch until they have subtitles. In those cases it is frustrating to go through all that work just to find there are no subtitles available...

This is why I started this project. The goal was to give a view which consolidates my shows' progress (from Trakt.TV) with the availability of subtitles for each un-watched episode. I chose Google Apps Script because of two reasons: 1- it is quite simple, it gives a variety of built-in services and it's free. 2- I did not know it and wanted to see what it has to offer.

Technical Overview

OK, so as I mentioned before, the project is based mainly on Google Apps Script. Although it has quite a lot native services, I had some gaps of missing functionality I needed to fill. The project currently includes the following modules/services:
  1. TrakTV & Subtitles-
    This script is published as a webapp. It is the main module, it implements the model generation and the UI rendering of the service.
  2. EnahancedCacheService- (post)
    This is a service for adding more features to GAS's native CacheService. It mainly supports various value types (not just string: e.g. number, boolean, object) and supports values larger than 128KB.
  3. TraktTVClient-
    This is a client for the Trakt.TV REST API. It currently implements only what is really needed for my needs, but it can be enhanced with much more quite easily if needed.
  4. SubsCenterOrgClient-
    This is a client for the SubsCenter.org subtitles provider. Since this provider has no API, so instead it parses the website - only what is currently required.
  5. OpenSubtitlesOrgClient-
    This is a client for the OpenSubtitles.org subtitles provider. This provider has an API based on XML-RPC.
  6. XmlRpcMessageService- (post)
    This is a service for creating and handling XML-RPC messages. It lets the user work with simple JavaScript objects instead of the verbose XML. Not all data types defined in the specification are supported by this implementation.
In the next posts I will elaborate on some of the services mentioned above.