Friday, October 31, 2014

XmlRpcMessageService - A Google Apps Script Library

This post is part of a series: Trakt.TV & Subtitles - A Google Apps Script Project

Overview

As part of my project I worked on fetching subtitles availability from several providers. One of the providers is OpenSubtitles.org. This site has an API which is implemented based on XML-RPC. From Wikipedia on XML-RPC: "XML-RPC is a remote procedure call (RPC) protocol which uses XML to encode its calls and HTTP as a transport mechanism". This protocol is quite verbose and is not that JavaScript friendly. Because of these reasons I decided to implement a library to ease the use of XML-RPC in JavaScript. This library enables the user to describe the request and get the response in plain JSON. The bottom line is that using this library, executing a method call in JavaScript, for example the "LogIn" method to OpenSubtitles.org, can be as easy as:
var methodCall = {
   methodName : "LogIn",
   params : [ "", "", "en", "OS Test User Agent" ]
}
var url = "http://api.opensubtitles.org/xml-rpc";
var methodResponse;
var onSuccess = function(mr) { methodResponse = mr; };
XmlRpcMessageService.execMethodCall(url, methodCall, onSuccess);

The First Translation - Verbose JSON

First I defined a quite verbose JSON model which translates to and from an XML-RPC in a straight forward conversion. Examples are better than words-

A simple value is described in XML-RPC:
<value><string>David</string></value>
In JSON:
{ type : "string", value : "David" }

An array in XML-RPC:
<value><array>
   <data>
      <value><string>David</string></value>
      <value><string>John</string></value>
   </data>
</array></value>
In JSON:
{ type : "array", value : [
      { type : "string", value : "David" },
      { type : "string", value : "John" }
   ] }

A struct value in XML-RPC:
<value><struct>
   <member>
      <name>qwerty</name>
      <value><double>1234</double></value>
   </member>
   ...
</struct></value>
In JSON:
{ type : "struct", value : [
      name : "qwerty",
      value : { type : "double", value : "1234" }
   ] 
}
So basically, you can see that a similar schema is kept for the different value types.
A method call in XML-RCP contains the name of the method and the values for the parameter:
<methodCall>
   <methodName>foo</methodName>
   <params>
      <param>...value...</param>
      ...
   </params>
</methodCall>
In JSON:
{
   methodName : "foo",
   params : [
      { type : "...", value : "..." },
      ...
   ]
}
The result of this transformation is still verbose, more or less the same as the XML representation. The main advantage of the JSON representation is that it is much easier to use in JavaScript since there is no need for string manipulation, XML encoding, etc.

The Second Transformation - "Semantic" JSON

(Maybe the name "semantic" is not that good, but I'll stick with it for now)
After implementing the first translation, which can be used in all scenarios, I decided to go a little bit further. I thought - instead of persisting the name of the value type, persist the value in its designated type. So for the next transformation I decided on the following mapping:

JSON - Verbose
JSON - Semantic
double value object
{ type: "double", value: 7 }
number
7
string value object
{ type: "string", value: "Hello" }
string
"Hello"
boolean value object
{ type: "boolean", value: true }
boolean
true
array value object
{ type: "array", value: [ ... ] }
array
[ ... ]
struct value object
{ type: "struct", value: [ { name: "bar", value: ... } ] }
object
{ bar: ... }

The method call itself stays the same, the values become more simple:
{
   methodName : "foo",
   params : [ "Hello", 7, true, { bar : 100 } ]
}

Of course this transformation can be applied on the response as well, which is simple as:
{ params : [ ... ] }

Conclusion

The idea was to make the use of XML-RPC web services easier in JavaScript in general, and in Google Apps Script specifically. This implementation has two-step transformation. The first step is as verbose as the original XML-RPC, but can be used easily in JavaScript. The second step makes it much less verbose but does not support, for now, some of the value types defined in XML-RPC specification (e.g. integer, base64, etc).


The code is available in GitHub:
https://github.com/yinonavraham/GoogleAppsScripts/tree/master/XmlRpcMessageService

Full Example for Comparison


XML-RPC
<methodCall>
   <methodName>foo</methodName>
   <params>
      <param>
         <value><string>Hello</string></value>
      </param>
      <param>
         <value><double>7</double></value>
      </param>
      <param>
         <value><boolean>1</boolean></value>
      </param>
      <param>
         <value><array>
            <data>
               <value><string>a</string></value>
               <value><string>b</string></value>
            </data>
         </array></value>
      </param>
      <param>
         <value><struct>
            <member>
               <name>bar</name>
               <value><double>100</double></value>
            </member>
         </struct></value>
      </param>
   </params>
</methodCall>

JSON - Verbose
{ 
   methodName : "foo", 
   params : [ 
      { type: "string", value : "Hello" }, 
      { type: "double", value : 7 },
      { type: "boolean", value: true },
      { type: "array", value: [ 
            { type: "string", value: "a" }, 
            { type: "string", value: "b" } 
         ] },
      { type: "struct", value: [ 
            { name: "bar", value: { type: "double", value: 100 } } 
         ] } 
   ]  
}

JSON - Semantic
{ 
   methodName : "foo", 
   params : [ "Hello", 7, true, [ "a", "b" ], { bar : 100 } ] 
}

Tuesday, October 21, 2014

GmailSearchBuilder - A Google Apps Script Library

Preface

I recently wanted to write a simple Google Apps Script for automatically delete old email messages (threads). The reason was a lot of email threads that started to accumulate, which are not important enough for me to keep.

There are solutions out there. All those I had seen are based on assigning a label to the messages to be deleted (e.g. "DELETE ME"). Some are quite naive - iterating over all the threads returned from the GmailLabel object, which is not the best way to go, the performance of this solution is poor. The better solutions use GmailApp.search method with a query such as "label:delete-me older_than:30d" to fetch the threads with the "DELETE ME" label which are older than 30 days. (Good post: Create time-based Gmail filters with Google Apps Script)

Also, you need to assign the label for that to work. Manual assignment is tedious. Of course it can be solved by filters which assign this label automatically. But what if you want to assign other labels, and the "DELETE ME" label is only the secondary label to assign? (e.g. when you filter messages into different labels, but all of them can be deleted after a while). This can also be done by a scheduled job which assigns the "DELETE ME" label to all messages with other specified labels (which are not yet marked for deletion). This can be done similarly using the search query "label:my-other-label -label:delete-me".

After writing those queries, and trying to figure out how to deal with some issues, like what to do with spaces in the label name (one solution- replace all spaces with '-'), I decided to implement a service for building search queries. The goal of this service is to solve for you those questions (another one for example: how to format dates), while letting you all the flexibility of the query syntax.

The Query Builder

Well, it's quite simple. The idea was to have a fluent API using method chaining. The builder needs to give flexibility similar to writing hard-coded query strings, with simplicity and to handle for the developer the tedious technical stuff. Since it's a builder, it can be used for building queries dynamically in run-time. The builder also lets you call the search directly on the builder itself as a syntactic sugar.

Some examples
Assume:
var GSB = GmailSearchBuilder;

1- Building and running a simple query
var query = GSB.newQuery().label("My Other Label").exclude().label("DELETE ME").build();
var threads = GmailApp.search(query);
//The query: label:my-other-label -label:delete-me

2- Building and running directly from the builder
var threads = GSB.newQuery().label("DELETE ME").olderThan(30,DateType.DAY).search();
//The query: label:delete-me older_than:30d

3- Building a query with a sub-query (group)
var threads = GSB.newQuery().subQuery(GSB.newQuery().label("My Label 1").or().label("My Label 2")).exclude().label("DELETE ME").search();
//The query: (label:my-label-1 OR label:my-label-2) -label:delete-me

3- Search for threads between dates
var threads = GSB.newQuery().from("john@acme.com").before(new Date(2014,11,31)).after(new Date(2014,0,1)).larger(1,SizeType.MB).search();
//The query: from:john@acme.com before:2014/12/31 after:2014/01/01 larger:1048576


The source code is available in GitHub:
https://github.com/yinonavraham/GoogleAppsScripts/tree/master/GmailSearchBuilder

Friday, October 17, 2014

EnhancedCacheService - A Google Apps Script Library

This post is part of a series: Trakt.TV & Subtitles - A Google Apps Script Project

Overview

GAS's native CacheService is a very good caching service. It provides several levels of caching (user, script, ...) with very good basics for caching string values. But this is also part of it drawbacks:
  1. It can store only string values
  2. The values are limited to 128KB
  3. Some additional features are lacking, such as additional information on entries (e.g. when an entry was last updated)
With these limitations, and the requirements I had in the project I was working on, I decided to implement an enhancement to this service: EnhancedCacheService
The purpose of the new service was to wrap the existing service and add additional features while preserving the existing features and the flexibility of choosing the cache type. The additional features which are currently implemented:
  1. Support for native JavaScript types, such as: number, boolean, object
  2. Support for values larger than 128KB
  3. Additional information on cache entries - get the date an entry was last updated
In order to instantiate an enhanced cache service, use:
var cache = EnhancedCacheService.wrap(CacheService.getUserCache());
As you can see, this gives you the freedom to choose the type of cache you want to use.

Basics

In order to support the requirements I have decided to store a value descriptor instead of the value itself. This answers two main requirements:
  • Ability to get additional information on cache entries
  • Support in the simple native data types, such as boolean and number
This is done by creating a value descriptor object containing the following information:
  • The value, in its original form (number, string, boolean, null)
  • The name of the type of the value (e.g. 'string', 'boolean', etc.)
  • The time-to-live that was set for the entry
  • The time the entry was set (to be used for last updated)
This value descriptor is being stringified (using JSON.stringify) and stored as the value of the entry.
The method structure that was chosen to support the various type was (where <Type> is replaced by the specific type, e.g. "Boolean"):
cache.put<Type>(String key, Type value, Number ttl) : Void
cache.get<Type>(String key) : Type
In each put & get method the key and value are verified to be of the correct type. The ttl parameter (i.e. time-to-live in seconds) is optional, same as in the native cache service.

Support For JavaScript Objects

Objects are a little bit more complex. Since the objects are stored also as strings, there is a need to allow custom methods for stringifying and parsing the object value (although in most cases JSON's default methods are enough, there are some cases such as the Date object where it is not). For this reason, both getObject and putObject methods take an optional parse/stringify method. If not specified, JSON's methods are used.

So, for example, in order to store and get a simple object:
cache.putObject('p1', { name: 'John', age: 30 });
var p = cache.getObject('p1');
The returned value is an object, stringified and parsed using the default methods.
In order to store an object for which the default methods are not enough, the following can be done (in this example, for the Date object):
var stringifyDate = function(o) { return '' + o.getTime(); };
cache.putObject('d1', new Date(), undefined, stringifyDate);
var parseDate = function(s) { return new Date(+s); };
var d = cache.getObject('d1', parseDate);

Support For Large Values

In order to support larger values but still use the existing cache service I have decided to split large values between several entries. String values are the main candidates for being too long (number, boolean and null have no chance of reaching the max size). Since objects can also be too large (and for some other reasons), I have decided to store them as strings. This allows me to use the same implementation for all relevant value types.

When storing an entry, the value in the value descriptor is checked - if it is a string too long, if so it needs to be split. In such cases the value is split to smaller parts, each is stored in a separate entry. The keys of the split entries are collected and stored as part of the value descriptor instead of the value itself.

When getting a value, the opposite operation is done. If the value descriptor has keys instead of values, the values are taken and rejoined to the original value. Only then the value is returned (or parsed in case of an object).


The code is available in GitHub:
https://github.com/yinonavraham/GoogleAppsScripts/tree/master/EnhancedCacheService

Trakt.TV & Subtitles - A Google Apps Script Project

Before I Get to the Technical Stuff

My wife and I like watching TV series, but we like choosing what and when, not what the satellite/cables providers dictate. This is why we mostly use XBMC for that. XBMC is great - you get all the new episodes from a variety of providers, quickly and with good quality. Since we're not native English speakers we also like having subtitles. XBMC is great for that as well.

A while ago, before we started using XBMC, I used to go once in a while browsing for new episodes, subtitles, etc. It was frustrating... XBMC solved most of it. I also found Trakt.TV a great tool for keeping track of what we haven't watched yet.

The only part that was left is keeping track of the subtitles availability. For that I had two options: start watching the episode, then try to find matching subtitles. The other option was to look for the subtitles in the providers' websites before that. Both options aren't nice. There are some series that we don't really mind watching without subtitles, but some we wont watch until they have subtitles. In those cases it is frustrating to go through all that work just to find there are no subtitles available...

This is why I started this project. The goal was to give a view which consolidates my shows' progress (from Trakt.TV) with the availability of subtitles for each un-watched episode. I chose Google Apps Script because of two reasons: 1- it is quite simple, it gives a variety of built-in services and it's free. 2- I did not know it and wanted to see what it has to offer.

Technical Overview

OK, so as I mentioned before, the project is based mainly on Google Apps Script. Although it has quite a lot native services, I had some gaps of missing functionality I needed to fill. The project currently includes the following modules/services:
  1. TrakTV & Subtitles-
    This script is published as a webapp. It is the main module, it implements the model generation and the UI rendering of the service.
  2. EnahancedCacheService- (post)
    This is a service for adding more features to GAS's native CacheService. It mainly supports various value types (not just string: e.g. number, boolean, object) and supports values larger than 128KB.
  3. TraktTVClient-
    This is a client for the Trakt.TV REST API. It currently implements only what is really needed for my needs, but it can be enhanced with much more quite easily if needed.
  4. SubsCenterOrgClient-
    This is a client for the SubsCenter.org subtitles provider. Since this provider has no API, so instead it parses the website - only what is currently required.
  5. OpenSubtitlesOrgClient-
    This is a client for the OpenSubtitles.org subtitles provider. This provider has an API based on XML-RPC.
  6. XmlRpcMessageService- (post)
    This is a service for creating and handling XML-RPC messages. It lets the user work with simple JavaScript objects instead of the verbose XML. Not all data types defined in the specification are supported by this implementation.
In the next posts I will elaborate on some of the services mentioned above.

Wednesday, June 11, 2014

"SQL like" Java Collections

In my daily work I do a lot of Java coding, but I also use SQL quite a lot.
I like SQL for its declarative nature. I think it is quite intuitive to spell out queries. In .NET there is LINQ which gives nice query capabilities. So I thought why not have similar API in Java - for querying collections. I tried to look for some existing libraries and found some. The closest solution I found was JoSQL. As stated in its website:
"JoSQL (SQL for Java Objects) provides the ability for a developer to apply a SQL statement to a collection of Java Objects. JoSQL provides the ability to search, order and group ANY Java objects and should be applied when you want to perform SQL-like queries on a collection of Java Objects."
JoSQL uses a Query object which gets a string with the SQL query. When it is called, the query is parsed and executed. I guess, but did not check, that reflection is used extensively there. It looks quite nice, but the main issue that I find with it is that it is error prone. The query validity is not enforced in design-time. If you have an error you will know it only in runtime, and there it is a matter of how much the error message is useful, and hopefully you caught it during development/QA.

With this issue I have decided to implement a different API which will give some query capabilities over collections and its validity will be evaluated in design-time. The main characteristics I chose for my API were:
  • Generics
  • Fluent API (i.e. method chaining)
  • Functional Programming
The library it-self is not complete, I did it only as a POC for myself. It is hosted on Google Code:
https://code.google.com/p/ynn-util-collections/wiki/SQLlikeCollections
Let's see some examples.
First, let's assume we have the following import statements:
import static ynn.util.collections.Predicates.*;
import static ynn.util.collections.SqlLikeCollections.*;
The first import statement imports some predefined static predicate methods such as "and", "or", "not", "lessThan", etc. The second import statement imports the entry point static methods for our queries.
Now that we have our basics imported we can write queries.

Examples


Example 1 -

Select all strings in a collection which are not null:
Collection<String> data = Arrays.asList(
    null, "A", "B", "C", null, "D", "E", "F", "G", "H", "I", null);
Collection<String> result = selectFrom(data).where(not(isNull()));
As you can see, this is quite a simple query. You can also notice that the result is typed and keeps the same type as the source (i.e. String). The syntax is quite similar to SQL, which would probably be something like:
SELECT * FROM "DATA" WHERE NOT "DATA"."VALUE" IS NULL

Let's examine some other more complex examples.

Example 2 -

In this example we will have a slightly more complex where clause and we'll also add some sorting. The API has some predefined predicates and utilities for Strings, so we'll import them as well:
import static ynn.util.collections.Predicates.StringPredicates.*;
Now for the example:
Collection<String> result = selectFrom(data)
    .where
        and(
            not(isNull()),
            stringStartsWith("A"),
            stringContains("a")
        )
    )
    .orderBy(LEXICOGRAPHIC_ORDER);
I this example you can see we select all strings which are not null, and it starts with "A", and it contains "a". The result is ordered by lexicographic order. Quite readable, right?

Of course the API is not limited to Strings or just simple objects. Let's see a more complex example.

Example 3 -

In this example we first need to define our Person class:
public class Person {

    private String name;
    private int age;
    private double weight;

    public Person() { }

    public Person(String name, int age, double weight) { 
        this.name = name;
        this.age = age;
        this.weight = weight;
    }

    // Getter & setter methods go here, I omitted them for simplicity
    // Also any other specific implementation (toString, ...)

    /*
     * VALUE PROVIDERS
     */

    public static ElementValueProvider<Person, String> name() {
        return new ElementValueProvider<Person, String>() {
            @Override
            public String getValue(Person element) {
                return element.getName();
            }
        };
    }
        
    public static ElementValueProvider<Person, Integer> age() {
        return new ElementValueProvider<Person, Integer>() {
            @Override
            public Integer getValue(Person element) {
                return element.getAge();
            }
        };
    }
	
}
You probably noticed the static methods under the "VALUE PROVIDERS" comment. Their purpose will be clearer in the example:
List<Person> data = Arrays.asList(
    new Person("John", 2, 2.2),
    new Person("David", 1, 1.1),
    new Person("Bob", 4, 4.4),
    new Person(null, 2, 7.7),
    new Person("Robin", 1, 3.3),
    new Person("Danny", 2, 5.5),
    new Person("Alice", 3, 6.6)
);

Collection<Person> result = 
    selectFrom(data)
    .where(
        not(isNull(Person.name())))
    .orderBy(
        desc(Person.age()), 
        asc(Person.name()));
As you can see above, we wrote a query for selecting all persons whom their name is not NULL. We then sorted the collection, first descending by age, then ascending by name. The corresponding SQL query would probably look something like:
SELECT * FROM "DATA"
WHERE NOT "PERSON"."NAME" IS NULL
ORDER BY "PERSON"."AGE" DESC, "PERSON"."NAME" ASC
Quite similar, right? And it's type safe as well!

One last example - I also started to play with the idea to support mass updates. See the example below.

Example 4 -

In this example we will update all persons with NULL name to have empty string for a name and their age to zero. For the example we need to add the following methods to our Person class:
    /*
     * UPDATERS
     */

    public static Updater<Person> nameTo(final String newName) {
        return new Updater<Person>() {
            @Override
            public void update(Person element) {
                element.setName(newName);
            }
        };
    }

    public static Updater<Person> ageTo(final int newAge) {
        return new Updater<Person>() {
            @Override
            public void update(Person element) {
                element.setAge(newAge);
            }
        };
    }
Now we can write our mass update statement:
update(data)
    .where(
        isNull(Person.name()))
    .set(
        Person.nameTo(""), 
        Person.ageTo(0));
See? nice, right? 

Conclusion

Of course there is a lot to extend, many capabilities missing, the performance of the current implementation can be easily improved, etc. This was a proof of concept. I think having such API can make developers' life easier, especially if they also write real SQL...
By-the-way - now that Java 1.8 is out, with all its new capabilities such as lambda expressions, method reference, functional interfaces, aggregate methods, streams, etc. It gives exactly what I was trying to accomplish here. The main difference is the syntax, but the usage is very similar.

Monday, May 26, 2014

myLOGO

One of the programming languages that I love is LOGO. Yes, it is a programming language, although it is limited for drawing. I started using it when I was about 7 years old. Using LOGO formed my perception of computers in general and of programming specifically.

One of the first project I did when I started learning C# by myself (when I was doing my B.Sc) was a tool for running LOGO commands and displaying the results - "myLOGO". It also supported definition of custom procedures (using "ED <proc-name> ... END"). You can download the project here (RAR file). Here is a screenshot:
myLOGO, implemented using C# and WinForms

Then, few years later I started focusing on Java. After a while I started developing plugins for Eclipse. This introduced me with SWT (Standard Widget Toolkit). As a result of my need for understanding SWT better, as a standalone, I decided to implement myLOGO again, only this time in Java & SWT. Doing so I also added support in loops (i.e. the REPEAT command). You can download the project here (ZIP file). Here is a screenshot:
myLOGO, implemented using Java & SWT
The code is hosted in GitHub: https://github.com/yinonavraham/myLOGO

It includes a parser, an AST (Abstract Syntax Tree), design-time and run-time models, UI, etc. I implemented it in a way it can be embedded as a library in other projects. I will elaborate in a future post on another project in which I did exactly that - an Eclipse plugin for developing LOGO scripts.

Griddlers Solver

You know these griddlers puzzles? Every day there is a new 2D 20x20 puzzle in the newspaper we were getting at work. So I decided to implement a solver for those problems.
Before I start talking, you can take a look at the Griddler Solver page. I have prepared some examples.

I chose client-side JavaScript as the runtime environment. Why? just because it's easy to use for coding the solver and it also integrates smoothly with a well known UI - HTML... Oh, and because I wanted to practice my JavaScript skills...
My first attempt with implementing the solver was using kind of DFS (Depth-First-Search). The algorithm was (roughly):
  1. Calculate possible solutions for each column
  2. Iterate recursively over the columns:
    1. Get the current column
    2. Iterate recursively over the possible solutions of the column:
      1. Get the current possible solution
      2. If this is the last column:
        1. If the problem is solved (the rows constraints are matched) - stop
This worked great for small problems (about 5x5), but for bigger problems the performance was horrible.
The next step I tried was to add some heuristics. For example, I added some heuristics for pruning branches which are not feasible. Some of the rules I used:

  • If the sum of colored cells in a row of the solution is higher than the some of colored cells in the constraint - this branch is not feasible
  • If the length of the current block is higher than its expected length - this branch is not feasible
With those rules the performance improved tremendously, but still not enough for the average problem (20x20). 

Then I decided to change direction and instead of implementing based on a generic algorithm (i.e. DFS), I decided to implement an algorithm which is based on the way a human will solve it. So, how do you solve these puzzles? I usually use an iterative solution:
  1. Find the large blocks and find overlaps - color them.
  2. Check for other constraints which are now easier to match, according to the new colored cells.
  3. If there is no single solution - find the minimal possible solutions, select one and try to move forward. If this fails, go back and select a different possible solution. (This step usually does not happen - you usually just need to look harder in steps #1 & #2).
What I have decided to implement was some variation of it. I decided to calculate all the possible solutions for all rows and columns and then do cross-checks for each cell and calculate its probability to be colored. In each step some possible solutions can be discarded since they do not match the decisions that were made during that step.
So the algorithm is basically:
  1. Initialize:
    1. Calculate all possible solutions for each row & column
    2. Initialize the grid so that each cell has an undefined value
  2. Solve - while the solution changes:
    1. Update probabilities:
      For each cell, calculate the probability for it to be colored by cross checking the possible solutions for both row and column - count the number of solutions where the cell is colored and not colored.
    2. Remove false solutions:
      For each cell, if the cell's value is known (i.e. 100% colored or empty) - remove all possible solution (columns and rows) where the cell's value is different from its expected value.
You can see above that I don't yet handle the case when there are multiple possible solutions which none of them can be removed - i.e. need to choose one of them and try to move forward. This is because I did not have such a case yet in any of the puzzles I tried. It's not that difficult to create such example, nor implement it, I just didn't... (for example: 2x2 puzzle with rows - [ [1], [1] ] , and columns - [ [1], [1] ]. Try it...).

In my implementation I implemented the solver as an Iterator which enabled me to add animation to the UI. Since in each step the grid has probabilities for each cell (for whether it is colored), it can make quite nice animations.

You can find the code in my GitHub project yinonavraham/GriddlerSolverJS.

Sunday, May 18, 2014

"My" proof for Euler's identity

Few years back, I was sitting in the classroom during my M.E. getting bored. Then I remembered of one of the most beautiful equations - Euler's identity:
\[e^{i\pi}+1=0\]
So I decided to prove it using Taylor series.
Yes, you can ask why? The answer is - because I was bored and because I can...

So, first a reminder - Taylor series definition:
\[
\sum_{n=0}^{\infty}\frac{f^{(n)}(a)\cdot(x-a)^n}{n!}
\]
Now, Taylor series for some basic functions (\(a=0\)):
\(

\begin{align*}
e^x &= e^0+\frac{e^0x}{1!}+\frac{e^0x^2}{2!}+\frac{e^0x^3}{3!}+\dots \\
&= 1+\frac{x}{1!}+\frac{x^2}{2!}+\frac{x^3}{3!}+\dots
\end{align*}
\)

\(
\begin{align*}sin(x) &= 0+\frac{cos(0)x}{1!}+\frac{-sin(0)x^2}{2!}+\frac{-cos(0)x^3}{3!}+\dots \\
&= \frac{x}{1!}-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\dots
\end{align*}
\)

\(
\begin{align*}cos(x) &= 1+\frac{-sin(0)x}{1!}+\frac{-cos(0)x^2}{2!}+\frac{sin(0)x^3}{3!}+\dots \\
&= 1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\dots
\end{align*}
\)

Let's start. If we assign \(x=ix\) in \(e^x\) we get:
\(
\begin{align*}
e^{ix} &= 1 + ix + \frac{(ix)^2}{2!} + \frac{(ix)^3}{3!} + \frac{(ix)^4}{4!} + \dots \\
&= 1 + ix - \frac{x^2}{2!} - \frac{ix^3}{3!} + \frac{x^4}{4!} + \frac{ix^5}{5!} - \frac{x^6}{6!} - \frac{ix^7}{7!} + \dots \\
\end{align*}
\)

If we reorder what we got, we get:
\(
\begin{align*}
e^{ix} &= \underbrace{1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \frac{x^6}{6!} + \dots}_\text{cos(x)} + i \cdot (\underbrace{x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \dots}_\text{sin(x)}) \\
&= cos(x) +i \cdot sin(x) \end{align*}
\)

Let's assign \( x = \pi \):
\(
\begin{align*}
e^{i\pi} &= cos(\pi) +i \cdot sin(\pi) \\
&= -1 + i \cdot 0 \\ &= -1 \end{align*}
\)

So, finally we got what we wished for:
\(
e^{i\pi} + 1 = 0
\)


OK, now - for all of you saying "of course - the identity is just a special case for Euler's formula: \(e^{ix}=cos(x) +i \cdot sin(x)\)" - I know and I "proved" it (well, it's not a real proof and it's not really mine to claim...). Oh, and I also wanted to write some \(\LaTeX\)...
So that was me goofing around out of boredom..


The \(\LaTeX\) in this post is powered by MathJax

Thursday, May 15, 2014

Using Java generics for proxy implementation

Overview

Java's generics is very powerful and easy to use. Its common use is straight forward, for example in the Collections API. In this post I would like to show another use, not so common, of generics. The purpose is to implement a proxy for remote class, e.g. when using RMI (Remote Method Invocation), JMX (Java Management Extensions), etc. This is handy since you usually have similar basic functionality in your proxy - connect, disconnect, etc.

Design

Class diagram of the participants in the design
The participants in the design - remote classes in green, local classes in blue (the interface is deployed remotely and locally)
As you can see in the class diagram above, there are 4 participants:
  • MyInterface -
    This is the interface of the remote class. This interface must be deployed both remotely and locally.
  • MyRemoteClass -
    This is the implementation of the remote class. It is deployed remotely and is the target for our proxy (in the client-side)
  • AbstractProxy -
    This is an abstract implementation of a proxy. It has some basic common proxy functionality such as connect, disconnect, etc. and it holds the actual proxy object encapsulated inside. This class is deployed locally (in the client-side).
  • MyProxy -
    This is the specific proxy for MyRemoteClass. It implements MyInterface and extends AbstractProxy setting the generic type to be MyInterface.

Implementation

Here is partial and simplified implementation of the classes in the design above.

MyInterface

public interface MyInterface {

    void doSomething();

}

MyRemoteClass

public class MyRemoteClass implements MyInterface {

    @Override
    public void doSomething() {
        // Do something...
    }

}

AbstractProxy

public abstract class AbstractProxy<T> {
 
    private T proxy;
 
    protected T getProxy() {
        return proxy;
    }
 
    public void connect() {
        // Connect to the remote class - set the encapsulated proxy object 
    }
 
    public void disconnect() {
        // Disconnect from the remote class - reset the encapsulated proxy object 
    }
 
    public boolean isConnected() {
        // Check whether the proxy is connected
        return true;
    }

}

MyProxy

public class MyProxy extends AbstractProxy<MyInterface> implements MyInterface {

    @Override
    public void doSomething() {
        // Delegate the call to the proxy
        getProxy().doSomething();
    }

}

Use Case Example

Here is a more realistic example of how I used this pattern (with some simplification and modification) for implementing an abstract proxy for JMX agents.

Background

(If you are not interested in the "why", you can skip to the Implementation part)

A few years ago, as part of a distributed system I was working on (school project...), I wanted to implement agents that will be deployed on the servers and listen for requests. This agents needed to answer 2 main requirements:

  1. Remote method invocation -
    The ability to connect from one machine to another and trigger actions on the remote server.
  2. Monitor the agents on the remote server -
    The ability to connect from a PC to the agent on the remote server and interact with it for monitoring, configuration, perform "manual" actions, etc.

In the beginning I chose Java RMI for the task. It's quite easy to use (in development), and once you run the server you're OK. But then, if you want to monitor it from a remote PC you need to develop a dedicated client UI for that. Therefore I looked for another technology that will give me similar capabilities (of Remote Method Invocation) using an API, but will also answer my other requirement with an existing UI (command line or graphical).
The answer I found was JMX (Java Management Extensions). Although this is not exactly the purpose of JMX, it was close enough and it answers my requirements. The UI for monitoring is JConsole (part of the SDK). It enables you to connect to a Java process (local or remote) and monitor its threads, memory, loaded classes, registered MBeans and more. In the MBeans section you can view & update the attributes each MBean exposes and invoke its methods. This is a simple generic UI which was quite enough for the requirements.

After implementing the MBeans I needed to implement the clients which will enable me to programmatically connect to the remote agents. When I started I noticed there is some basic code that I repeats itself in all clients, just with small differences. These are good candidates for refactoring.

Implementation


public abstract class AbstractJmxClient<MBeanInterface> {
    private MBeanInterface proxy = null;
    private JMXConnector conn = null;
    private MBeanServerConnection mbsc = null;

    // JMX properties which are used to connect to the registered MBean 
    private String jmxHost = null;
    private String jmxDomain = null;
    private String jmxKeyName = null;
    private String jmxKeyValue = null;
    private int    jmxPort = -1;

    // Constructor getting all the JMX properties...
    // Getters for all the JMX properties...

    protected MBeanInterface getProxy() {
        return _proxy;
    }

    /**
     * Connect to the remote JMX agent
     */
    @SuppressWarnings("unchecked")
    public void connect() throws Exception
    {
        String urlSuffix = null;
        try {
            urlSuffix = this.jmxHost + ":" + this.jmxPort + "/jmxrmi";
            ObjectName oName = new ObjectName(this.jmxDomain, this.jmxKeyName, this.jmxKeyValue);
            JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + urlSuffix);
            if (this.conn == null) {
                this.conn = JMXConnectorFactory.connect(url, null);
            }
            if (this.mbsc == null) {
                this.mbsc = this.conn.getMBeanServerConnection();
            }
            Type type = getClass().getGenericSuperclass();
            Class<MBeanInterface> mbiClass = null;
            if (type instanceof ParameterizedType) {
                ParameterizedType paramType = (ParameterizedType) type;
                mbiClass = (Class<MBeanInterface>) paramType.getActualTypeArguments()[0];
            }
            this.proxy = JMX.newMBeanProxy(this.mbsc, oName, mbiClass);
        } catch (RuntimeException e) {
            String errMsg = String.format(
                "Could not connect to the JMX agent: [%s:%s=%s] at %s due to the following error: %s",
                this.jmxDomain, this.jmxKeyName, this.jmxKeyValue, urlSuffix, e.getMessage());
            throw new Exception(errMsg, e);
        }
    }

    /**
     * Disconnect from the remote JMX agent
     */
    public void disconnect() {
        this.proxy = null;
        this.mbsc = null;
        if (this.conn != null) {
            try {
                this.conn.close();
            } catch (IOException e) {
                // Write to the trace
            }
            this.conn = null;
        }
    }

    /**
     * Check whether the client is connected to the remote JMX agent
     * 
     * @return true if the client is connected
     */
    public boolean isConnected() {
        if (this.proxy == null || this.conn == null) {
            return false;
        }
        try {
            this.conn.getConnectionId();
            return true;
        } catch (IOException e) {
            return false;
        }
    }

}