MethodMissing in Javascript

If one compares ES6 to Groovy (that for me is, based on some playing with it years ago, the most expressive language that I can think of) one could spot some missing features, like methodMissing (present in many other languages) or the superclean interception via invokeMethod. The truth is that you can get both features by using proxies and a get trap.

I'll use the term "class" regardless of whether we use ES6 class syntax or the "traditional" style. Proxies looked powerful to me, but I used to see a main problem with them: having to create a proxy for each object for which I want the feature. I'd like to do it directly at the class level. Well, as everything in javascript is an object, you just have to proxy the correct object, the prototype. You'll see that people use this technique to get other features, like multiple inheritance. I've cooked a sample of how to implement method missing.

My first approach was just to wrap "MyClass.prototype" in a proxy and reassign it, like this:

let aux = MyClass.prototype;
MyClass.prototype = new Proxy(aux, {
 //your get trap here
});

This does not work because when using the class syntax, the prototype of the constructor function is neither writable nor configurable, so you can not do the reassignment. To work around it we have to create a subclass using the old syntax so that we can do the prototype reassignment. So if I have an Employee class that I want to do "method missing aware" I'll define a sort of subclass like this.

class Employee{
 //whatever
}

function EmployeeWithMethodMissing(name){
  //Employee.call(this); not allowed to call a constructor other than with new
  let emp = new Employee(name);
  Reflect.setPrototypeOf(emp, EmployeeWithMethodMissing.prototype);
  return emp;
 }
EmployeeWithMethodMissing.prototype = new Employee();
EmployeeWithMethodMissing.prototype.constructor = EmployeeWithMethodMissing;

And I define a function that will receive a constructor function and wrap its prototype in a proxy that adds the method missing ability to it

//manageMissingItem: function that will be called when an access to a missing property takes place
function addMethodMissingCapabilityToClass(classConstructor, manageMissingItem){
 let _proto = classConstructor.prototype;
 classConstructor.prototype = new Proxy(_proto, {
  get: function(target, key, receiver){
   //console.log("get trap invoked");
   
   if (key in target){
    return target[key];
   }
   else{
    //the problem here is that I can not know if they were asking for a method of for a data field
    //so this makes sense for a missing method, but for missing properties it does not
    return manageMissingItem(target, key);
   }
  }
 });
}

The function above receives as second parameter a function where we will define specific behaviours for when a missing method call happens. I use it below to enable synomymous (alias) to a method. I consider calls to the missing methos "tell" or "speak" as if they were calls to "say"

addMethodMissingCapabilityToClass(EmployeeWithMethodMissing, 
  //manageMissingItem function
  function(target, key){
   if (key === "tell" || key === "speak"){
    console.log(`method missing, but ${key} is synonymous with "say"`);
    return function(...args){
     console.log("calling the returned function with " + this.name);
     return this.say(...args);
    };
   }
   else{
    return undefined;
   }
  }
 );
 
 let e1 = new  EmployeeWithMethodMissing("Laurent");

 console.log(e1.doWork("task1", "task2"));
 console.log(e1.say("Hi"));
 console.log(e1.tell("Hi Again"));
 

I've put it all in a gist

This technique works fine when you have a usage sample like the above, you have a restricted set of possible names of the missing methods. If you just want to trap any missing method invokation you have a problem. In javascript method invokation is done in 2 steps. First the function for that method is retrieved (we get it) and then it's invoked (with the corresponding "this" and parameters). This means that in our get trap we can not know if what is being retrieved is a function or a data field. We can not make the distinction that we make in groovy between methodMissing and propertyMissing. If we return a function for just any "missing retrieval", we'll have a problem for those cases where what they were expecting to get were data. The consumer would be expecting for example a string (or undefined) and we are returning a function that will not get invoked, as he was just accessing data only the retrieval happens, but not the invokation.

Trick: return a function with a "missingMethodReturn" property, so we can dosomething like: let data = p.dataField; data = data.missingMethodReturn ? undefined : data; -->

String Interpolation

Both ES6 and C# 6 feature string interpolation. It was an oddly missing feature bearing in mind that it has been present for so long in other languages (e.g. perl). The feature is nice, but don't fool yourself thinking that it can do more than it really can. It's well explained here. Though the article is about C#, the same limitation applies to javascript.

The basic idea is that you can not use string interpolation for a string that is defined outside of the source code (for example in a template that you get from disk or via htttp request). In C# the the compiler takes care of parsing the string literal into a FormattableString (you can think of it as doing something similar to String.Format), so it needs to know what variables to use for the place holders at compile time. In javascript compilation happens at the time of running, but the idea is the same, you need the compiler to parse the string, and for that the string has to be declared as a literal in code.

Well, I guess the solution is clear, if we need the compiler to work with a string that is not defined in code, we'll just have to call the compiler ourselves from our code.

The solution in javascript is so easy as using eval (yes, but sure don't use eval in a user provided string, it could contain malicious code).

let template = "hello ${user}";

function parseTemplate(template, user){
 return eval("`" + template + "`");
};

parsedTemplate = parseTemplate(template, "xose");
console.log(parsedTemplate);

In C#, we can use any of the existing options for compiling code at runtime. The Roslyn scripting is pretty straightforward.

public class Globals2
{
 public string Val1;
 public string Val2;
}

static string ParseStringTemplate(string st, string val1, string val2)
{
 var gl = new Globals2
 {
  Val1 = val1,
  Val2 = val2
 };
 string result = CSharpScript.EvaluateAsync<string>("$\"" + st + "\"", globals:gl).Result;
 return result;
}

var template = "This is a test with val1: {Val1} and val2: {Val2}";
 var parsed = ParseStringTemplate(template, "first", "second");

Notice however that Roslyn scripting is not an ideal solution as it will compile the code into a new assembly that will be loaded in the current Application Domain and hence will never be unloaded until the application exits. If you are usingthis it'll have an effect in the memory footprint. To avoid this we could use LCG or Expression Trees, maybe one day I'll give it a try, right now I don't feel like going low level. Another option that seems pretty more comfortable is using cs-script. Its script engine will use CodeDom and will load the compiled assembly in a separate Application Domain than then can be unloaded. Problem is that making it work with C# 6 is a bit tricky due to the stupid decisions taken by Microsoft regarding CodeDom Providers for C# 6.

ES6 and Super

Exploring ES6 is an amazing book. I've been going through some of its chapters randomly and at different levels of depth. It lately caught my attention that in ES6 the syntax for defining methods in one class can be also used in object literals. This means that you can write this:

let user = {
	sayHi: function(st){
	......
	}
};

or this:

let user = {
	sayHi(st){
	......
	}
};

They are almost the same, but not exactly. The first point is to realize that in ES6 we have different kinds of callable entities, though all of them are functions. Reading this chapter these callables are instances of function, but on creation (depending on whether it's a method, an arrow function...) they will get different internal data and methods ([[Construct]], [[HomeObject]]... We could say that each particular callable that we create in our code (via function, arrow, method definition) are instances of different function types.

For method declarations, we can read in this other chapter:

You can’t move a method that uses super: Such a method has the internal slot [[HomeObject]] that ties it to the object it was created in. If you move it via an assignment, it will continue to refer to the superproperties of the original object. In future ECMAScript versions, there may be a way to transfer such a method, too.

I've done a small test to verify that I had got it right:

let stringManagerABC = {
		process(st){
		st = st + " [ProcessingABC applied]";
		return st;		
	}
};

//a normal case, we can say that stringManagerDEF -> stringManagerABC 
let stringManagerDEF = {
	process(st){
		st = super.process(st) + " [ProcessingDEF applied]";
		return st;		
	}
};

Object.setPrototypeOf(stringManagerDEF, stringManagerABC); 
//
console.log(stringManagerDEF.validate("text"));

console.log("------------------------------");

let stringManager123 = {
	process(st){
		st = st + " [Processing123 applied]";
		return st;		
	}
};

//a rather particular case, stringManagerMixed -> stringManager123, but for the process method we are directly giving it the one from stringManagerDEF 
let stringManagerMixed = {
};


Object.setPrototypeOf(stringManagerMixed, stringManager123); 

stringManagerMixed.process = stringManagerDEF.process;

console.log(stringManagerMixed.process("text"));
//text [ProcessingABC applied] [ProcessingDEF applied]
//Not good, it's calling the method in the original parent rather than in the real parent

My understanding from some additional reading is that the [[HomeObject]] field of one method points to the object where the function has been attached at creation time (MyClass.prototype when defined in a class or the "this" at the definition time when in an object literal), so it's static. Finding a generic way to obtain [[HomeObject]] in a dynamic way does not seem simple. One could think of using "this", hasOwnProperty and getPrototypeOf, but that would only work if the inheritance chain has only 2 levels. With more levels we would be going down again in the prototype chain. Of course we could forget about this [[HomeObject]] thing and directly do the super calls by means of MyParentClass.prototype.method. That is not good either, for this same case of moving a method to another object it does not apply, and it would fail in cases like the mixins that we saw last week (you are dynamically creating subclasses).
For the especific case of the sample above, we could use something like the below, but it does not extend to other situations:

let stringManagerABC = {
	validate: function(st){
		st = st + " [ValidationABC applied]";
		return st;		
	}
};

//a normal case, we can say that stringManagerDEF -> stringManagerABC 
let stringManagerDEF = {
	validate: function(st){
		let _proto = Object.getPrototypeOf(this);
		if (_proto.validate){
			st = _proto.validate(st);
		}
		//I can not use super in a method declared via function
		//st = super.validate(st) + " [ValidationDEF applied]";
		return st + " [ValidationDEF applied]";		
	}
};

Object.setPrototypeOf(stringManagerDEF, stringManagerABC); 
console.log(stringManagerDEF.validate("text"));

console.log("------------------------------");

let stringManager123 = {
	validate: function(st){
		st = st + " [Validation123 applied]";
		return st;		
	}
};

//a rather particular case, stringManagerMixed -> stringManager123, but for the process method we are directly giving it the one from stringManagerDEF 
let stringManagerMixed = {
};


Object.setPrototypeOf(stringManagerMixed, stringManager123); 

stringManagerMixed.validate = stringManagerDEF.validate;

console.log(stringManagerMixed.validate("text"));
//text [Validation123 applied] [ValidationDEF applied]
//so this is good, I'm calling to the method in my real parent

ES6 Mixins

This article about Mixins in ES6 has been one of the best technical reads in a good while. It's pretty amazing how straight forward it's to create mixins thanks to ES6 class expressions, and how much better this approach is (you can use "super" for example) than the old one of augmenting existing objects. Read the article, it's an eye opener.

Bearing in mind that ES6 classes are mainly sugar to create prototype chains, one question arises then, how could we do this with "traditional" javascript?

First let's think a bit of how this new technique works. Both at the "sugar level - class level" and the prototype level, Javascript's has a single inheritance model. Each object in the prototype chain points only to another object. What we do here when defining a class made up by adding mixins to another existing class, is to put these mixins one after another in the new prototype chain. So each time we mixin the same block of behaviour to a different class we have to create a new mixing object holding that behavior to add it into this specific prototype chain. That's the key, we are defining the mixins by means of factory functions that create new classes (subclasses), classes that then we mix into the inheritance chain . I'd never thought of mixins exacty in this way. The definition given in the article is excellent:

A mixin is an abstract subclass; i.e. a subclass definition that may be applied to different superclasses to create a related family of modified classes.

The first point is just to check that this is really working this way. Let's create a new class based on some mixins and print the prototype chain. I'm using named class expressions for the mixins, so that I get something meaningful when printing the prototype chain with my simple traversal function.

function printPrototypeChain(obj){
 let protoNames = [];
 let curProto = Object.getPrototypeOf(obj);
 while (curProto){
  protoNames.push(curProto.constructor.name);
  curProto = Object.getPrototypeOf(curProto);
 }
 console.log("prototype chain:\n" + protoNames.join("."));
}

function testNewStyleMixins(){
 console.log("- testNewStyleMixins");
 var calculatorMixin = Base => class __calculatorMixinGeneratedSubclass extends Base {
   calc() { }
 };

 var randomizerMixin = Base => class __randomizerMixinGeneratedSubclass extends Base {
   randomize() { }
 };

 //A class that uses these mix-ins can then be written like this:

 class Foo { }

 class Bar extends calculatorMixin(randomizerMixin(Foo)) { }

 let bar1 = new Bar();

 printPrototypeChain(bar1);
 //Bar.__calculatorMixinGeneratedSubclass.__randomizerMixinGeneratedSubclass.Foo.Object
}

To set up something like this in "traditional" javascript, we'll use factories of functions (constructor functions that we turn into "classes" by attacching functions to its prototype). The previous example translates to this:

function testOldStyleMixins(){
 console.log("- testOldStyleMixins");
 function Foo(){
 }

 function CalculatorMixin(fn){
  let _calculatorMixin = function _calculatorMixinGeneratedFunction(){
   
  };
  _calculatorMixin.prototype = new fn();
  _calculatorMixin.prototype.constructor = _calculatorMixin;
  _calculatorMixin.prototype.calc = function(){};
  return _calculatorMixin;
 }


 function RandomizerMixin(fn){
  let _randomizerMixin = function __randomizerMixinGeneratedFunction(){
   
  };
  _randomizerMixin.prototype = new fn();
  _randomizerMixin.prototype.constructor = _randomizerMixin;
  _randomizerMixin.prototype.randomize = function(){};
  return _randomizerMixin;
 }

 function Bar(){
 };
 //WATCH OUT!!! if I do "new CalculatorMixin(RandomizerMixin(Foo));" I get a completely different thing!!!
 Bar.prototype = new (CalculatorMixin(RandomizerMixin(Foo)));

 Bar.prototype.constructor = Bar;

 let bar1 = new Bar();
 printPrototypeChain(bar1);
 //Bar._calculatorMixinGeneratedFunction.__randomizerMixinGeneratedFunction.Foo.Object

}

It's important to note that with ES6 mixins the access to properties and method in the base classes via super works perfectly fine. This is something to bear in mind, while ES6 classes are for the most part syntactic sugar, in order to make them work ES6 added new internal structures, like target.new and [[HomeObject]], as explained in the fantastic Exploring ES6

I got a bit confused when in the article he gives examples of inheriting and composing mixins. I'll add some notes to his examples for further reference.

Applying multiple Mixins. This is just the normal use. I'm creating a new class that extends the class resulting from mixing into MyBaseClass Mixin1 and Mixin2

class MyClass extends Mixin2(Mixin1(MyBaseClass)) { 
 //specific methods here
 //... 
}

Mixin Inheritance. Here I create a new class factory function (Mixin3). It applies other 2 existing mixins to the provided subclass and adds new methods to it. For that it's indeed creating an additional class in the prototype chain: ClassA -> Mixin3GeneratedSubclass -> Mixin2GeneratedSubClass -> Mixin1GeneratedSubclass -> superclass

let Mixin3 = (superclass) => class Mixin3GeneratedSubclass extends Mixin2(Mixin1(superclass)) {  
 //specific methods here
 //...
}

Mixin Composition. Here I create a new class factory function by combining 2 existing mixins, but it's not adding new methods of its own. That's the difference with the previous case, it does not need to add one more class. When used we'll get: ClassA -> Mixin2GeneratedSubClass -> Mixin1GeneratedSubclass -> superclass.

let Mixin3 = (superclass) => Mixin2(Mixin1(superclass));  

I have put a sample in this gist

There's a follow up article that moves into more advanced features, like cacing and making instaceof work with them. Real food for brain.