How to Serialize a Lambda

How to serialize a lambda?

Java 8 introduces the possibility to cast an object to an intersection of types by adding multiple bounds. In the case of serialization, it is therefore possible to write:

Runnable r = (Runnable & Serializable)() -> System.out.println("Serializable!");

And the lambda automagically becomes serializable.

How to safely serialize a lambda?

It depends on which kind of safety you want. It’s not the case that serialized lambdas cannot be shared between different JREs. They have a well defined persistent representation, the SerializedLambda. When you study, how it works, you’ll find that it relies on the presence of the defining class, which will have a special method that reconstructs the lambda.

What makes it unreliable is the dependency to compiler specific artifacts, e.g. the synthetic target method, which has some generated name, so simple changes like the insertion of another lambda expression or recompiling the class with a different compiler can break the compatibility to existing serialized lambda expression.

However, using manually written classes isn’t immune to this. Without an explicitly declared serialVersionUID, the default algorithm will calculate an id by hashing class artifacts, including private and synthetic ones, adding a similar compiler dependency. So the minimum to do, if you want reliable persistent forms, is to declare an explicit serialVersionUID.

Or you turn to the most robust form possible:

public enum IsNonEmpty implements Predicate<String> {
    INSTANCE;

    @Override
    public boolean test(String s) {
        return !s.isEmpty();
    }
}

Serializing this constant does not store any properties of the actual implementation, besides its class name (and the fact that it is an enum, of course) and a reference to the name of the constant. Upon deserialization, the actual unique instance of that name will be used.

Note that serializable lambda expressions may create security issues because they open an alternative way of getting hands on an object that allows to invoke the target methods. However, this applies to all serializable classes, as all variant shown in your question and this answer allow to deliberately deserialize an object allowing to invoke the encapsulated operation. But with explicit serializable classes, the author is usually more aware of this fact.

Java 8 Lambda expression with Serialization

You can create a serializable lambda expression via

Collections.sort(people, (Comparator<Person>&Serializable)
    (p1, p2) -> p1.getLastName().compareTo(p2.getLastName()));

but it should be noted, that creating a Comparator via

(p1, p2) -> p1.getLastName().compareTo(p2.getLastName())

bears a discouraged redundancy. You are calling getLastName() twice and have to care to invoke it on the right parameter variable in either case. It is more straight-forward to use

Comparator.comparing(Person::getLastName)

instead. You can also make this comparator serializable, though that implies loosing much of the conciseness:

Collections.sort(people,
    Comparator.comparing((Function<Person,String>&Serializable)Person::getLastName));

This is also more robust. The serialized form of a lambda expression contains a reference to the implementation method, which is in the first variant a synthetic method with a compiler generated name that might change when you use another lambda expression within the defining method. In contrast, Person::getLastName points the the named method getLastName as implementation method (at least with javac).

But generally, serializable lambda expressions might contain surprising compiler dependencies and should be used with care.

Since they are meant to describe behavior rather than data, there is no point in long-term storage of them anyway. For transferring them between JVMs with the same code base, they are sufficient.

How to serialize a lambda in kotlin

Kotlin lambdas are serializable by default, see https://discuss.kotlinlang.org/t/are-closures-serializable/1620.

So this will work:

 val r = { println("Hallo")} as java.io.Serializable

If you really need a Runnable then this does not work, because Kotlin creates only a Runnable instance:

 val r = Runnable { println("Hallo")} as java.io.Serializable

In this case you have to explicitly create an object:

val r = object: Runnable, java.io.Serializable {
    override fun run() :  Unit {
        println("Hallo")
    }
}

Serializing and Deserializing Lambda with Jackson

Jackson was created to keep object state not behaviour. This is why it tries to serialise POJO's properties using getters, setters, etc. Serialising lambdas break this idea. Theres is no any property to serialise, only a method which should be invoked. Serialising raw lambda object is really bad idea and you should redesign your app to avoid uses cases like this.

In your case SerializableRunnable interface extends java.io.Serializable which gives one option - Java Serialisation. Using java.io.ObjectOutputStream we can serialise lambda object to byte array and serialise it in JSON payload using Base64 encoding. Jackson supports this scenario providing writeBinary and getBinaryValue methods.

Simple example could look like below:

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;

public class JsonLambdaApp {
    public static void main(String[] args) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.INDENT_OUTPUT);

        SerializableRunnable action = () -> System.out.println("Serializable!");

        String json = mapper.writeValueAsString(new RuleMessage("1", action));
        System.out.println(json);

        RuleMessage ruleMessage = mapper.readValue(json, RuleMessage.class);
        ruleMessage.getsRunnable().run();
    }
}

@JsonSerialize(using = LambdaJsonSerializer.class)
@JsonDeserialize(using = LambdaJsonDeserializer.class)
interface SerializableRunnable extends Runnable, Serializable {
}

class LambdaJsonSerializer extends JsonSerializer<SerializableRunnable> {

    @Override
    public void serialize(SerializableRunnable value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
        try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
             ObjectOutputStream outputStream = new ObjectOutputStream(byteArrayOutputStream)) {
            outputStream.writeObject(value);
            gen.writeBinary(byteArrayOutputStream.toByteArray());
        }
    }
}

class LambdaJsonDeserializer extends JsonDeserializer<SerializableRunnable> {
    @Override
    public SerializableRunnable deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
        byte[] value = p.getBinaryValue();
        try (ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(value);
             ObjectInputStream inputStream = new ObjectInputStream(byteArrayInputStream)) {
            return (SerializableRunnable) inputStream.readObject();
        } catch (ClassNotFoundException e) {
            throw new IOException(e);
        }
    }
}

class RuleMessage {
    private String id;
    private SerializableRunnable sRunnable;

    @JsonCreator
    public RuleMessage(@JsonProperty("id") String id, @JsonProperty("sRunnable") SerializableRunnable sRunnable) {
        this.id = id;
        this.sRunnable = sRunnable;
    }

    public String getId() {
        return id;
    }

    public SerializableRunnable getsRunnable() {
        return sRunnable;
    }
}

Above code prints JSON:

{
  "id" : "1",
  "sRunnable" : "rO0ABXNyACFqYXZhLmxhbmcuaW52b2tlLlNlcmlhbGl6ZWRMYW1iZGFvYdCULCk2hQIACkkADmltcGxNZXRob2RLaW5kWwAMY2FwdHVyZWRBcmdzdAATW0xqYXZhL2xhbmcvT2JqZWN0O0wADmNhcHR1cmluZ0NsYXNzdAARTGphdmEvbGFuZy9DbGFzcztMABhmdW5jdGlvbmFsSW50ZXJmYWNlQ2xhc3N0ABJMamF2YS9sYW5nL1N0cmluZztMAB1mdW5jdGlvbmFsSW50ZXJmYWNlTWV0aG9kTmFtZXEAfgADTAAiZnVuY3Rpb25hbEludGVyZmFjZU1ldGhvZFNpZ25hdHVyZXEAfgADTAAJaW1wbENsYXNzcQB+AANMAA5pbXBsTWV0aG9kTmFtZXEAfgADTAATaW1wbE1ldGhvZFNpZ25hdHVyZXEAfgADTAAWaW5zdGFudGlhdGVkTWV0aG9kVHlwZXEAfgADeHAAAAAGdXIAE1tMamF2YS5sYW5nLk9iamVjdDuQzlifEHMpbAIAAHhwAAAAAHZyABxjb20uY2Vsb3hpdHkuSnNvblR5cGVJbmZvQXBwAAAAAAAAAAAAAAB4cHQAIWNvbS9jZWxveGl0eS9TZXJpYWxpemFibGVSdW5uYWJsZXQAA3J1bnQAAygpVnQAHGNvbS9jZWxveGl0eS9Kc29uVHlwZUluZm9BcHB0ABZsYW1iZGEkbWFpbiQ1YzRiNmEwOCQxcQB+AAtxAH4ACw=="
}

and lambda:

Serializable!

serializing and deserializing lambdas

Surprisingly, checking whether a lambda will work without its associated closure is actually fairly easy. According to the data model documentation, you can just check the func_closure attribute:


>>> def get_lambdas():
...     bar = 42
...     return (lambda: 1, lambda: bar)
...
>>> no_vars, vars = get_lambdas()
>>> print no_vars.func_closure
None
>>> print vars.func_closure
(<cell at 0x1020d3d70: int object at 0x7fc150413708>,)
>>> print vars.func_closure[0].cell_contents
42
>>>

Then serializing + loading the lambda is fairly straight forward:


>>> import marshal, types
>>> old = lambda: 42
>>> old_code_serialized = marshal.dumps(old.func_code)
>>> new_code = marshal.loads(old_code_serialized)
>>> new = types.FunctionType(new_code, globals())
>>> new()
42

It's worth taking a look at the documentation for the FunctionType:


function(code, globals[, name[, argdefs[, closure]]])

Create a function object from a code object and a dictionary.
The optional name string overrides the name from the code object.
The optional argdefs tuple specifies the default argument values.
The optional closure tuple supplies the bindings for free variables.

Notice that you can also supply a closure… Which means you might even be able to serialize the old function's closure then load it at the other end :)

Serialization of a lambda after its creation

This is correct, and by design. Just as you cannot take a non-serializable object and make it serializable after instantiation, once a lambda is created, its serializability is set.

A lambda is serializable if its target type is serializable (and its captured arguments are serializable.) Your first example is serializable because the target type is the intersection (Runnable & Serializable). Your two attempts to convert r fail because in both cases, r is a captured variable that is not serializable, and so the resulting lambda expression / method reference is not serializable. (The receiver for a bound method reference acts as a captured variable.)

Serialize Lambda Functions with Cereal

The binary value of a pointer in one processes address space is random bits in another processes address space.

Dynamic libraries are often loaded at literally random addresses (called address space randomization), and even when they are not they are loaded at dynamic addresses (which may by chance be the same address, until they are not because there was another library loaded there first).

Static functions are no better than lambdas.

You need an explicit table of functions guaranteed to be in the same order in both processes, and pass an index into that table.