Eventual Send with E()

In web browsers, a common pattern of remote communication is using the asynchronous fetch API with promises:

const init = fetch('products.json')
  .then(response => response.json())
  .then(products => initialize(products))
  .catch(err => {
    console.log(`Fetch problem: ${err.message}`);
  });

In the Agoric platform, communicating with remote objects is similar, using the E() wrapper. For example, a deploy script may want to use the Zoe Service API to install a contract on a blockchain. But the deploy script cannot call zoe.install(bundle), because it does not have local access to the zoe object. However, the deploy script is given a zoe remote presence. To call methods on the actual Zoe object, the deploy script can do:

import { E } from '@endo/eventual-send';

E(zoe).install(bundle)
  .then(installationHandle => { ... })
  .catch(err => { ... });

Eventual Send

One of the ways Zoe partitions risk is by running in its own vat, separate from any smart contract that might use too much compute time or heap space. The smart contracts also run in separate vats.

What happens when we call E(zoe).install(bundle) is an eventual send:

1. A message consisting of the method name install with the bundle argument marshaled to a flat string and queued for delivery to the vat that zoe comes from.
2. E(zoe).install(bundle) returns a promise for the result.
3. The then and catch methods queue callbacks for when the promise is resolved or rejected. Execution continues until the stack is empty and thus this turn through the event loop completes.
4. Eventually zoe responds, which results in a new message in this vat's message queue and a new turn through the event loop. The message is de-serialized and the results are passed to the relevant callback.

This way, you can communicate with objects in separate vats as easily as objects in the same vat with one wrinkle: the communication must be asynchronous.

The E() wrapper works with:

• Remote presences (local proxies for objects in remote vats).
• Local objects (in the same vat).
• Promises for remote presences or local objects.

In all cases, E(x).method(...args) returns a promise.

Promise Pipelining

Since E() accepts promises, we can compose eventual sends: E(E(object1).method1(...args1)).method2(...args2). This way we can take advantage of promise pipelining so that a single round trip suffices for both method calls.

Troubleshooting remote calls

The E() function creates a forwarder that doesn't know what methods the remote object has. If you misspell or incorrectly capitalize the method name, the local environment can't tell you've done so. You'll only find out at runtime when the remote object complains that it doesn't know that method.

If an ordinary synchronous call (obj.method()) fails because the method doesn't exist, the obj may be remote, in which case E(obj).method() might work.

Testing with Deep Stacks

To get stack traces that cross vats:

TRACK_TURNS=enabled DEBUG=track-turns yarn test test/contract.test.js

See:

• $DEBUG
• $TRACK_TURNS
• $LOCKDOWN_OPTIONS for better diagnositcs

E() and Marshal: A Closer Look

Watch: Office Hours Discussion of Marshal

• Office Hours: ... marshalling w/board ids, ... #8069 July 2023

If you just want to use the SDK to write smart contracts, feel free to skip this section. But in case you're working on something that requires more detailed understanding, let's take a look at how E(x).method(...args) is marshalled.

In @endo/marshal docs, we see:

The marshal module helps with conversion of "capability-bearing data", in which some portion of the structured input represents "pass-by-proxy" or "pass-by-presence" objects that should be serialized into values referencing special "slot identifiers". The toCapData() function returns a "CapData" structure: an object with a body containing a serialization of the input data, and a slots array holding the slot identifiers. fromCapData() takes this CapData structure and returns the object graph. There is no generic way to convert between pass-by-presence objects and slot identifiers, so the marshaller is parameterized with a pair of functions to create the slot identifiers and turn them back into proxies/presences.

For example, we can marshal a remotable counter using the slot identifier c1:

const makeCounter = () => {
  let count = 0;
  return Far('counter', {
    incr: () => (count += 1),
    decr: () => (count -= 1),
  });
};

const counter1 = makeCounter();
t.is(passStyleOf(counter1), 'remotable');

const valToSlot = new Map([[counter1, 'c1']]);
const slotToVal = new Map([['c1', counter1]]);
const convertValToSlot = v => valToSlot.get(v);
const convertSlotToVal = (slot, _iface) => slotToVal.get(slot);
const m = makeMarshal(convertValToSlot, convertSlotToVal, smallCaps);

const capData = m.toCapData(counter1);
t.deepEqual(capData, {
  body: '#"$0.Alleged: counter"',
  slots: ['c1'],
});
t.deepEqual(m.fromCapData(capData), counter1);

Each end of a connection between vats typically keeps a table to translate slots to capabilities and back:

const makeSlot1 = (val, serial) => {
  const prefix = Promise.resolve(val) === val ? 'promise' : 'object';
  return `${prefix}${serial}`;
};

const makeTranslationTable = (makeSlot, makeVal) => {
  const valToSlot = new Map();
  const slotToVal = new Map();

  const convertValToSlot = val => {
    if (valToSlot.has(val)) return valToSlot.get(val);
    const slot = makeSlot(val, valToSlot.size);
    valToSlot.set(val, slot);
    slotToVal.set(slot, val);
    return slot;
  };

  const convertSlotToVal = (slot, iface) => {
    if (slotToVal.has(slot)) return slotToVal.get(slot);
    if (makeVal) {
      const val = makeVal(slot, iface);
      valToSlot.set(val, slot);
      slotToVal.set(slot, val);
      return val;
    }
    throw Error(`no such ${iface}: ${slot}`);
  };

  return harden({ convertValToSlot, convertSlotToVal });
};

Each call to E(rx) makes a proxy for the reciver rx; each E(rx).p property reference invokes the get proxy trap. From the get trap, E returns a function that queues rx, p, and its arguments (in marshalled form) and returns a promise:

const { convertValToSlot, convertSlotToVal } = makeTranslationTable(
  makeSlot1,
);
const m = makeMarshal(convertValToSlot, convertSlotToVal, smallCaps);

const outgoingMessageQueue = [];
// E work-alike for illustration
const E2 = obj =>
  new Proxy(obj, {
    get: (target, method) => (...args) => {
      const msg = harden([target, [method, args]]);
      outgoingMessageQueue.push(m.toCapData(msg));
      return new Promise(_resolve => {});
    },
  });

Now we can see the result in some detail. Note the way the promise from E(zoe).install() is passed to E(zoe).startInstance().

const zoe = Far('ZoeService', {});
const bundle = { bundleFormat: 'xyz' };
const installationP = E2(zoe).install(bundle);
const startP = E2(zoe).startInstance(installationP);
harden(startP); // suppress usage lint
t.deepEqual(outgoingMessageQueue, [
  {
    body:
      '#["$0.Alleged: ZoeService",["install",[{"bundleFormat":"xyz"}]]]',
    slots: ['object0'],
  },
  {
    body: '#["$0.Alleged: ZoeService",["startInstance",["&1"]]]',
    slots: ['object0', 'promise1'],
  },
]);

Watch: How Agoric Solves Reentrancy Hazards (November 2020)

for more on eventual send and remote communication