Skip to content
Hardened JavaScript
Blog

Blog

SES 1.9.0 introduces immutable ArrayBuffers

SES 1.9.0 introduces immutable ArrayBuffer, exposes its console shim, and permits a global ModuleSource in the shared intrinsics.

Immutable ArrayBuffer

JavaScript strings are immutable, but JavaScript lacks an immutable analog for ArrayBuffer. So, one can have a string of text and send it without making a copy to defend its invariance, but not a string of bytes.

Agoric’s Mark Miller presented a proposal at the October plenary meeting of the JavaScript standard committee, ECMA TC39. The committee adopted Immutable Array Buffers into the body of problems under consideration for a solution in a future standard, called “Stage 1”.

SES 1.9.0 is the first release to introduce an emulation of the proposed immutable ArrayBuffer.

On platforms without Array.prototype.transfer but with a global structuredClone, the ses shim’s lockdown will now install an emulation of Array.prototype.transfer. On platforms with neither, ses will currently not install such an emulation. However, once we verify that ses is not intended to support platforms without both, we may change lockdown to throw, failing to lock down.

  • XS and Node >= 22 already have Array.prototype.transfer.
  • Node 18, Node 20, and all browsers have structuredClone
  • Node <= 16 have neither, but are also no longer supported by ses.

Console Shim

The ses shim consists of mostly separable layers:

  • ses/assert-shim.js
  • ses/lockdown-shim.js
  • ses/compartment-shim.js
  • ses/console-shim.js

Version 1.9.0 is the first release to expose an entry point for ses/console-shim.js so the user can choose whether to wrap and replace the global console from the hardened JavaScript realm with a version that can unredact error messages.

ModuleSource

Agoric champions a number of JavaScript standard proposals under the umbrellas of Hardened JavaScript, Compartments, and a broad effort toward “Module Harmony”. These include a proposal for a global ModuleSource that takes the source code of a JavaScript module and prepares it for evaluation.

The package @endo/module-source provides an emulation of a ModuleSource constructor suitable for use in combination with ses and Moddable’s XS provides a native implementation. The module @endo/module-source/shim.js can install the emulated ModuleSource in global scope so it becomes available as a shared intrinsic.

Version 1.9.0 adds ModuleSource to the shared intrinsics that lockdown will repair if necessary and then harden, which consequently is available by default in every new Compartment.

SES 1.8.0 adds flexibility to Lockdown

Agoric has released ses version 1.8.0 with more implementation-specific options to Hardened JavaScript’s lockdown() function to improve ecosystem compatibility.

Regenerator Runtime Compatibility

Old versions of some npm packages used regenerator to anticipate JavaScript’s async functions before language support was ubiquitous. These rely on regenerator-runtime to approximate the language feature. However, in the few cases where a dependency on versions 0.10.5 to 0.13.7 of regenerator-runtime persist, applications are incompatible with ses due to misalignment of the global objects the runtime introduces and the environment that Hardened JavaScript expects from the base language.

With SES 1.8.0, lockdown accepts a new option legacyRegeneratorRuntimeTaming: 'unsafe-ignore' that repairs the intrinsics shared by all compartments so that lockdown will allow them. This remediation is not strictly safe because it converts Iterator.prototype[Symbol.iterator] to a getter and setter, such that all assignments to it are ignored. This could cause code to procede on false assumption that it successfully overwrote the Iterator.prototype[Symbol.iterator] instead of throwing an error.

Error Trapping: Report

Starting with SES 1.8.0, the 'report' mode for the errorTrapping option to lockdown will write errors to standard error with the new "SES_UNCAUGHT_EXCEPTION: " prefix. The 'report' mode is sometimes implied by 'platform', 'exit', or 'abort'. This is intended to give valuable context to users of the system, especially when an uncaught exception is not an Error object, and therefore its origin may be hard to find in source code.

This is not likely to affect most systems built with SES, as stderr is generally reserved for user-only messages. If your SES system sends its stderr to a program which parses it, you may need to adapt that program to be tolerant of the SES_UNCAUGHT_EXCEPTION: prefix. Even for such programs, it is unlikely they are that sensitive to stderr formatting.

SES 1.6.0 improves compatibility with XS

Agoric has released ses version 1.6.0 with changes that improve compatibility with Moddable’s XS. The version maintains backward-compatibility with existing usage and some usage becomes deprecated, for which support will be removed in a future, major version.

__options__

With this change, the Compartment constructor now has a preferred single-argument form, which will become the only accepted form in a future, major version. Until then, Compartment accepts both the old and new signatures.

The deprecated form accepts three optional arguments:

const compartment = new Compartment(globals, modules, {
  name: 'my-compartment',
});

The new form accepts a single options bag, and currently requires the __options__ option to distinguish the global endowments from an options object.

This is technically a breaking change, but we expect that it is vanishingly rare for a compartment to be endowed with an __options__ global. If such code exists, it will need to adopt the new form.

const compartment = new Compartment({
  __options__: true,
  name: 'my-compartment',
  globals,
  modules,
});

In a future, major version, the __options__ field will become vestigial and the three-argument form will be an error.

const compartment = new Compartment({
  name: 'my-compartment',
  globals,
  modules,
});

__noNamespaceBox__

Likewise, the import method of compartments currently returns a promise for an object with a namespace property, which is the namespace. This differs from XS and the behavior of standard, dynamic import.

const compartment = new Compartment({
  __options__: true,
  modules: {
    'my-module-specifier': {
      namespace: {default: 42},
    },
  }
});
// Note the destructuring of namespace:
const { namespace } = await compartment.import('my-module-specifier');
console.log(namespace.default); // 42

The import method should simply return a promise for the module namespace object, just like dynamic import and XS.

Version 1.6.0 introduces a __noNamespaceBox__ option that makes the eventual correct behavior available today.

const compartment = new Compartment({
  __options__: true,
  __noNamespaceBox__: true,
  modules: {
    'my-module-specifier': {
      namespace: {default: 42},
    },
  }
});
// Note the absence of destructuring { namespace }:
const namespace = await compartment.import('my-module-specifier');
console.log(namespace.default); // 42

This will become the only supported behavior in a future, major version.

Module descriptors

Version 1.6.0 increases module descriptor parity with XS. Module descriptors appear in several places in the Compartment interface.

  • The modules option is an object that maps module specifiers to their corresponding module descriptor. These modules are “pre-loaded” at time of construction.
  • The moduleMapHook is a function that accepts a module specifier and may return a module descriptor if import or importNow cannot complete without it.
  • The importHookNowHook accepts a module specifier and may return a module descriptor if importNow cannot complete without it.
  • The importHook accepts a module specifier and may return a module descriptor or a promise for a module descriptor if import cannot complete without it.

With version 1.6.0, module descriptors can take most of the forms accepted by XS and code should begin migrating to prefer those forms.

  • Prior to version 1.6.0, a StaticModuleRecord, ModuleSource, virtual static module record, or virtual module source could be used in place of a module descriptor. These should now be boxed in a module descriptor with a source property. (Relatedly, we have renamed the @endo/static-module-record package to @endo/module-source to better reflect the direction of movement in the ECMAScript standardization process.)

    // before:
    import { StaticModuleRecord } from '@endo/static-module-record';
    const compartment = new Compartment(
      {},
      {
        'my-module-specifier': new StaticModuleRecord(`
          export default 42;
        `),
      },
    );
    console.log(compartment.importNow('my-module-specifier').default); // 42
    // after
    const compartment = new Compartment({
      __options__: true,
      modules: {
        'my-module-specifier': {
          source: new ModuleSource(`
            export default 42;
          `),
        },
      },
    });
    console.log(compartment.importNow('my-module-specifier').default); // 42

  • Prior to version 1.6.0, a module namespace object could be used in place of a module descriptor. Particularly, the Compartment implementation included a module method that could be used to acquire the module namespace object of a module that has not yet been loaded, such that a module namespace could be used to link compartments or settle cyclic dependencies. This practice is deprecated now in favor of using a namespace module descriptor.

    // before:
    const c1 = new Compartment(
      {},
      {
        'c1-module-specifier': new StaticModuleRecord(`
          export default 42;
        `),
      },
    );
    const c2 = new Compartment(
      {},
      {
        'c2-module-specifier': c1.module('c1-module-specifier'),
      },
    );
    console.log(c2.importNow('c2-module-specifier').default); // 42
    // after
    const c1 = new Compartment({
      __options__: true,
      modules: {
        'c1-module-specifier': new ModuleSource(`
          export default 42;
        `),
      },
    });
    const c2 = new Compartment({
      __options__: true,
      modules: {
        'c2-module-specifier': {
          namespace: 'c1-module-specifier',
          compartment: c1,
        },
      },
    });
    console.log(c2.importNow('c2-module-specifier').default); // 42

    Module descriptors with the source key will construct a fresh instance of the source in this compartment. Module descriptors with the namespace key will link to a module instance in the designated compartment.

  • Prior to version 1.6.0, a module descriptor with record property could be used to instantiate a module with a different import.meta.url than the associated module specifier. These descriptors should migrate to namespace module descriptors and must explicitly designate the compartment property if they previously relied on the default compartment. The default compartment for namespace and source descriptors is the “parent compartment”: the compartment for which Compartment is the initial, intrinsic compartment.globalThis.Compartment. The default compartment for record descriptors is child compartment.

    Because you can otherwise only refer to a compartment with a reference to that compartment, source and namespace descriptors that refer back to their own compartment, instead of the default parent compartment, are not expressible in the modules option and must move to a hook like moduleMapHook, importHook, or importNowHook.

    // before:
    const compartment = new Compartment(
      {},
      {
        'submodule/dependency': new StaticModuleRecord(`
          export default 42;
        `),
        dependent: {
          record: new StaticModuleRecord(`
            import meaning from 'dependency';
            export default meaning;
          `),
          specifier: 'submodule/dependent',
        },
      },
      {
        resolveHook(importSpecifier, referrerSpecifier) {
          const path = referrerSpecifier.split('/');
          path.pop();
          path.push(...importSpecifier.split('/'));
          return path.join('/');
        },
      },
    );
    console.log(compartment.importNow('dependent').default); // 42
    // after
    const compartment = new Compartment({
      __options__: true,
      modules: {
        'submodule/dependency': new ModuleSource(`
          export default 42;
        `),
      },
      importNowHook(specifier) {
        if (specifier === 'dependent') {
          return {
            source: new ModuleSource(`
              import meaning from 'dependency';
              export default meaning;
            `),
            specifier: 'submodule/dependent',
            compartment, // reflexive
          };
        }
      },
      resolveHook(importSpecifier, referrerSpecifier) {
        const path = referrerSpecifier.split('/');
        path.pop();
        path.push(...importSpecifier.split('/'));
        return path.join('/');
      },
    });
    console.log(compartment.importNow('dependent').default); // 42

Support for the deprecated forms will be removed in a future, major version.

Line numbers

When running transpiled code on Node, the SES error taming gives line-numbers into the generated JavaScript, which often don’t match the the original lines. This happens even with the normal development-time lockdown options setting,

errorTaming: 'unsafe'

or setting the environment variable

Terminal window
$ export LOCKDOWN_ERROR_TAMING=unsafe

To get the original line numbers, this release adds 'unsafe-debug'. This errorTaming: 'unsafe-debug' setting should be used during development only when you can sacrifice more security for a better debugging experience, as explained at errorTaming Options. With this setting, when running transpiled code on Node (e.g. tests written in TypeScript), the stacktrace line-numbers point back into the original source, as they do on Node without SES.