Skip to content

Commit cdb631e

Browse files
legendecasaduh95
legendecas
authored and
aduh95
committed
esm: add experimental support for addon modules
PR-URL: #55844 Fixes: #40541 Fixes: #55821 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
1 parent 3a3f5c9 commit cdb631e

20 files changed

+331
-26
lines changed

‎doc/api/cli.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,11 @@ Otherwise, the file is loaded using the CommonJS module loader. See
4545

4646
When loading, the [ES module loader][Modules loaders] loads the program
4747
entry point, the `node` command will accept as input only files with `.js`,
48-
`.mjs`, or `.cjs` extensions; and with `.wasm` extensions when
49-
[`--experimental-wasm-modules`][] is enabled.
48+
`.mjs`, or `.cjs` extensions. With the following flags, additional file
49+
extensions are enabled:
50+
51+
* [`--experimental-wasm-modules`][] for files with `.wasm` extension.
52+
* [`--experimental-addon-modules`][] for files with `.node` extension.
5053

5154
## Options
5255

@@ -879,6 +882,16 @@ and `"` are usable.
879882
It is possible to run code containing inline types by passing
880883
[`--experimental-strip-types`][].
881884

885+
### `--experimental-addon-modules`
886+
887+
<!-- YAML
888+
added: REPLACEME
889+
-->
890+
891+
> Stability: 1.0 - Early development
892+
893+
Enable experimental import support for `.node` addons.
894+
882895
### `--experimental-async-context-frame`
883896

884897
<!-- YAML
@@ -3046,6 +3059,7 @@ one is included in the list below.
30463059
* `--enable-source-maps`
30473060
* `--entry-url`
30483061
* `--experimental-abortcontroller`
3062+
* `--experimental-addon-modules`
30493063
* `--experimental-async-context-frame`
30503064
* `--experimental-detect-module`
30513065
* `--experimental-eventsource`
@@ -3617,6 +3631,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
36173631
[`--diagnostic-dir`]: #--diagnostic-dirdirectory
36183632
[`--env-file-if-exists`]: #--env-file-if-existsconfig
36193633
[`--env-file`]: #--env-fileconfig
3634+
[`--experimental-addon-modules`]: #--experimental-addon-modules
36203635
[`--experimental-sea-config`]: single-executable-applications.md#generating-single-executable-preparation-blobs
36213636
[`--experimental-strip-types`]: #--experimental-strip-types
36223637
[`--experimental-wasm-modules`]: #--experimental-wasm-modules

‎doc/api/esm.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1039,18 +1039,21 @@ _isImports_, _conditions_)
10391039
> 5. If `--experimental-wasm-modules` is enabled and _url_ ends in
10401040
> _".wasm"_, then
10411041
> 1. Return _"wasm"_.
1042-
> 6. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_url_).
1043-
> 7. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_).
1044-
> 8. Let _packageType_ be **null**.
1045-
> 9. If _pjson?.type_ is _"module"_ or _"commonjs"_, then
1046-
> 1. Set _packageType_ to _pjson.type_.
1047-
> 10. If _url_ ends in _".js"_, then
1042+
> 6. If `--experimental-addon-modules` is enabled and _url_ ends in
1043+
> _".node"_, then
1044+
> 1. Return _"addon"_.
1045+
> 7. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_url_).
1046+
> 8. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_).
1047+
> 9. Let _packageType_ be **null**.
1048+
> 10. If _pjson?.type_ is _"module"_ or _"commonjs"_, then
1049+
> 1. Set _packageType_ to _pjson.type_.
1050+
> 11. If _url_ ends in _".js"_, then
10481051
> 1. If _packageType_ is not **null**, then
10491052
> 1. Return _packageType_.
10501053
> 2. If the result of **DETECT\_MODULE\_SYNTAX**(_source_) is true, then
10511054
> 1. Return _"module"_.
10521055
> 3. Return _"commonjs"_.
1053-
> 11. If _url_ does not have any extension, then
1056+
> 12. If _url_ does not have any extension, then
10541057
> 1. If _packageType_ is _"module"_ and `--experimental-wasm-modules` is
10551058
> enabled and the file at _url_ contains the header for a WebAssembly
10561059
> module, then
@@ -1060,7 +1063,7 @@ _isImports_, _conditions_)
10601063
> 3. If the result of **DETECT\_MODULE\_SYNTAX**(_source_) is true, then
10611064
> 1. Return _"module"_.
10621065
> 4. Return _"commonjs"_.
1063-
> 12. Return **undefined** (will throw during load phase).
1066+
> 13. Return **undefined** (will throw during load phase).
10641067
10651068
**LOOKUP\_PACKAGE\_SCOPE**(_url_)
10661069

‎doc/api/module.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1160,6 +1160,7 @@ The final value of `format` must be one of the following:
11601160
11611161
| `format` | Description | Acceptable types for `source` returned by `load` |
11621162
| ------------ | ------------------------------ | -------------------------------------------------- |
1163+
| `'addon'` | Load a Node.js addon | {null} |
11631164
| `'builtin'` | Load a Node.js builtin module | {null} |
11641165
| `'commonjs'` | Load a Node.js CommonJS module | {string\|ArrayBuffer\|TypedArray\|null\|undefined} |
11651166
| `'json'` | Load a JSON file | {string\|ArrayBuffer\|TypedArray} |

‎doc/node.1

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,9 @@ Enable Source Map V3 support for stack traces.
163163
.It Fl -entry-url
164164
Interpret the entry point as a URL.
165165
.
166+
.It Fl -experimental-addon-modules
167+
Enable experimental addon module support.
168+
.
166169
.It Fl -experimental-import-meta-resolve
167170
Enable experimental ES modules support for import.meta.resolve().
168171
.

‎lib/internal/modules/esm/formats.js

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ const fsBindings = internalBinding('fs');
1010
const { fs: fsConstants } = internalBinding('constants');
1111

1212
const experimentalWasmModules = getOptionValue('--experimental-wasm-modules');
13+
const experimentalAddonModules = getOptionValue('--experimental-addon-modules');
1314

1415
const extensionFormatMap = {
1516
'__proto__': null,
@@ -23,6 +24,10 @@ if (experimentalWasmModules) {
2324
extensionFormatMap['.wasm'] = 'wasm';
2425
}
2526

27+
if (experimentalAddonModules) {
28+
extensionFormatMap['.node'] = 'addon';
29+
}
30+
2631
if (getOptionValue('--experimental-strip-types')) {
2732
extensionFormatMap['.ts'] = 'module-typescript';
2833
extensionFormatMap['.mts'] = 'module-typescript';

‎lib/internal/modules/esm/load.js

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ async function defaultLoad(url, context = kEmptyObject) {
105105
if (urlInstance.protocol === 'node:') {
106106
source = null;
107107
format ??= 'builtin';
108+
} else if (format === 'addon') {
109+
// Skip loading addon file content. It must be loaded with dlopen from file system.
110+
source = null;
108111
} else if (format !== 'commonjs') {
109112
if (source == null) {
110113
({ responseURL, source } = await getSource(urlInstance, context));

‎lib/internal/modules/esm/translators.js

Lines changed: 88 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
const {
44
ArrayPrototypeMap,
55
ArrayPrototypePush,
6-
Boolean,
76
FunctionPrototypeCall,
87
JSONParse,
98
ObjectKeys,
@@ -52,6 +51,7 @@ let debug = require('internal/util/debuglog').debuglog('esm', (fn) => {
5251
});
5352
const { emitExperimentalWarning, kEmptyObject, setOwnProperty, isWindows } = require('internal/util');
5453
const {
54+
ERR_INVALID_RETURN_PROPERTY_VALUE,
5555
ERR_UNKNOWN_BUILTIN_MODULE,
5656
} = require('internal/errors').codes;
5757
const { maybeCacheSourceMap } = require('internal/source_map/source_map_cache');
@@ -185,7 +185,7 @@ function createCJSModuleWrap(url, source, isMain, format, loadCJS = loadCJSModul
185185
// In case the source was not provided by the `load` step, we need fetch it now.
186186
source = stringify(source ?? getSource(new URL(url)).source);
187187

188-
const { exportNames, module } = cjsPreparseModuleExports(filename, source, isMain, format);
188+
const { exportNames, module } = cjsPreparseModuleExports(filename, source, format);
189189
cjsCache.set(url, module);
190190

191191
const wrapperNames = [...exportNames, 'module.exports'];
@@ -229,6 +229,47 @@ function createCJSModuleWrap(url, source, isMain, format, loadCJS = loadCJSModul
229229
}, module);
230230
}
231231

232+
/**
233+
* Creates a ModuleWrap object for a CommonJS module without source texts.
234+
* @param {string} url - The URL of the module.
235+
* @param {boolean} isMain - Whether the module is the main module.
236+
* @returns {ModuleWrap} The ModuleWrap object for the CommonJS module.
237+
*/
238+
function createCJSNoSourceModuleWrap(url, isMain) {
239+
debug(`Translating CJSModule without source ${url}`);
240+
241+
const filename = urlToFilename(url);
242+
243+
const module = cjsEmplaceModuleCacheEntry(filename);
244+
cjsCache.set(url, module);
245+
246+
if (isMain) {
247+
setOwnProperty(process, 'mainModule', module);
248+
}
249+
250+
// Addon export names are not known until the addon is loaded.
251+
const exportNames = ['default', 'module.exports'];
252+
return new ModuleWrap(url, undefined, exportNames, function evaluationCallback() {
253+
debug(`Loading CJSModule ${url}`);
254+
255+
if (!module.loaded) {
256+
wrapModuleLoad(filename, null, isMain);
257+
}
258+
259+
/** @type {import('./loader').ModuleExports} */
260+
let exports;
261+
if (module[kModuleExport] !== undefined) {
262+
exports = module[kModuleExport];
263+
module[kModuleExport] = undefined;
264+
} else {
265+
({ exports } = module);
266+
}
267+
268+
this.setExport('default', exports);
269+
this.setExport('module.exports', exports);
270+
}, module);
271+
}
272+
232273
translators.set('commonjs-sync', function requireCommonJS(url, source, isMain) {
233274
initCJSParseSync();
234275

@@ -280,26 +321,38 @@ translators.set('commonjs', function commonjsStrategy(url, source, isMain) {
280321
return createCJSModuleWrap(url, source, isMain, 'commonjs', cjsLoader);
281322
});
282323

324+
/**
325+
* Get or create an entry in the CJS module cache for the given filename.
326+
* @param {string} filename CJS module filename
327+
* @returns {CJSModule} the cached CJS module entry
328+
*/
329+
function cjsEmplaceModuleCacheEntry(filename, exportNames) {
330+
// TODO: Do we want to keep hitting the user mutable CJS loader here?
331+
let cjsMod = CJSModule._cache[filename];
332+
if (cjsMod) {
333+
return cjsMod;
334+
}
335+
336+
cjsMod = new CJSModule(filename);
337+
cjsMod.filename = filename;
338+
cjsMod.paths = CJSModule._nodeModulePaths(cjsMod.path);
339+
cjsMod[kIsCachedByESMLoader] = true;
340+
CJSModule._cache[filename] = cjsMod;
341+
342+
return cjsMod;
343+
}
344+
283345
/**
284346
* Pre-parses a CommonJS module's exports and re-exports.
285347
* @param {string} filename - The filename of the module.
286348
* @param {string} [source] - The source code of the module.
287-
* @param {boolean} isMain - Whether it is pre-parsing for the entry point.
288-
* @param {string} format
349+
* @param {string} [format]
289350
*/
290-
function cjsPreparseModuleExports(filename, source, isMain, format) {
291-
let module = CJSModule._cache[filename];
292-
if (module && module[kModuleExportNames] !== undefined) {
351+
function cjsPreparseModuleExports(filename, source, format) {
352+
const module = cjsEmplaceModuleCacheEntry(filename);
353+
if (module[kModuleExportNames] !== undefined) {
293354
return { module, exportNames: module[kModuleExportNames] };
294355
}
295-
const loaded = Boolean(module);
296-
if (!loaded) {
297-
module = new CJSModule(filename);
298-
module.filename = filename;
299-
module.paths = CJSModule._nodeModulePaths(module.path);
300-
module[kIsCachedByESMLoader] = true;
301-
CJSModule._cache[filename] = module;
302-
}
303356

304357
if (source === undefined) {
305358
({ source } = loadSourceForCJSWithHooks(module, filename, format));
@@ -340,7 +393,7 @@ function cjsPreparseModuleExports(filename, source, isMain, format) {
340393

341394
if (format === 'commonjs' ||
342395
(!BuiltinModule.normalizeRequirableId(resolved) && findLongestRegisteredExtension(resolved) === '.js')) {
343-
const { exportNames: reexportNames } = cjsPreparseModuleExports(resolved, undefined, false, format);
396+
const { exportNames: reexportNames } = cjsPreparseModuleExports(resolved, undefined, format);
344397
for (const name of reexportNames) {
345398
exportNames.add(name);
346399
}
@@ -462,6 +515,25 @@ translators.set('wasm', async function(url, source) {
462515
}).module;
463516
});
464517

518+
// Strategy for loading a addon
519+
translators.set('addon', function translateAddon(url, source, isMain) {
520+
emitExperimentalWarning('Importing addons');
521+
522+
// The addon must be loaded from file system with dlopen. Assert
523+
// the source is null.
524+
if (source !== null) {
525+
throw new ERR_INVALID_RETURN_PROPERTY_VALUE(
526+
'null',
527+
'load',
528+
'source',
529+
source);
530+
}
531+
532+
debug(`Translating addon ${url}`);
533+
534+
return createCJSNoSourceModuleWrap(url, isMain);
535+
});
536+
465537
// Strategy for loading a commonjs TypeScript module
466538
translators.set('commonjs-typescript', function(url, source) {
467539
emitExperimentalWarning('Type Stripping');

‎src/node_options.cc

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -409,6 +409,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
409409
"Treat the entrypoint as a URL",
410410
&EnvironmentOptions::entry_is_url,
411411
kAllowedInEnvvar);
412+
AddOption("--experimental-addon-modules",
413+
"experimental import support for addons",
414+
&EnvironmentOptions::experimental_addon_modules,
415+
kAllowedInEnvvar);
412416
AddOption("--experimental-abortcontroller", "", NoOp{}, kAllowedInEnvvar);
413417
AddOption("--experimental-eventsource",
414418
"experimental EventSource API",

‎src/node_options.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ class EnvironmentOptions : public Options {
120120
bool require_module = true;
121121
std::string dns_result_order;
122122
bool enable_source_maps = false;
123+
bool experimental_addon_modules = false;
123124
bool experimental_eventsource = false;
124125
bool experimental_fetch = true;
125126
bool experimental_websocket = true;

‎test/addons/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ Makefile
55
*.mk
66
gyp-mac-tool
77
/*/build
8+
/esm/node_modules/*/build

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy