Combining source code analysis of Node.js module loading and operation principles


Author: Ma Ling Yang

efe.baidu.com/blog/nodejs-module-analyze/

The advent of Node.js has taken JavaScript out of the browser and into the vast realm of server-side development. The introduction of Node.js' modular specification for CommonJS has made JavaScript a truly adaptable language for large projects.

Working with modules in Node.js is very simple, and we've almost all done it in our daily development: write a piece of JavaScript code, require some desired packages, and export the code products. But are we clear about the loading and running principles behind Node.js modularity. First throw out the following questions.

What file types are supported by the modules in Node.js?

What is the difference in the process of loading and running core modules and third-party modules?

How do I go about writing a C/C++ extension module other than a JavaScript module?

……

In this post, we'll explore the answers behind these questions in the context of Node.js source code.

1. Node.js module types

In Node.js, modules can be divided into the following main types.

Core modules: included in the Node.js source code and compiled into the Node.js executable binary JavaScript modules, also called native modules, such as the common http, fs, etc.

C/C++ modules, also called built-in modules, are generally not called directly, but in the native module, and then we require

native modules, such as buffer, fs, os, and other native modules that we commonly use in Node.js, all have calls to built-in modules underneath.

Third-party modules: Modules that do not come with Node.js source code can be collectively referred to as third-party modules, such as express, webpack, etc.

JavaScript modules, which are the most common and what we usually write when we develop

JSON module, this is simple, it's a JSON file

C/C++ extensions, written in C/C++ and compiled with a .node extension

In this post, we'll cover the principles of loading and running each of these modules.

2. Node.js source code structure at a glance

The Node.js 6.x version of the source code is used here as an example to do the analysis. Go to github and download the appropriate version of the Node.js source code and you can see the general structure of the code as follows.

├── AUTHORS

├── BSDmakefile

├── BUILDING.md

├── CHANGELOG.md

├── CODE_OF_CONDUCT.md

├── COLLABORATOR_GUIDE.md

├── CONTRIBUTING.md

├── GOVERNANCE.md

├── LICENSE

├── Makefile

├── README.md

├── android-configure

├── benchmark

├── common.gypi

├── configure

├── deps

├── doc

├── lib

├── node.gyp

├── node.gypi

├── src

├── test

├── tools

└── vcbuild.bat

Of which.

The ./lib folder contains mainly various JavaScript files, where our common JavaScript native modules are located.

The ./src folder contains mostly C/C++ source files for Node.js, where many of the built-in modules are located.

The ./deps folder contains various libraries that Node.js relies on, typically v8, libuv, zlib, etc.

The release version we use in development is actually the executable compiled from the source code. If we want to make some personalized customizations to Node.js, we can modify the source code and then run the compilation to get a customized version of Node.js. Here's a brief overview of the Node.js compilation process, using the Linux platform as an example.

First, we need to get acquainted with the organizing tool used for compilation, namely gyp. In the Node.js source code we can see a node.gyp, a file that contains some JSON-like configuration written in python that defines a sequence of build project tasks. Let's take an example where one of the fields is as follows.

{

'target_name':'node_js2c',

'type':'none',

'toolsets':['host'],

'actions':[

{

'action_name':'node_js2c',

'inputs':[

'

'./config.gypi',

],

'outputs':[

'

],

'conditions':[

['node_use_dtrace=="false" and node_use_etw=="false"',{

'inputs':['src/notrace_macros.py']

}],

['node_use_lttng=="false"',{

'inputs':['src/nolttng_macros.py']

}],

['node_use_perfctr=="false"',{

'inputs':['src/perfctr_macros.py']

}]

],

'action':[

'python',

'tools/js2c.py',

'

'

],

},

],

},#endnode_js2c

The main purpose of this task, as evidenced by the name node_js2c, is to convert JavaScript to C/C++ code. This task we will also mention below.

To begin by compiling Node.js, a number of tools need to be installed in advance: the

gcc and g++ 4.9.4 and above

clang and clang++

python 2.6 or 2.7, note here that only these two versions are allowed, not python 3+

GNU MAKE version 3.81 and above

With these tools in place and access to the Node.js source directory, we simply need to run the following commands in sequence.

./configuration

make

make install

It is ready to be compiled to generate the executable and installed.

3. Start with node index.js

Let's start with the simplest case。 Suppose there is a index.js documents, There's only one very simple line in there. console.log('hello world') code。 When the input node index.js at the time of,Node.js How is it compiled、 What about running this file??

When the Node.js command is entered, the main function in the Node.js source code is called, in src/node_main.cc.

// src/node_main.cc

#include "node.h"

#ifdef _WIN32

#include

intwmain(intargc,wchar_t*wargv[]){

// Entry under windows

}

#else

// UNIX

intmain(intargc,char*argv[]){

// Disable stdio buffering, it interacts poorly with printf()

// calls elsewhere in the program (e.g., any logging from V8.)

setvbuf(stdout,nullptr,_IONBF,);

setvbuf(stderr,nullptr,_IONBF,);

// Focus on the following line

returnnode::Start(argc,argv);

}

#endif

This file is for entry purposes only and distinguishes between Windows and Unix environments. Let's take the Unix example, the last call to node::Start in the main function, which is in the src/node.cc file.

// src/node.cc

intStart(intargc,char**argv){

// ...

{

NodeInstanceData instance_data(NodeInstanceType::MAIN,

uv_default_loop(),

argc,

const_cast(argv),

exec_argc,

exec_argv,

use_debug_agent);

StartNodeInstance(&instance_data);

exit_code=instance_data.exit_code();

}

// ...

}

// ...

staticvoidStartNodeInstance(void*arg){

// ...

{

Environment::AsyncCallbackScope callback_scope(env);

LoadEnvironment(env);

}

// ...

}

// ...

voidLoadEnvironment(Environment*env){

// ...

Localscript_name=FIXED_ONE_BYTE_STRING(env->isolate(),

"bootstrap_node.js");

Localf_value=ExecuteString(env,MainSource(env),script_name);

if(try_catch.HasCaught()){

ReportException(env,try_catch);

exit(10);

}

// The bootstrap_node.js file returns a function 'f'

CHECK(f_value->IsFunction());

Localf=Local::Cast(f_value);

// ...

f->Call(Null(env->isolate()),1,&arg);

}

The whole document is rather long, In the above code snippet, Only the pieces of the process we need to focus on the most have been taken, The call relationship is as follows: Start -> StartNodeInstance -> LoadEnvironment。

(located) at LoadEnvironment Needs our attention, The main thing to do is, extract bootstrap_node.js The code string in, parse into a function, and finally by f->Call go ahead and implement。

OK, here's the kicker, we finally see the first JavaScript file bootstrap_node.js since Node.js was launched, and we can see from the filename that this is an entry-level file. So let's take a quick look, the file path is lib/internal/bootstrap_node.js:.

// lib/internal/boostrap_node.js

(function(process){

functionstartup(){

// ...

elseif(process.argv[1]){

constpath=NativeModule.require('path');

process.argv[1]=path.resolve(process.argv[1]);

constModule=NativeModule.require('module');

// ...

preloadModules();

run(Module.runMain);

}

// ...

}

// ...

startup();

}

// lib/module.js

// ...

// bootstrap main module.

Module.runMain=function(){

// Load the main module--the command line argument.

Module._load(process.argv[1],null,true);

// Handle any nextTicks added in the first tick of the program

process._tickCallback();

};

// ...

Here we still focus on the main process and we can see that in bootstrap_node.js, a startup() function is executed. Get the filename via process.argv[1], which in our node index.js is obviously index.js, and call path.resolve to resolve the file path. At the end, run(Module.runMain) to compile and execute our index.js.

And the Module.runMain function is defined in lib/module.js, which is listed at the end of the above code snippet, and as you can see, mainly calls Module._load to load the execution process.argv[1].

Below, when we analyze the module's require, we also come to lib/module.js, and we also analyze Module._load. So we can see that the process of starting a file with Node.js is actually, at the end of the day, the process of requiring a file, which can be interpreted as immediately requiring a file. Here's a breakdown of how require works.

4. The key to the module loading principle: require

Let's go further and assume that our index.js has the following content.

var http = require('http');

So what happens when this line of code is executed?

The definition of require is still in lib/module.js: the

// lib/module.js

// ...

assert(path,'missing path');

assert(typeofpath==='string','path must be a string');

returnModule._load(path,this,/* isMain */false);

};

// ...

The require method is defined in the Module's prototype chain. You can see that in this method, Module._load is called.

We're back to Module._load so soon to see what this key method actually does.

// lib/module.js

// ...

Module._load=function(request,parent,isMain){

if(parent){

debug('Module._load REQUEST %s parent: %s',request,parent.id);

}

varfilename=Module._resolveFilename(request,parent,isMain);

varcachedModule=Module._cache[filename];

if(cachedModule){

returncachedModule.exports;

}

if(NativeModule.nonInternalExists(filename)){

debug('load native module %s',request);

returnNativeModule.require(filename);

}

varmodule=newModule(filename,parent);

if(isMain){

process.mainModule=module;

module.id='.';

}

Module._cache[filename]=module;

tryModuleLoad(module,filename);

returnmodule.exports;

};

// ...

The flow of this code is relatively clear, specifically.

Based on the filename, call Module._resolveFilename to resolve the path to the file

See if the module is available in the cache Module._cache, and if so, return it directly

Determine if the module is a core module by NativeModule.nonInternalExists, and if so, call the core module's load method NativeModule.require

If it's not a core module, create a new Module object and call the tryModuleLoad function to load the module

Let's start by looking at Module._resolveFilename. It's helpful to understand how file path resolution works in Node.js to see this method.

// lib/module.js

// ...

Module._resolveFilename=function(request,parent,isMain){

// ...

varfilename=Module._findPath(request,paths,isMain);

if(!filename){

varerr=newError("Cannot find module '"+request+"'");

err.code='MODULE_NOT_FOUND';

throwerr;

}

returnfilename;

};

// ...

Module._findPath is called in Module._resolveFilename, the logic of module loading is actually concentrated in this method, as this method is long, directly attached github the code of this method.

https://github.com/nodejs/node/blob/v6.x/lib/module.js#L158

It can be seen that the logical flow of file path resolution is as follows.

Mr. cacheKey, determine whether the corresponding cache exists, if it exists, return directly

If the last character of path is not /.

If the path is a file and exists, then the path to the file is returned directly

If the path is a directory, call the tryPackage function to parse the package.json in the directory and retrieve the path to the file written in the main field

Determine if the path exists and return it directly

Try adding .js, .json, or .node to the path to see if it exists, and return if it does

Try adding index.js, index.json, index.node to the path in order to determine if it exists, and return if it does

If that doesn't work, try directly adding the .js, .json, .node suffixes to the current path

If the last character of path is /.

Call tryPackage and the parsing process is similar to the above case

If it doesn't work., Try adding index.js, index.json, index.node to the path in order to determine if it exists, and return if it does

The github links for the tryPackage and tryExtensions methods used in parsing the file: https://github.com/nodejs/node/blob/v6.x/lib/module.js#L108 https://github.com/nodejs/node/blob/v6.x/lib/module.js#L146

The entire process can be seen in the following diagram.

After the file path is resolved, we check if the cache exists based on the file path, and return it directly if it exists, or go to step 3 or 4 if it doesn't.

here, (located) at 3、4 The two steps produce two branches, That is, core and third-party modules are loaded differently。 Since we have assumed that our index.js suffer from var http = require('http'),http is a core module, So let's start by analyzing this branch that the core module loads。

4.1 Core module loading principle

The core module is loaded via NativeModule.require, the NativeModule is defined in bootstrap_node.js, attached is the github link: https://github.com/nodejs/node/blob/v6.x/lib/internal/bootstrap_node.js#L401

As you can see from the code, the flow of NativeModule.require is as follows.

Determines if the cache has been loaded, and if so, returns exports directly

Create a new nativeModule object, then cache it and load it to compile

// lib/internal/bootstrap_node.js

// ...

NativeModule._source=process.binding('natives');

// ...

NativeModule.getSource=function(id){

returnNativeModule._source[id];

};

direct from NativeModule._source acquired, And where is this assigned?? It is also intercepted in the above code, It is through NativeModule._source = process.binding('natives') acquired。

This is where we insert a description of how the JavaScript native module code is stored. When the Node.js source code is compiled, the js2c.py utility that comes with v8 is used to convert all the code of the js module under the lib folder to an array in C, generating a node_natives.h header file that documents the array.

namespacenode{

constcharnode_native[]={47,47,32,67,112…}

constcharconsole_native[]={47,47,32,67,112…}

constcharbuffer_native[]={47,47,32,67,112…}

}

struct_native{constcharname;constchar*source;size_tsource_len;};

staticconststruct_nativenatives[]={

{“node”,node_native,sizeof(node_native)-1},

{“dgram”,dgram_native,sizeof(dgram_native)-1},

{“console”,console_native,sizeof(console_native)-1},

{“buffer”,buffer_native,sizeof(buffer_native)-1},

}

And above NativeModule._source = process.binding('natives'); effect, Just take this out. natives arrays, assign toNativeModule._source, So in getSource method, You can directly use the module name as an index, Retrieve the source code of the module from the array。

Here we insert a review of the above, when we introduced Node.js compilation, we introduced node.gyp, one of the tasks was node_js2c, at that time the author mentioned that from the name this task was converting JavaScript to C code, and the C code in the natives array here is the product of this build task. And here, we finally know what this compilation task does.

Now that you know how to get the source code, move on to the compile method to see how the source code is compiled.

// lib/internal/bootstrap_node.js

NativeModule.wrap=function(script){

returnNativeModule.wrapper[]+script+NativeModule.wrapper[1];

};

NativeModule.wrapper=[

'(function (exports, require, module, __filename, __dirname) { ',

' });'

];

varsource=NativeModule.getSource(this.id);

source=NativeModule.wrap(source);

this.loading=true;

try{

constfn=runInThisContext(source,{

filename:this.filename,

lineOffset:,

displayErrors:true

});

fn(this.exports,NativeModule.require,this,this.filename);

this.loaded=true;

}finally{

this.loading=false;

}

};

// ...

This basically explains the process of loading Node.js core modules. Having said that, you may have a doubt, the above analysis process seems to involve only the JavaScript native module in the core module, what about for the C/C++ built-in module?

Actually, it's like this., as far as sth is concerned built-in modularity, They are not passed require incoming, Rather, it is through precess.binding(' Module Name') introduced。 Generally we rarely use directly in our own code process.binding introducingbuilt-in module, Rather, it is through require quotenative module, but (not) native Inside the module will be introduced built-in module。 For example, we commonly use the buffer module, Its internal implementation then introduces the C/C++ built-in module, This is to avoid the v8 The memory limit of the:

// lib/buffer.js

'use strict';

// Introduce a C/C++ built-in module called buffer via process.binding

constbinding=process.binding('buffer');

// ...

thus, We are in require('buffer') at the time of, It's actually an indirect use of C/C++ built-in module。

Here comes process.binding again! In fact, the method process.binding is defined in node.cc.

// src/node.cc

// ...

staticvoidBinding(constFunctionCallbackInfo&args){

// ...

node_module*mod=get_builtin_module(*module_v);

// ...

}

// ...

env->SetMethod(process,"binding",Binding);

// ...

The key step in the Binding function is get_builtin_module. Here again it is necessary to insert an introduction to the storage of C/C++ built-in modules.

In Node.js, built-in modules are defined through a structure called node_module_struct. So the built-in modules are put into an array called node_module_list. The role of process.binding is to use get_builtin_module to retrieve the corresponding built-in module code from this array.

In summary, we have a complete overview of how core modules are loaded, mainly distinguishing between native modules of the JavaScript type and built-in modules of the C/C++ type. A diagram is drawn here to describe the core module loading process.

Whereas, as we described at the beginning, native modules are stored in the lib/ directory in the source code, and built-in modules are stored in the src/ directory in the source code, the following diagram sorts out how native and built-in modules are compiled into the Node.js executable from a compilation perspective.

4.2 Third-party module loading principle

Let's move on to the second branch. Assuming our index.js requires not http but a user-defined module, then in module.js, we would go to the tryModuleLoad method.

// lib/module.js

// ...

functiontryModuleLoad(module,filename){

varthrew=true;

try{

module.load(filename);

threw=false;

}finally{

if(threw){

deleteModule._cache[filename];

}

}

}

// ...

debug('load %j for module %j',filename,this.id);

assert(!this.loaded);

this.filename=filename;

this.paths=Module._nodeModulePaths(path.dirname(filename));

varextension=path.extname(filename)||'.js';

if(!Module._extensions[extension])extension='.js';

Module._extensions[extension](this,filename);

this.loaded=true;

};

// ...

// lib/module.js

// ...

// Native extension for .js

Module._extensions['.js']=function(module,filename){

varcontent=fs.readFileSync(filename,'utf8');

module._compile(internalModule.stripBOM(content),filename);

};

// Native extension for .json

Module._extensions['.json']=function(module,filename){

varcontent=fs.readFileSync(filename,'utf8');

try{

module.exports=JSON.parse(internalModule.stripBOM(content));

}catch(err){

err.message=filename+': '+err.message;

throwerr;

}

};

//Native extension for .node

Module._extensions['.node']=function(module,filename){

returnprocess.dlopen(module,path._makeLong(filename));

};

// ...

As you can see, a total of three types of module loading are supported: .js, .json, .node. The .json type file loading method is the simplest, reading the contents of the file directly and then returning the object after JSON.parse.

Here's a look at the processing of the .js, which also reads the contents of the file synchronously via the fs module, and then calls module._compile to see the relevant code.

// lib/module.js

// ...

Module.wrap=NativeModule.wrap;

// ...

Module.prototype._compile=function(content,filename){

// ...

// create wrapper function

varwrapper=Module.wrap(content);

varcompiledWrapper=vm.runInThisContext(wrapper,{

filename:filename,

lineOffset:,

displayErrors:true

});

// ...

varresult=compiledWrapper.apply(this.exports,args);

if(depth===)stat.cache=null;

returnresult;

};

// ...

First call Module.wrap to wrap the source code, then call the vm.runInThisContext method to compile and execute, and finally return the value of exports. And as you can see from the sentence Module.wrap = NativeModule.wrap, the wrap method of the third-party module, is the same as the wrap method of the core module. Let's recall the key code for loading the core js module we just talked about.

// lib/internal/bootstrap_node.js

NativeModule.wrap=function(script){

returnNativeModule.wrapper[]+script+NativeModule.wrapper[1];

};

NativeModule.wrapper=[

'(function (exports, require, module, __filename, __dirname) { ',

' });'

];

varsource=NativeModule.getSource(this.id);

source=NativeModule.wrap(source);

this.loading=true;

try{

constfn=runInThisContext(source,{

filename:this.filename,

lineOffset:,

displayErrors:true

});

fn(this.exports,NativeModule.require,this,this.filename);

this.loaded=true;

}finally{

this.loading=false;

}

};

Comparison of the two compartments, The compilation and execution of the source code was found to be almost identical between the two。 In terms of the overall process, core JavaScript Modules and third parties JavaScript The biggest difference in the modules is that, core JavaScript The module source code is provided through the process.binding('natives') Fetched from memory, And third parties JavaScript The module source code is provided through the fs.readFileSync method reads from the file。

Finally, take a look at loading third-party C/C++ modules (.node suffix). Intuitively, it's simple: the process.dlopen method is called. The definition of this method is in node.cc.

// src/node.cc

// ...

env->SetMethod(process,"dlopen",DLOpen);

// ...

voidDLOpen(constFunctionCallbackInfo&args){

// ...

constboolis_dlopen_error=uv_dlopen(*filename,&lib);

// ...

}

// ...

The DLOpen function is actually eventually called, and the most important part of the function is to open the dynamic link library using the uv_dlopen method, and then load the C/C++ module. The uv_dlopen method is defined in the libuv library. The libuv library is a cross-platform library for asynchronous IO. For the dynamic loading of extensions, the dlopen() method defined in dlfcn.h is actually called under *nix, while under Windows, it is the LoadLibraryExW() method.

Regarding the libuv library, which is the core driver of Node.js asynchronous IO, this piece is worthy of a dedicated topic in itself, so I won't go into it here.

So far, we have sorted out the process of loading and compiling the three third-party modules.

5. Development of C/C++ extension modules and application scenarios

The above article analyzes the loading process of the various modules in Node.js. You should be familiar with JavaScript module development, but you may be a bit new to C/C++ extension development. This section will briefly describe the development of the extension module and talk about its application scenarios.

The development of Node.js extensions is covered in a special section of the official Node.js documentation, which can be found at https://nodejs.org/docs/latest-v6.x/api/addons.html. Here is just one of the hello world examples to introduce some of the more important concepts in writing extension modules.

Suppose we want to implement a JavaScript function equivalent to the following by extending the module.

First create a hello.cc file and write the following code.

// hello.cc

#include

namespacedemo{

usingv8::FunctionCallbackInfo;

usingv8::Isolate;

usingv8::Local;

usingv8::Object;

usingv8::String;

usingv8::Value;

voidMethod(constFunctionCallbackInfo&args){

Isolate*isolate=args.GetIsolate();

args.GetReturnValue().Set(String::NewFromUtf8(isolate,"world"));

}

voidinit(Localexports){

NODE_SET_METHOD(exports,"hello",Method);

}

NODE_MODULE(NODE_GYP_MODULE_NAME,init)

}// namespace demo

The file is short, but some of our more unfamiliar code already appears, so it's helpful to go over each one here to understand the basics of extension modules.

First at the beginning we introduce node.h, the mandatory header file for writing Node.js extensions, which contains almost all the libraries and data types we need.

Second, see a lot of code like using v8:xxx. We know that Node.js is based on the v8 engine, which, in turn, is written in C++. To develop C++ extensions, we need to use many of the data types available in v8, and this is a series of code that declares the need to use those data types in the v8 namespace.

Then look. Method approach, Its parameter type FunctionCallbackInfo& args, this one args just from JavaScript The parameters passed in the, at the same time, If you want to be in Method suffer from JavaScript Return variable, then you need to call args.GetReturnValue().Set approach。

Next you need to define the initialization method for the extension module, Here it is. Init function, Just one simple sentence. NODE_SET_METHOD(exports, "hello", Method);, represent exports Giving a name of hello promotion of the method, The specific definition of this method is Method function。

Finally there is a macro definition: NODE_MODULE(NODE_GYP_MODULE_NAME, init), the first parameter is the name of the desired extension module and the second parameter is the initialization method for that module.

In order to compile this module, we need to install the node-gyp compiler tool via npm. This tool wraps Google's gyp tool for building Node.js extensions. After installing this tool, we add a configuration file called bingding.gyp under the source folder, and for our example, the file simply reads

{

"targets":[

{

"target_name":"addon",

"sources":["hello.cc"]

}

]

}

This way, running node-gyp build will compile the extension module. During this process, node-gyp will also go to the specified directory (usually ~/.node-gyp) and search for some header and library files for our current version of Node.js, and if they don't exist, it will download them for us from the Node.js website. This way, when writing extensions, we can use all Node.js headers directly via #include.

If the build succeeds, you will see an addon.node in the current folder under the build/Release/ path, which is our compiled and requireable extension.

From the above example, we can see in general how the extension module works, it can receive parameters from JavaScript, then in the middle it can call the C/C++ language's ability to do various operations, processing, and then finally the result can be returned to JavaScript.

It is worth noting that different versions of Node.js rely on different versions of v8, resulting in many API differences, so the process of developing extensions using native C/C++ will also need to be compatible with different versions of Node.js. For example, declaring a function, in versions below v6.x and v0.12, respectively, would need to be written like this

Handle Example(const Arguments& args); // 0.10.x

void Example(FunctionCallbackInfo& args); // 6.x

As you can see, the declaration of the function, including the writing of the arguments in the function, is not the same. This is reminiscent of the need to use Babel to help with compatibility conversions in order to write ES6 in Node.js development as well. So in the Node.js extension development space, is there a library like Babel that helps us deal with compatibility issues? The answer is yes, it's called NAN (Native Abstraction for Node.js). It's essentially a bunch of macros that help us detect different versions of Node.js and call different APIs. For example, with the help of NAN, declaring a function, instead of thinking about the Node.js version, we can just write a piece of code that looks like this

#include

NAN_METHOD(Example){

// ...

}

NAN's macros are automatically determined at compile time, unfolding different results depending on the Node.js version, thus solving compatibility issues. For a more detailed description of NAN, interested students can visit the project's github page: https://github.com/nodejs/nan.

After introducing the development of so many extension modules, some students may ask, like these extension modules to achieve the function, it seems to use js can also be quickly achieved, why bother to develop extensions? This begs the question of the scenarios in which C/C++ extensions are applicable.

The author has roughly summarized here several types of scenarios to which C/C++ applies.

Computationally intensive applications. As we know, the programming model of Node.js is single-threaded + asynchronous IO, where single-threading causes it to be a weakness for computationally intensive applications, where large amounts of computation block the main JavaScript thread, making it impossible to respond to other requests. For this scenario, a C/C++ extension can be used to speed up the computation. After all, although the v8 engine is fast, it is still not as fast as C/C++. Also, using C/C++ allows us to open multiple threads to avoid blocking the main JavaScript thread, and there are already some Node.js multithreading solutions based on extension modules in the community, the most popular of which is probably a project called thread-a-gogo, which can be moved to github: https://github.com/xk/node-threads-a-gogo.

Applications with high memory consumption. Node.js is based on v8, which was originally designed for browsers, so it has a relatively strict memory limit, so for some applications that need more memory, directly based on v8 may be a bit overwhelming, so you need to use extension modules to bypass the memory limit of v8.

Regarding the first point, I have also implemented a test example here using native Node.js and Node.js extensions respectively to compare the computational performance. The test case is the classic computation of the Fibonacci series, which starts by implementing a function to compute the Fibonacci series using the Node.js native language, named fibJs.

functionfibJs(n){

if(n===||n===1){

returnn;

}

else{

returnfibJs(n-1)+fibJs(n-2);

}

}

Then use C++ to write an extension function that does the same thing, named fibC:

// fibC.cpp

#include

#include

usingnamespacev8;

intfib(intn){

if(n==||n==1){

returnn;

}

else{

returnfib(n-1)+fib(n-2);

}

}

voidMethod(constFunctionCallbackInfo&args){

Isolate*isolate=args.GetIsolate();

intn=args[]->NumberValue();

intresult=fib(n);

args.GetReturnValue().Set(result);

}

voidinit(Localexports,Localmodule){

NODE_SET_METHOD(module,"exports",Method);

}

NODE_MODULE(fibC,init)

In the test, these two functions are used to compute the Fibonacci series from 1 to 40, respectively:

functiontestSpeed(fn,testName){

varstart=Date.now();

for(vari=;i

fn(i);

}

varspend=Date.now()-start;

console.log(testName,'spend time: ',spend);

}

// Testing with extension modules

varfibC=require('./build/Release/fibC');// Here is the path where the compiled product of the extension module is stored

testSpeed(fibC,'c++ test:');

// Testing with JavaScript functions

functionfibJs(n){

if(n===||n===1){

returnn;

}

else{

returnfibJs(n-1)+fibJs(n-2);

}

}

testSpeed(fibJs,'js test:');

// c++ test: spend time: 1221

// js test: spend time: 2611

In several tests, the extension module took about 1.2s on average, while the JavaScript module took about 2.6s, which shows that the C/C++ extension performance is much faster in this scenario.

Of course, these points are only based on the author's knowledge. In practice, when you encounter problems, you can also try to consider whether the problem can be better solved if you use C/C++ extension modules.

concluding remarks

As we read the article, let's go back and see if all those questions raised at the beginning were answered in the course of the article's analysis? To review the logic of this article again.

Let's start with a rationale for running node index.js, pointing out that running a file with node is the same as executing require once immediately.

Then the require method in node is introduced, where several cases of core, built-in and non-core modules are distinguished, and the flow principles for loading and compiling are detailed separately. In this process, there are also separate descriptions of module path resolution, module caching, and other knowledge points.

Finally, the development of less familiar c/c++ extension modules is presented, with a performance comparison example to illustrate the scenarios in which they are applicable.

In fact, learning the Node.js module loading process will help us gain a deeper understanding of the underlying Node.js operating principles, while mastering the development of extension modules and learning to use them in appropriate scenarios will enable us to develop higher performance Node.js applications.

Learning the principles of Node.js is a long path. It is recommended that readers who understand the underlying module mechanics go deeper into v8, libuv, etc. to become proficient in Node.js.

[About the submission]

If you have an original and good article to submit, please send a direct message to the public.

① Message format.

[Contribute] + 《 Article Title》 + Article Link

② Example.

[Contribution] "Don't Call Yourself a Programmer, A Summary of My IT Career for More Than a Decade": http://blog.jobbole.com/94148/

Please include your personal information at the end~

Find this article helpful? Please share this with as many people as possible

Follow "Front End Book" to improve your front end skills


Recommended>>
1、There are more girls in shorts Fewer girls in dresses What magic is at work
2、Home webcam use advice dont point it at sensitive areas of your routine
3、Goodfellow helps blockchain industry project land in Foshan Chancheng
4、Rereleased again latest DLC Happy Laundry Day OneClick Upgrade Pack Evil Pack simultaneous update
5、Ganzhou Citys first tax robot on board to open a new mode of intelligent taxation

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送

    已发送

    朋友将在看一看看到

    确定
    分享你的想法...
    取消

    分享想法到看一看

    确定
    最多200字,当前共

    发送中

    网络异常,请稍后重试

    微信扫一扫
    关注该公众号