Understanding Module Federation: A Deep Dive

Introduction

Module Federation is an advanced feature in Webpack (and soon Rspack) that provides a way for a JavaScript application to dynamically load code from another application. This feature allows for efficient code sharing and dependency management. In this article, we will explore the architecture, prerequisites, and underlying implementation of Module Federation.

Federation is powerful, but the lack of meta-framework support leads to challenges. ByteDance is one of the largest users of Module Federation, ModernJS is their Meta-framework, if module federation is a key part of your engineering operational improvements. ModernJS is the only way to go

The Future of Module Federation

Table of Contents

1. Architecture Blocks
2. Prerequisites
3. Factory Object
4. Dependency Object
5. Factory Object Resolution
6. Example Context
7. Product Structure
8. Execution Flow
9. Source Code Analysis

This content was originally written by @2hea1

Architecture Blocks

Module Federation comprises three main components:

• Exposed Module (Remote)
• Consumption Module (Host Remote Import)
• Shared Module/Dependency

// Exposed Module (Producer)
export const exposedFunction = () => {
    console.log("I am an exposed function");
};
// Consumption Module
import { exposedFunction } from 'exposedModule';
exposedFunction();

// Shared Module/Dependency
// shared.js
export const sharedFunction = () => {
    console.log("I am a shared function");
};

The sections that follow will first present the overall operational flow and then delve into the specific code implementations within each module.

Prerequisites

Before diving deeper, it’s essential to understand the following Webpack fundamentals:

• Webpack Product Execution Flow
• Immediate-Invoked Function Expression (IIFE) in Webpack
• Modules are stored in __webpack_modules__
• Webpack uses a global array (webpackChunk) to cache loaded resources

Webpack exposes a webpackChunk array to the global (globalObject). This array is used to store loaded chunk resources. If the resource is loaded, the cache will be read, and if it is not loaded, the content will be synchronously added to the __webpack_modules__ by calling the overridden push method (aka webpackJsonpCallback ) for module integration.

Factory Object

During the Webpack compilation process, code is initially written into a custom Webpack module. Then multiple modules are aggregated into a chunk based on their file reference relationships.

// Webpack custom module
class CustomModule {
    constructor(code) {
        this.code = code;
    }
}

Dependency Object

A Dependency object is essentially an unresolved module instance. For example, the entry module ('./src/index.js') or other modules that a module relies upon are transformed into Dependency objects.

// Dependency Object
class Dependency {
    constructor(module) {
        this.module = module;
    }
}

Factory Object Resolution

Each derived class of Dependency pre-determines its corresponding factory object, and this information is stored in the dependencyFactories property of the Compilation object.

class EntryPlugin {
    apply(compiler) {
        compiler.hooks.compilation.tap(
            "EntryPlugin",
            (compilation, { normalModuleFactory }) => {
                compilation.dependencyFactories.set(
                    EntryDependency,
                    normalModuleFactory
                );
            }
        );

        const { entry, options, context } = this;
        const dep = EntryPlugin.createDependency(entry, options);

        compiler.hooks.make.tapAsync("EntryPlugin", (compilation, callback) => {
            compilation.addEntry(context, dep, options, err => {
                callback(err);
            });
        });
    }
}
class Compilation extends Tapable {
    addModuleTree({ context, dependency, contextInfo }, callback) {
        // ...
        const Dep = /** @type {DepConstructor} */ (dependency.constructor);
        const moduleFactory = this.dependencyFactories.get(Dep);
        if (!moduleFactory) {
            return callback(
                new WebpackError(
                    `No dependency factory available for this dependency type: ${dependency.constructor.name}`
                )
            );
        }
        this.handleModuleCreation(
            {
                factory: moduleFactory,
                dependencies: [dependency],
                originModule: null,
                contextInfo,
                context
            },
            (err, result) => {
                if (err && this.bail) {
                    callback(err);
                    this.buildQueue.stop();
                    this.rebuildQueue.stop();
                    this.processDependenciesQueue.stop();
                    this.factorizeQueue.stop();
                } else if (!err && result) {
                    callback(null, result);
                } else {
                    callback();
                }
            }
        );
    }
}

Example Context

Consider two projects: App1 and App2.

• App1 exposes a Button component and sets a shared React dependency:

new ModuleFederationPlugin({
    name: 'component_app',
    filename: 'remoteEntry.js',
    exposes: {
        '.': './src/Button.jsx',
    },
    shared: {
        'react': {
            version: '2.3.2'
        }
    }
})

• App2 consumes the components and libraries provided by App1:

new ModuleFederationPlugin({
    library: { type: 'module' },
    remotes: {
        'component-app': 'component_app@http://localhost:3001/remoteEntry.js',
    },
    shared: ['react'],
})

Build Flow

Product Structure

Two types of products are generated after the build process:

• Entry file
• Module chunk

The entry file exposes get and init methods along with a moduleMap.

var moduleMap = {
    "./src/Button.jsx": () => {
        return __webpack_require__.e(507).then(() => (() => ((__webpack_require__(507)))));
    }
};
var get = (module, getScope) => {
    __webpack_require__.R = getScope;
    getScope = (
        __webpack_require__.o(moduleMap, module)
            ? moduleMap[module]()
            : Promise.resolve().then(() => {
                throw new Error('Module "' + module + '" does not exist in container.');
            })
    );
    __webpack_require__.R = undefined;
    return getScope;
};
var init = (shareScope, initScope) => {
    if (!__webpack_require__.S) return;
    var oldScope = __webpack_require__.S["default"];
    var name = "default"
    if(oldScope && oldScope !== shareScope) throw new Error("Container initialization failed as it has already been initialized with a different share scope");
    __webpack_require__.S[name] = shareScope;
    return __webpack_require__.I(name, initScope);
};

__webpack_require__.d(exports, {
    get: () => (get),
    init: () => (init)
});

Execution Flow

The execution sequence involves the following steps:

• Load shared dependencies (import react from 'react')
• Load the entry file asynchronously
• Consume remote modules (import Button from 'component-app')

Reference shared dependencies ( import react from 'react ')

• Load the entry file, since the entry is processed asynchronously at this time, you can see that __webpack_require__ will be executed (ensureChunk: load asynchronous chunk content)

• __webpack_require__ will traverse ensureChunkHandlers ( __webpack_require__ .f.xxxx ), which contains __webpack_require__ .f.consumes

• Here you can see the react configured in shared , this time will traverse chunkMapping ['bootstrap_js - webpack_sharing_consume_default_react_react'], and get the corresponding loadSingletonVersionCheckFallback callback function through moduleToHandlerMapping mapping table, the execution returns a promise, where resolve returns a factory function.
• loadSingletonVersionCheckFallback will call __webpack_require. I ( RuntimeGlobals.initializeSharing ) before execution to initialize the sharedScope of the current project.

• When __webpack_require__. I is executed, the shared dependencies of the current project ( react ) are registered, and then the init method exposed by the remote module's entry file (if it exists) is called

• Execute the init method exposed by app1 , which will pass in the sharedScope of app2 , synchronizing the shared dependency information set by app2 to app1

If the shared dependency versions are different, there will be multiple version information. The shareScope information is as follows

• Similarly, calling __webpack_require__. I in app1 will also take the same method as in app2 . However, unlike app1 , there is no remotes field, so there is no initExternal method. Here is an explanation of the register function used to initialize shareScope .

var register = (name, version, factory, eager) => {
 // Get all versions of react that have been registered
  var versions = (scope[name] = scope[name] || {});

  // Find out if 17.0.1 has been initialized
  var activeVersion = versions[version];

 // If one of the following conditions is met (i.e. the if statement below is true),
   // Mount app1's shared dependencies (such as react), otherwise reuse app2's shared dependencies (such as react):

   // 1. There is no corresponding module version in app2, that is, activeVersion is empty
   // 2. The old version is not loaded, and eager is not configured (forced loading)
   // 3. The old version is not loaded, and uniqueName of app1 > uniqueName of source module, see #12124 for details
   //(uniqueName is actually the main field of packagejson)

  if (
    !activeVersion ||
    (!activeVersion.loaded &&
      (!eager != !activeVersion.eager
        ? eager
 : uniqueName > activeVersion.from))
  )
    versions[version] = {
      get: factory,
      from: uniqueName,
      eager: !!eager,
    };
};

• Back to app2, the __webpack_require__. Iexecution is complete, and the getSingletonVersion method is executed. This method is mainly used to obtain dependent versions that meet the requirements.

• After getting the version, execute the get method to get the shared dependencies.

Reference remote module ( import Button from 'component-app ' )

In app1, there is also a get method exposed. When referencing the corresponding component, the basic process and reference sharing dependency are consistent, the difference is that the final initialization will call app1's getmethod

Source Code Analysis

The underlying source code can be divided into three main parts:

• Expose Module: Exposes specific modules for other projects
• Consumption Module: Consumes remote modules
• Shared Dependency: Shares dependencies across modules

Exposing Modules: The Role of ContainerPlugin

The Expose module is principally concerned with making specific modules in a project available for use by other projects. The primary code for this functionality resides in webpack/lib/container/ContainerPlugin.

Why is it Named ContainerPlugin?

The name ContainerPlugin is derived from its function. Module Federation itself establishes a container and furnishes the modules within this created container. These containers can then be connected across different projects, enabling the modules exposed from one project to be utilized in another.

Core Functions of ContainerPlugin

ContainerPlugin performs two key operations:

1. Adding Specified Module to Compile Entry: It utilizes compilation.addEntry to add the designated module that needs to be exposed.
2. Setting the Factory Object: It sets the factory object using compilation.dependencyFactories.set.

Expose Module Workflow

The overall process of exposing a module comprises the following steps:

1. Add Entry File (remoteEntry): An entry file named remoteEntry is added to the project.
2. Set Exposed Module to Async Chunk: The exposed module is set as an asynchronous chunk, allowing for dynamic imports.
3. Inject Runtime Code: Additional runtime code is appended to the entry file, typically including methods like get and init that manage the module's behavior.

Add entry file

compiler.hooks.make.tapAsync(PLUGIN_NAME, (compilation, callback) => {
    const dep = new ContainerEntryDependency(name, exposes, shareScope);
    dep.loc = { name };
    compilation.addEntry(
        compilation.options.context,
        dep,
        {
            name,
            filename,
            runtime,
            library
        },
        error => {
            if (error) return callback(error);
            callback();
        }
    );
});

Set Factory Object

compiler.hooks.thisCompilation.tap(
    PLUGIN_NAME,
    (compilation, { normalModuleFactory }) => {
        compilation.dependencyFactories.set(
            ContainerEntryDependency,
            new ContainerEntryModuleFactory()
        );

        compilation.dependencyFactories.set(
            ContainerExposedDependency,
            normalModuleFactory
        );
    }
);

Setting up the expose module

In the Factory Object you can see that the ContainerEntryDependency factory object is ContainerEntryModuleFactory .

ContainerEntryModuleFactory will provide a create method. This will get the ContainerEntryDependency instance ( dependency ) added in the previous step, which contains the name , exposes , and shareScope information set by the user.

Then create ContainerEntryModule .

When building the ContainerEntryModule , you can see that the modules exposed by exposes are added to the block as asynchronous code. And the ContainerExposedDependency will be created based on the actual file path, which explains why the ContainerExposedDependency was added in the previous addEntry step. After adding dependency and block , the module build process will be called recursion.

Add specific runtime code to the entry file

CodeGeneration in the ContainerEntryModule will get the module information specified after compile and generate the final exposed module entry:

codeGeneration({ moduleGraph, chunkGraph, runtimeTemplate }) {
    const source = Template.asString([
        `var moduleMap = {`,
        Template.indent(getters.join(",\n")),
        "};",
        `var get = ${runtimeTemplate.basicFunction("module, getScope", [
            `${RuntimeGlobals.currentRemoteGetScope} = getScope;`,
            // reusing the getScope variable to avoid creating a new var (and module is also used later)
            "getScope = (",
            Template.indent([
                `${RuntimeGlobals.hasOwnProperty}(moduleMap, module)`,
                Template.indent([
                    "? moduleMap[module]()",
                    `: Promise.resolve().then(${runtimeTemplate.basicFunction(
                        "",
                        "throw new Error('Module \"' + module + '\" does not exist in container.');"
                    )})`
                ])
            ]),
            ");",
            `${RuntimeGlobals.currentRemoteGetScope} = undefined;`,
            "return getScope;"
        ])};`,
        `var init = ${runtimeTemplate.basicFunction("shareScope, initScope", [
            `if (!${RuntimeGlobals.shareScopeMap}) return;`,
            `var oldScope = ${RuntimeGlobals.shareScopeMap}[${JSON.stringify(
                this._shareScope
            )}];`,
            `var name = ${JSON.stringify(this._shareScope)}`,
            `if(oldScope && oldScope !== shareScope) throw new Error("Container initialization failed as it has already been initialized with a different share scope");`,
            `${RuntimeGlobals.shareScopeMap}[name] = shareScope;`,
            `return ${RuntimeGlobals.initializeSharing}(name, initScope);`
        ])};`,
        "",
        "// This exports getters to disallow modifications",
        `${RuntimeGlobals.definePropertyGetters}(exports, {`,
        Template.indent([
            `get: ${runtimeTemplate.returningFunction("get")},`,
            `init: ${runtimeTemplate.returningFunction("init")}`
        ]),
        "});"
    ]);
}

Summarized Flow:

RemoteModule Consumption

The ContainerReference module is mainly used to consume remote modules.

Its main implementation is in webpack/lib/container/ContainerReferencePlugin .

ContainerReferencePlugin It is simple and understandable to do four pieces of content:

1. Add remote module to external
2. Set the factory object (compilation.dependencyFactories.set)
3. Intercepts request parsing for remote modules ( normalModuleFactory.hooks.factorize ) and returns to generate RemoteModule
4. Add a runtime module RemoteRuntimeModule

Add remote module to external

After parsing the passed parameters, the remote module is added to external:

This also explains why the remotes parameter and the externals parameter feel so similar.

Set Factory Object

Then add the corresponding factory object for the module to be used later.

Note: fallbackis set here, this module will only trigger if multiple external settings are set

Intercept request resolution for remote modules normalModuleFactory.hooks.factorize

Next, block the module request, which returns a custom RemoteModule:

The RemoteModuleis mainly used to collect the corresponding request dependencies and collect the remote modules that need to be initialized and their chunkIDs, and place the results in the codeGenerationResultsfor use when initializing shared dependencies (if you set shared dependencies, you need to initialize shared dependencies first, and then initialize remote modules).

Add the RemoteRuntimeModule

This module will collect the compiled chunkid of all remote modules and place it in chunkMaping. Set the RuntimeGlobals.ensureChunkHandlers method accordingly. This method will be called when referencing an asynchronous chunk, and when referencing a remote module, the corresponding get method will be called to obtain the corresponding remote module.

Corresponding code address: RemoteRuntimeModule

Shared Dependency

Shared dependencies are mainly used to share the same dependency across modules.

Its main implementation is in webpack/lib/sharing/SharePlugin

Shared dependencies are divided into two parts: consuming shared dependencies ( ConsumeSharedPlugin) and providing shared dependencies ( ProvideSharedPlugin).

And SharePluginonly does parameter parsing and applies these two plugins. Therefore, it will split the two parts for parsing.

Consumption Sharing Dependency

ConsumeSharedPluginsimply understands four things:

1. Set Factory Object
2. Intercepts request resolution for shared dependencies ( normalModuleFactory.hooks.factorize ) and returns a custom ConsumeSharedModule
3. Intercepting Shared Dependency Request Parsing for Absolute Paths ( normalModuleFactory.hooks.createModule )
4. Add runtime module

Set Factory Object

Set the module factory object to be used later

Intercept request resolution for shared dependencies normalModuleFactory.hooks.factorize

Mainly intercepts module requests (import react from react ) and suffix requests (import react from 'react/’).

After interception, it will be resolved according to the locally installed dependencies/modules, and recorded to the resolveContext, createConsumeSharedModule return a custom ConsumeSharedModule

The ConsumeSharedModulecollects the shared dependencies and their chunkIDs and places the results in sourcesfor consumption by the shared dependencies #runtime module ConsumeSharedRuntimeModule .

ConsumeSharedModulealso adds shared dependencies to the AsyncDependenciesBlockof asynchronous modules

Intercepting Shared Dependency Request Resolution for Absolute Paths normalModuleFactory.hooks.createModule

Intercept requests with absolute paths and return a custom ConsumeSharedModule

Add runtime module ConsumeSharedRuntimeModule

The module consumes the data provided by the ConsumeSharedModuleand generates the initialization shared dependency runtime code:

After setting, the final production code is as follows:

Asynchronous entry:

When referencing a remote module, shared dependencies are registered first, and then the remote module is loaded.

Sync entry:

After loading the shared dependency, hang it on the shareScope for use by other modules

Provide shared dependencies

The ProvideSharedPlugin does three things:

1. Intercept request resolution of shared dependencies ( normalModuleFactory.hooks.module ) to collect information about all shared dependencies
2. Add Include
3. Set Factory Object

Intercept request resolution for shared dependencies normalModuleFactory.hooks.module

Intercept the request resolution of shared dependencies and collect all information.

Add Include

Add Include to finishMake

The whole is very similar to addEntry, the difference may be that this module has no other dependencies?

Note: This hook does not appear in the official doc, and is only used by the module federation function

Set Factory Object

Set the module factory object to be used later

Where ProvideSharedDependencycreates a ProvideSharedModulemodule.

The ProvideSharedModulewill sharedinformation and set the register function based on intercepting requests for shared dependencies. This data will be used when initializing shared dependencies.

Congrats You Made it!

And thats how Module Federation works.

If you like deep dives:

Bundler Design Trade-offs: The Rationale Behind Creating Rspack