The Art of Generators

The topic shares referenced articles from Teacher Ruanyifeng's ECMAScript6 introduction => https://es6.ruanyifeng.com/#docs/generator#%E7%AE%80%E4%BB%8B

Refer to the list:

Teacher Ruanyifeng's "The Meaning and Usage of Generator Functions": Click to refer
The article "Asynchronous Programming Solution Generator" by the author Frontend Ronaldo on the Xitu platform: Click to refer

Intuitive Use#

function* printStrGenerator() {
    let str1 = yield "Hello";
    console.log(str1);
    let str2 = yield str1 + "CodeGorgeous";
    console.log(str2);
    return "Done";
}

const print = printStrGenerator();
console.log(print); // printStrGenerator {<suspended>}
console.log(print.next()); // {done: false, value: "Hello"}
console.log(print.next("My is")); // {done: false, value: "CodeGorgeous"}
console.log(print.next("Test")); // {done: true, value: "Done"}
console.log(print.next()); // {done: true, value: undefined}
console.log(print); // printStrGenerator {<closed>}

Background of Generator#

In JavaScript, because JavaScript operates in a single-threaded manner, if we wait for a request response before continuing to execute code, it will inevitably waste a very unstable amount of time. Consequently, to solve this problem, a method called asynchronous code programming was introduced. It can be roughly understood that when the code reaches an asynchronous operation, it will continue to execute subsequent code, waiting for the asynchronous operation to complete before triggering the corresponding operation and then continuing to execute synchronous code. Accordingly, several solutions were proposed:

Callback functions
Event listeners
Publish/Subscribe
Promise

Let's revisit these solutions and their pros and cons through a few examples

Callback Functions

Example:

// File copy operation
fs.readFile("./test/index.html", 'utf-8', (err, data) => {
    if (err) return;
    // Create a file and copy the content
    fs.writeFile("./test/index-copy.html", data, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful");
    })
});

Advantages:
1. Does not block code execution, easy to write
Disadvantages:
1. An asynchronous operation depends on the previous asynchronous operation, easily forming callback hell (high coupling)
2. Code is not easy to maintain

Event Listeners

Example:

const fn = () => {
    fs.readFile("./test/index.html", 'utf-8', (err, data) => {
        if (err) return;
        fn.trigger("readFileOver", data);
    });
};

fn.eventList = [];

fn.on = (type, callback) => {
    fn.eventList.push({
        type,
        fn: callback
    });
};

fn.trigger = (type, ...args) => {
    fn.eventList.forEach(item => {
        (item.type === type) && (item.fn(...args));
    });
};

fn.on("readFileOver", (data) => {
    fs.writeFile("./test/index-copy.html", data, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful");
    });
});

fn();

Advantages:
1. Compared to the callback function method, it basically does not form callback hell (reducing coupling)
2. Can bind multiple events and configure callback functions separately

Publish/Subscribe (Observer Pattern)

Example:

class EventBus {
    constructor() {
        this.event = [];
    };
    /**
    * Subscribe
    * @param {*} eventType Event type
    * @param {*} callback Callback function
    */
    add(eventType, callback) {
        this.event.push({
            type: eventType,
            fn: callback
        });
    };
    /**
    * Publish
    * @param {*} eventType Event type
    * @param  {...any} args Parameters
    */
    trigger(eventType, ...args) {
        // No cleanup operation for triggered events here
        this.event.forEach(item => {
            if (item.type === eventType) {
                item.fn(...args);
            }
        });
    }
};
// Create an instance
const eventBus = new EventBus();

// Subscribe to readFileOver event
eventBus.add("readFileOver", (data) => {
    fs.writeFile("./test/index-copy1.html", data, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful 1");
    });
});
eventBus.add("readFileOver", (data) => {
    fs.writeFile("./test/index-copy2.html", data, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful 2");
    });
});

fs.readFile("./test/index.html", 'utf-8', (err, data) => {
    if (err) return;
    // Publish readFileOver event
    eventBus.trigger("readFileOver", data);
});

Advantages:
1. Compared to event listeners, the publish/subscribe model is easier to manage, and the triggering mechanism is clearer to understand the current event situation

Promise

Example:

new Promise((resolve, reject) => {
    fs.readFile("./test/index.html", 'utf-8', (err, data) => {
        if (err) reject(err);
        resolve(data);
    });
}).then(resp => {
    // Create a file and copy the content
    fs.writeFile("./test/index-copy.html", resp, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful");
    });
}).catch(err => {
    console.log("Error:", err);
});

Advantages:
1. Changes from traditional callback function embedded mode to chain call mode
2. Clearer code flow
3. Improved readability

Asynchronous programming solutions have long existed in traditional programming languages, one of which is called coroutines, which generally means that individual threads cooperate to complete a certain asynchronous task.
In JavaScript, it appears in the form of Generator functions, taking file copying as an example:

function* copyFile() {
    const data = yield fs.readFile("./test/index.html", 'utf-8', (err, data) => {
        if (err) return;
        else fn.next(data);
    });
    yield fs.writeFile("./test/index-copy.html", data, 'utf-8', (err) => {
        if (err) return;
        console.log("Copy successful");
        fn.next();
    })
};
const fn = copyFile();
fn.next()

Why can Generator solve asynchronous problems?
One of the most important characteristics of Generator is that it can yield the execution right of the function, which represents whether the code inside the function can execute and the power to pause the execution of the code inside the function. By manually controlling the execution right, the function can continue executing its code at the appropriate time (for example: after receiving the request response).

Using Generators#

How to create a Generator function
The biggest difference between a Generator and a regular function is that when declaring a Generator function, it needs to have an asterisk (*) symbol, for example:

function* generator() {}
function * generator() {}
function*generator() {}
function *generator() {}

All four are valid ways to write a Generator function (no superiority among them). Generally, the first format is recommended, as most code formatters will format Generators to this style. It should also be noted that Generator functions cannot be used as constructors.

Execution characteristics of Generator functions
When you execute a Generator function, you will find that nothing you wrote inside that function starts executing. This is a manifestation of your ability to manage the function's execution right. The return value you receive from that function will default to a suspended generator object, and you can use the next() method on that object to execute the code execution right once. The range of code executed this time is: if it is the first execution, it is from the start of the function to the next yield expression/return position; otherwise, it is from the last yield expression position to the next yield expression/return position. Each time you execute the next() method, you will get an object: {value: undefined, done: false}, where value represents the return value of the yield expression/return expression, and done indicates whether this is the last execution of the function's code right; true means that subsequent uses of the next() function will be meaningless, while false means otherwise.

Illustration of Generator execution
Generator Execution Illustration

yield expression#

From the above introduction and characteristics of Generators, we can understand the importance of the yield expression in Generators. yield represents the range of code execution each time (from the previous yield to this yield). Similarly, can yield be used in other functions? Obviously not, because using it in other functions has no meaning, as other functions cannot control the execution right of the function inside.

One important point to note is that the return value of yield will not be assigned to a variable; the value of yield will appear as the return value of the next() function execution. The parameter you pass when calling the next function will appear as the result of the last yield execution in the assignment operation. Describing this in words can be quite confusing, so let’s use a diagram to explain:
Relationship between yield and next

Generator and Iterator#

Understanding Iterator
Iterator is a new traversal mechanism introduced in ES6, which has the following characteristics:

Has a Symbol.iterator interface property
Can be traversed using for...of (for...of can only traverse data with the Iterator interface)
The function returned by the Symbol.iterator property has Generator characteristics, i.e., it is a generator that controls execution rights.
Since Generator is an iterator generator, we can manually add an iterator interface to objects that do not have the Symbol.iterator property, allowing those objects to be traversed using for...of and other methods. If interested, you can take a look at the following code:

const obj = {};
obj[Symbol.iterator] = function* () {
    yield "CodeGorgeous";
    yield "MaoMao";
    yield "BestFriend";
};
for (const item of arr) {
    console.log(item);// CodeGorgeous // MaoMao // BestFriend
};
const arrObj = [...obj];
console.log(arrObj);// [ 'CodeGorgeous', 'MaoMao', 'BestFriend' ]

Generator and throw#

In a normal piece of JavaScript code, we can wrap a piece of code with a try-catch expression to prevent it from affecting the execution of other code when an error occurs. If you have written or seen some third-party library code, you may sometimes see that a function first checks whether the passed parameters meet the expected types, and if not, it manually uses throw to throw an error to prompt the developer. In Generators, there is an additional throw method used to throw errors within the generator.

function* printStrGenerator() {
    let index = 0;
    while (true) {
        try {
            yield `This is ${index ++}`
        } catch (error) {
            console.log("Error caught inside Generator", error);
        }
    }
};

const g = printStrGenerator();
console.log(g.next()); // {value: 'This is 0', done: false}
try {
    g.throw(new Error("An expected error occurred in the code")); // Error caught inside Generator Error: An expected error occurred in the code
    g.throw(new Error("An expected error occurred in the code")); // Error caught inside Generator Error: An expected error occurred in the code
} catch (error) {
    console.log("Error caught outside");
}
console.log(g.next()); // {value: 'This is 3', done: false}

From this example, we can see that Generator.prototype.throw not only has the function of throwing errors but also has the characteristic of executing next() once. In this example, because I used an infinite while loop + try-catch, every time an error is thrown, it will be caught inside the Generator (but you must be clear that each time this part is in a different block scope, not the same scope continuously). If you write each yield expression and it is not caught by the function's internal try-catch, or if the current Generator is already closed but you still throw, these two situations will be caught by the corresponding try-catch outside (internal and external catches have completely different meanings). It should also be noted that if an error is thrown inside the Generator function execution process internally but not caught (ps: it must be caught externally, because if the external does not catch it, the program will crash), it will terminate the execution of the Generator. If you continue to execute next afterwards, it will only return {value: undefined, done: true}. It feels like this explanation is getting a bit vague, so let’s establish a control group to clearly observe the differences:

Generator internal not caught, caught by external

function* printStrGenerator() {
    let index = 0;
    while (true) {
        yield `This is ${index ++}`;
    }
};

const g = printStrGenerator();
try {
    console.log(g.next()); // {value: 'This is 0', done: false}
    console.log(g.throw(new Error("This is Error"))); // This step will cause the Generator to throw an error
} catch (error) {
    console.log(g.next()); // {value: undefined, done: true}
    console.log(g); // printStrGenerator {<closed>}
}

Generator internal caught, external not caught

function* printStrGenerator() {
    let index = 0;
    while (true) {
        try {
            yield `This is ${index ++}`;
        } catch (error) {
            console.log(error);
        }
    }
};

const g = printStrGenerator();
console.log(g.next()); // {value: 'This is 0', done: false}
console.log(g.throw(new Error("This is Error"))); // Error: This is Error // {value: 'This is 1', done: false}
console.log(g.next()); // {value: 'This is 2', done: false}
console.log(g); // printStrGenerator {<suspended>}

Generator and return#

In normal circumstances, return appears to terminate the function execution and return undefined by default. Here we discuss Generator.prototype.return, why I mention these two different levels (one is syntax level, the other is a function), but in Generators, the return in the prototype chain (if you are not familiar with the prototype chain, you can look at the additional knowledge section) can achieve the same effect. Let’s observe an example to see the effect:

function* printStrGenerator() {
    let index = 0;
    while (true) {
        yield `This is ${index ++}`;
    }
};

const g = printStrGenerator();

console.log(g.next()); // {value: 'This is 0', done: false}
console.log(g.return("Done")); // {value: 'Done', done: true}
console.log(g.next()); // {value: undefined, done: true}

Handling Nested Generators#

One scenario to consider is that I have separated different asynchronous tasks into different generators, and I can use a main Generator to manage them. This idea is indeed nice; on one hand, it enhances reusability, on the other hand, it allows for logical separation (no need to think about the internal workings of the function when using it), and on another hand, it improves maintainability. When a certain Generator has an issue, you only need to fix that function without adjusting other parts of the code. Now, let’s try it out:

function* getUserInfoGenerator(generator) {
    return yield (setTimeout(() => {
        const useInfo = {
            nickname: "CodeGorgeous",
            openid: "aszxzxcISadaxzxxxxxx"
        };
        console.log("User information retrieved successfully");
        generator && generator(useInfo);
    }, Math.random() * 5000));
}

function* getProjectConfigGenerator(generator) {
    return yield (setTimeout(() => {
        const projectConfig = {
            theme: "default",
            lang: "zh-CN",
            title: "Focus on one point, reach the peak⚡"
        };
        console.log("Project configuration retrieved successfully");
        generator && generator(projectConfig);
    }, Math.random() * 5000))
}

function* dispatchGenerator() {
    yield "1";
    console.log("Starting to retrieve user information");
    const useInfo = yield getUserInfoGenerator;
    yield "2";
    console.log(`Starting to retrieve project configuration`, useInfo);
    // console.log(`Starting to retrieve project configuration for ${useInfo && useInfo.nickname} user`, useInfo);
    const projectConfig = yield getProjectConfigGenerator;
    console.log(projectConfig);
}

// Execute dispatch
const g = dispatchGenerator();
// Here is a pure spontaneous idea to create a Generator auto-executor (writing this is quite torturous, my hair is starting to fall out again)

function run(generator) {
    function next(data) {
        var result = generator.next(data);
        if (result.done) return;
        if (typeof result.value === 'function' && result.value[Symbol.toStringTag] === "GeneratorFunction") {
            const temp = result.value(next);
            run(temp);
        } else {
            next();
        }
    }
    next();
};
run(g);

However, in Generators, the yield expression provides another way to flatten nested Generators, which is to use yield* when encountering an inner Generator. Let’s slightly modify the code we wrote above:

function* getUserInfoGenerator(generator) {
    return yield (setTimeout(() => {
        const useInfo = {
            nickname: "CodeGorgeous",
            openid: "aszxzxcISadaxzxxxxxx"
        };
        console.log("User information retrieved successfully");
        generator && generator.next(useInfo);
    }, Math.random() * 5000));
}

function* getProjectConfigGenerator(generator) {
    return yield (setTimeout(() => {
        const projectConfig = {
            theme: "default",
            lang: "zh-CN",
            title: "Focus on one point, reach the peak⚡"
        };
        console.log("Project configuration retrieved successfully");
        generator && generator.next(projectConfig);
    }, Math.random() * 5000))
}

function* dispatchGenerator() {
    console.log("Starting to retrieve user information");
    const useInfo = yield* getUserInfoGenerator(g);
    console.log(`Starting to retrieve project configuration for ${useInfo.nickname} user`, useInfo);
    const projectConfig = yield* getProjectConfigGenerator(g);
    console.log(projectConfig);
}

// Execute dispatch
const g = dispatchGenerator();
g.next();

One point to note is that the handling of asynchronous operations in the two examples I demonstrated above is not reasonable. Although it can achieve functionality, for real applications in asynchronous operations, please refer to Generator Asynchronous Scenario Applications, where I will rewrite the code in a standardized and reasonable manner.

A Casual Talk on Generators and Coroutines#

This part is intended to be documented in a separate article, as I feel it might stray off topic.

Generator Asynchronous Scenario Applications#

Generators themselves cannot handle asynchronous scenarios; their main function is to serve as containers for asynchronous operations. In regular development, Generators are generally not used. One library I encountered (redux-saga) is primarily developed based on Generators, which you might want to explore in the React ecosystem. So how can we solve this in regular development? At this point, we can use the asynchronous solutions we mentioned at the beginning. Below, I will demonstrate two solutions:

Solutions:

Callback Functions

The timing of the callback function returns the function execution right promptly

    function getUserInfoGenerator(callback) {
        return (setTimeout(() => {
            const useInfo = {
                nickname: "CodeGorgeous",
                openid: "aszxzxcISadaxzxxxxxx"
            };
            console.log("User information retrieved successfully");
            callback(useInfo);
        }, Math.random() * 5000))
    }

    function getProjectConfigGenerator(callback) {
        return (setTimeout(() => {
            const projectConfig = {
                theme: "default",
                lang: "zh-CN",
                title: "Focus on one point, reach the peak⚡"
            };
            console.log("Project configuration retrieved successfully");
            callback(projectConfig);
        }, Math.random() * 5000))
    }

    function* dispatchGenerator() {
        console.log("Starting to retrieve user information");
        const useInfo = yield getUserInfoGenerator;
        console.log(`Starting to retrieve project configuration for ${useInfo && useInfo.nickname} user`, useInfo);
        const projectConfig = yield getProjectConfigGenerator;
        console.log(projectConfig);
    }

    // Execute dispatch
    const g = dispatchGenerator();

    function run(generator) {
        function next(data) {
            var result = generator.next(data);
            if (result.done) return;
            if (typeof result.value === 'function') {
                result.value((data) => {
                    next(data);
                });
            } else {
                next(result.value);
            }
        }
        next();
    };
    run(g);

Promise

The timing of the then returns the function execution right promptly

        function getUserInfoGenerator() {
            return new Promise(resolve => {
                setTimeout(() => {
                    const useInfo = {
                        nickname: "CodeGorgeous",
                        openid: "aszxzxcISadaxzxxxxxx"
                    };
                    console.log("User information retrieved successfully");
                    resolve(useInfo);
                }, Math.random() * 5000)
            });
        }

        function getProjectConfigGenerator() {
            return new Promise(resolve => {
                setTimeout(() => {
                    const projectConfig = {
                        theme: "default",
                        lang: "zh-CN",
                        title: "Focus on one point, reach the peak⚡"
                    };
                    console.log("Project configuration retrieved successfully");
                    resolve(projectConfig);
                }, Math.random() * 5000)
            })
        }

        function* dispatchGenerator() {
            console.log("Starting to retrieve user information");
            const useInfo = yield getUserInfoGenerator();
            console.log(`Starting to retrieve project configuration for ${useInfo && useInfo.nickname} user`, useInfo);
            const projectConfig = yield getProjectConfigGenerator();
            console.log(projectConfig);
        }

        // Execute dispatch
        const g = dispatchGenerator();

        function run(generator) {
            function next(data) {
                var result = generator.next(data);
                if (result.done) return;
                if (typeof result.value === 'object' && result.value.constructor.name === "Promise") {
                    result.value.then(data => {
                        next(data);
                    });
                } else {
                    next(result.value);
                }
            }
            next();
        };
        run(g);

Generators and async#

async is a function introduced in ES2017, and its emergence makes managing asynchronous operations more convenient and concise. Why do we transition from discussing Generators to async? Let’s observe the differences between the two approaches:

Generator approach

function getUserInfoGenerator(callback) {
    return (setTimeout(() => {
        const useInfo = {
            nickname: "CodeGorgeous",
            openid: "aszxzxcISadaxzxxxxxx"
        };
        console.log("User information retrieved successfully");
        callback(useInfo);
    }, Math.random() * 5000))
}

function getProjectConfigGenerator(callback) {
    return (setTimeout(() => {
        const projectConfig = {
            theme: "default",
            lang: "zh-CN",
            title: "Focus on one point, reach the peak⚡"
        };
        console.log("Project configuration retrieved successfully");
        callback(projectConfig);
    }, Math.random() * 5000))
}

function* dispatchGenerator() {
    console.log("Starting to retrieve user information");
    const useInfo = yield getUserInfoGenerator;
    console.log(`Starting to retrieve project configuration for ${useInfo && useInfo.nickname} user`, useInfo);
    const projectConfig = yield getProjectConfigGenerator;
    console.log(projectConfig);
}

// Execute dispatch
const g = dispatchGenerator();

function run(generator) {
    function next(data) {
        var result = generator.next(data);
        if (result.done) return;
        if (typeof result.value === 'object' && result.value.constructor.name === "Promise") {
            result.value.then(data => {
                next(data);
            });
        } else {
            next(result.value);
        }
    }
    next();
};
run(g);

Async approach

function getUserInfoGenerator() {
    return new Promise(resolve => {
        setTimeout(() => {
            const useInfo = {
                nickname: "CodeGorgeous",
                openid: "aszxzxcISadaxzxxxxxx"
            };
            console.log("User information retrieved successfully");
            resolve(useInfo);
        }, Math.random() * 5000)
    });
}

function getProjectConfigGenerator() {
    return new Promise(resolve => {
        setTimeout(() => {
            const projectConfig = {
                theme: "default",
                lang: "zh-CN",
                title: "Focus on one point, reach the peak⚡"
            };
            console.log("Project configuration retrieved successfully");
            resolve(projectConfig);
        }, Math.random() * 5000)
    })
}

async function dispatchGenerator() {
    console.log("Starting to retrieve user information");
    const useInfo = await getUserInfoGenerator();
    console.log(`Starting to retrieve project configuration for ${useInfo && useInfo.nickname} user`, useInfo);
    const projectConfig = await getProjectConfigGenerator();
    console.log(projectConfig);
}
dispatchGenerator();

We can see that the differences in writing between the two approaches are:

The asterisk (*) in the Generator function is replaced by async.
yield is replaced by await.
The Generator function requires an executor to automatically execute to completion, while the async function can be called directly.
Therefore, theoretically, async is syntactic sugar for Generator functions, providing a more concise way to write code, eliminating the need for a Generator executor. Thus, in theory, Generators are indirectly present in the code, even if we do not perceive them directly.