feat: initial commit
This commit is contained in:
125
node_modules/jest-worker/README.md
generated
vendored
125
node_modules/jest-worker/README.md
generated
vendored
@@ -2,7 +2,7 @@
|
||||
|
||||
Module for executing heavy tasks under forked processes in parallel, by providing a `Promise` based interface, minimum overhead, and bound workers.
|
||||
|
||||
The module works by providing an absolute path of the module to be loaded in all forked processes. Files relative to a node module are also accepted. All methods are exposed on the parent process as promises, so they can be `await`'ed. Child (worker) methods can either be synchronous or asynchronous.
|
||||
The module works by providing an absolute path of the module to be loaded in all forked processes. All methods are exposed on the parent process as promises, so they can be `await`'ed. Child (worker) methods can either be synchronous or asynchronous.
|
||||
|
||||
The module also implements support for bound workers. Binding a worker means that, based on certain parameters, the same task will always be executed by the same worker. The way bound workers work is by using the returned string of the `computeWorkerKey` method. If the string was used before for a task, the call will be queued to the related worker that processed the task earlier; if not, it will be executed by the first available worker, then sticked to the worker that executed it; so the next time it will be processed by the same worker. If you have no preference on the worker executing the task, but you have defined a `computeWorkerKey` method because you want _some_ of the tasks to be sticked, you can return `null` from it.
|
||||
|
||||
@@ -11,7 +11,7 @@ The list of exposed methods can be explicitly provided via the `exposedMethods`
|
||||
## Install
|
||||
|
||||
```sh
|
||||
$ yarn add jest-worker
|
||||
yarn add jest-worker
|
||||
```
|
||||
|
||||
## Example
|
||||
@@ -20,11 +20,11 @@ This example covers the minimal usage:
|
||||
|
||||
### File `parent.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
import {Worker as JestWorker} from 'jest-worker';
|
||||
|
||||
async function main() {
|
||||
const worker = new JestWorker(require.resolve('./Worker'));
|
||||
const worker = new JestWorker(require.resolve('./worker'));
|
||||
const result = await worker.hello('Alice'); // "Hello, Alice"
|
||||
}
|
||||
|
||||
@@ -33,45 +33,29 @@ main();
|
||||
|
||||
### File `worker.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
export function hello(param) {
|
||||
return 'Hello, ' + param;
|
||||
return `Hello, ${param}`;
|
||||
}
|
||||
```
|
||||
|
||||
## Experimental worker
|
||||
|
||||
Node 10 shipped with [worker-threads](https://nodejs.org/api/worker_threads.html), a "threading API" that uses SharedArrayBuffers to communicate between the main process and its child threads. This experimental Node feature can significantly improve the communication time between parent and child processes in `jest-worker`.
|
||||
Node shipped with [`worker_threads`](https://nodejs.org/api/worker_threads.html), a "threading API" that uses `SharedArrayBuffers` to communicate between the main process and its child threads. This feature can significantly improve the communication time between parent and child processes in `jest-worker`.
|
||||
|
||||
Since `worker_threads` are considered experimental in Node, you have to opt-in to this behavior by passing `enableWorkerThreads: true` when instantiating the worker. While the feature was unflagged in Node 11.7.0, you'll need to run the Node process with the `--experimental-worker` flag for Node 10.
|
||||
To use `worker_threads` instead of default `child_process` you have to pass `enableWorkerThreads: true` when instantiating the worker.
|
||||
|
||||
## API
|
||||
|
||||
The `Worker` export is a constructor that is initialized by passing the worker path, plus an options object.
|
||||
|
||||
### `workerPath: string` (required)
|
||||
### `workerPath: string | URL` (required)
|
||||
|
||||
Node module name or absolute path of the file to be loaded in the child processes. Use `require.resolve` to transform a relative path into an absolute one.
|
||||
Node module name or absolute path or file URL of the file to be loaded in the child processes. You can use `require.resolve` to transform a relative path into an absolute one.
|
||||
|
||||
### `options: Object` (optional)
|
||||
|
||||
#### `exposedMethods: $ReadOnlyArray<string>` (optional)
|
||||
|
||||
List of method names that can be called on the child processes from the parent process. You cannot expose any method named like a public `Worker` method, or starting with `_`. If you use method auto-discovery, then these methods will not be exposed, even if they exist.
|
||||
|
||||
#### `numWorkers: number` (optional)
|
||||
|
||||
Amount of workers to spawn. Defaults to the number of CPUs minus 1.
|
||||
|
||||
#### `maxRetries: number` (optional)
|
||||
|
||||
Maximum amount of times that a dead child can be re-spawned, per call. Defaults to `3`, pass `Infinity` to allow endless retries.
|
||||
|
||||
#### `forkOptions: Object` (optional)
|
||||
|
||||
Allow customizing all options passed to `childProcess.fork`. By default, some values are set (`cwd`, `env` and `execArgv`), but you can override them and customize the rest. For a list of valid values, check [the Node documentation](https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options).
|
||||
|
||||
#### `computeWorkerKey: (method: string, ...args: Array<any>) => ?string` (optional)
|
||||
#### `computeWorkerKey: (method: string, ...args: Array<unknown>) => string | null` (optional)
|
||||
|
||||
Every time a method exposed via the API is called, `computeWorkerKey` is also called in order to bound the call to a worker. This is useful for workers that are able to cache the result or part of it. You bound calls to a worker by making `computeWorkerKey` return the same identifier for all different calls. If you do not want to bind the call to any worker, return `null`.
|
||||
|
||||
@@ -79,19 +63,61 @@ The callback you provide is called with the method name, plus all the rest of th
|
||||
|
||||
By default, no process is bound to any worker.
|
||||
|
||||
#### `setupArgs: Array<mixed>` (optional)
|
||||
#### `enableWorkerThreads: boolean` (optional)
|
||||
|
||||
By default, `jest-worker` will use `child_process` threads to spawn new Node.js processes. If you prefer [`worker_threads`](https://nodejs.org/api/worker_threads.html) instead, pass `enableWorkerThreads: true`.
|
||||
|
||||
#### `exposedMethods: ReadonlyArray<string>` (optional)
|
||||
|
||||
List of method names that can be called on the child processes from the parent process. You cannot expose any method named like a public `Worker` method, or starting with `_`. If you use method auto-discovery, then these methods will not be exposed, even if they exist.
|
||||
|
||||
#### `forkOptions: ForkOptions` (optional)
|
||||
|
||||
Allow customizing all options passed to `child_process.fork`. By default, some values are set (`cwd`, `env`, `execArgv` and `serialization`), but you can override them and customize the rest. For a list of valid values, check [the Node documentation](https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options).
|
||||
|
||||
#### `idleMemoryLimit: number` (optional)
|
||||
|
||||
Specifies the memory limit for workers before they are recycled and is primarily a work-around for [this issue](https://github.com/jestjs/jest/issues/11956);
|
||||
|
||||
After the worker has executed a task the memory usage of it is checked. If it exceeds the value specified the worker is killed and restarted. If no limit is set this process does not occur. The limit can be specified in 2 ways:
|
||||
|
||||
- `<= 1` - The value is assumed to be a percentage of system memory. So 0.5 sets the memory limit of the worker to half of the total system memory
|
||||
- `\> 1` - Assumed to be a fixed byte value. Because of the previous rule if you wanted a value of 1 byte (I don't know why) you could use `1.1`.
|
||||
|
||||
#### `maxRetries: number` (optional)
|
||||
|
||||
Maximum amount of times that a dead child can be re-spawned, per call. Defaults to `3`, pass `Infinity` to allow endless retries.
|
||||
|
||||
#### `numWorkers: number` (optional)
|
||||
|
||||
Amount of workers to spawn. Defaults to the number of CPUs minus 1.
|
||||
|
||||
#### `resourceLimits: ResourceLimits` (optional)
|
||||
|
||||
The `resourceLimits` option which will be passed to `worker_threads` workers.
|
||||
|
||||
#### `silent: Boolean` (optional)
|
||||
|
||||
Set to false for `stdout` and `stderr` to be logged to console.
|
||||
|
||||
By default this is true.
|
||||
|
||||
#### `setupArgs: Array<unknown>` (optional)
|
||||
|
||||
The arguments that will be passed to the `setup` method during initialization.
|
||||
|
||||
#### `WorkerPool: (workerPath: string, options?: WorkerPoolOptions) => WorkerPoolInterface` (optional)
|
||||
#### `taskQueue: TaskQueue` (optional)
|
||||
|
||||
Provide a custom worker pool to be used for spawning child processes. By default, Jest will use a node thread pool if available and fall back to child process threads.
|
||||
The task queue defines in which order tasks (method calls) are processed by the workers. `jest-worker` ships with a `FifoQueue` and `PriorityQueue`:
|
||||
|
||||
#### `enableWorkerThreads: boolean` (optional)
|
||||
- `FifoQueue` (default): Processes the method calls (tasks) in the call order.
|
||||
- `PriorityQueue`: Processes the method calls by a computed priority in natural ordering (lower priorities first). Tasks with the same priority are processed in any order (FIFO not guaranteed). The constructor accepts a single argument, the function that is passed the name of the called function and the arguments and returns a numerical value for the priority: `new require('jest-worker').PriorityQueue((method, filename) => filename.length)`.
|
||||
|
||||
`jest-worker` will automatically detect if `worker_threads` are available, but will not use them unless passed `enableWorkerThreads: true`.
|
||||
#### `WorkerPool: new (workerPath: string, options?: WorkerPoolOptions) => WorkerPoolInterface` (optional)
|
||||
|
||||
### `workerSchedulingPolicy: 'round-robin' | 'in-order'` (optional)
|
||||
Provide a custom WorkerPool class to be used for spawning child processes.
|
||||
|
||||
#### `workerSchedulingPolicy: 'round-robin' | 'in-order'` (optional)
|
||||
|
||||
Specifies the policy how tasks are assigned to workers if multiple workers are _idle_:
|
||||
|
||||
@@ -100,13 +126,6 @@ Specifies the policy how tasks are assigned to workers if multiple workers are _
|
||||
|
||||
Tasks are always assigned to the first free worker as soon as tasks start to queue up. The scheduling policy does not define the task scheduling which is always first-in, first-out.
|
||||
|
||||
### `taskQueue`: TaskQueue` (optional)
|
||||
|
||||
The task queue defines in which order tasks (method calls) are processed by the workers. `jest-worker` ships with a `FifoQueue` and `PriorityQueue`:
|
||||
|
||||
- `FifoQueue` (default): Processes the method calls (tasks) in the call order.
|
||||
- `PriorityQueue`: Processes the method calls by a computed priority in natural ordering (lower priorities first). Tasks with the same priority are processed in any order (FIFO not guaranteed). The constructor accepts a single argument, the function that is passed the name of the called function and the arguments and returns a numerical value for the priority: `new require('jest-worker').PriorityQueue((method, filename) => filename.length)`.
|
||||
|
||||
## JestWorker
|
||||
|
||||
### Methods
|
||||
@@ -121,11 +140,17 @@ Returns a `ReadableStream` where the standard output of all workers is piped. No
|
||||
|
||||
Returns a `ReadableStream` where the standard error of all workers is piped. Note that the `silent` option of the child workers must be set to `true` to make it work. This is the default set by `jest-worker`, but keep it in mind when overriding options through `forkOptions`.
|
||||
|
||||
#### `start()`
|
||||
|
||||
Starts up every worker and calls their `setup` function, if it exists. Returns a `Promise` which resolves when all workers are running and have completed their `setup`.
|
||||
|
||||
This is useful if you want to start up all your workers eagerly before they are used to call any other functions.
|
||||
|
||||
#### `end()`
|
||||
|
||||
Finishes the workers by killing all workers. No further calls can be done to the `Worker` instance.
|
||||
|
||||
Returns a Promise that resolves with `{ forceExited: boolean }` once all workers are dead. If `forceExited` is `true`, at least one of the workers did not exit gracefully, which likely happened because it executed a leaky task that left handles open. This should be avoided, force exiting workers is a last resort to prevent creating lots of orphans.
|
||||
Returns a `Promise` that resolves with `{ forceExited: boolean }` once all workers are dead. If `forceExited` is `true`, at least one of the workers did not exit gracefully, which likely happened because it executed a leaky task that left handles open. This should be avoided, force exiting workers is a last resort to prevent creating lots of orphans.
|
||||
|
||||
**Note:**
|
||||
|
||||
@@ -135,7 +160,7 @@ Consider deliberately leaving this Promise floating (unhandled resolution). Afte
|
||||
|
||||
### Worker IDs
|
||||
|
||||
Each worker has a unique id (index that starts with `1`), which is available inside the worker as `process.env.JEST_WORKER_ID`.
|
||||
Each worker has a unique id (index that starts with `'1'`), which is available inside the worker as `process.env.JEST_WORKER_ID`.
|
||||
|
||||
## Setting up and tearing down the child process
|
||||
|
||||
@@ -152,11 +177,11 @@ This example covers the standard usage:
|
||||
|
||||
### File `parent.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
import {Worker as JestWorker} from 'jest-worker';
|
||||
|
||||
async function main() {
|
||||
const myWorker = new JestWorker(require.resolve('./Worker'), {
|
||||
const myWorker = new JestWorker(require.resolve('./worker'), {
|
||||
exposedMethods: ['foo', 'bar', 'getWorkerId'],
|
||||
numWorkers: 4,
|
||||
});
|
||||
@@ -176,13 +201,13 @@ main();
|
||||
|
||||
### File `worker.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
export function foo(param) {
|
||||
return 'Hello from foo: ' + param;
|
||||
return `Hello from foo: ${param}`;
|
||||
}
|
||||
|
||||
export function bar(param) {
|
||||
return 'Hello from bar: ' + param;
|
||||
return `Hello from bar: ${param}`;
|
||||
}
|
||||
|
||||
export function getWorkerId() {
|
||||
@@ -196,11 +221,11 @@ This example covers the usage with a `computeWorkerKey` method:
|
||||
|
||||
### File `parent.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
import {Worker as JestWorker} from 'jest-worker';
|
||||
|
||||
async function main() {
|
||||
const myWorker = new JestWorker(require.resolve('./Worker'), {
|
||||
const myWorker = new JestWorker(require.resolve('./worker'), {
|
||||
computeWorkerKey: (method, filename) => filename,
|
||||
});
|
||||
|
||||
@@ -226,7 +251,7 @@ main();
|
||||
|
||||
### File `worker.js`
|
||||
|
||||
```javascript
|
||||
```js
|
||||
import babel from '@babel/core';
|
||||
|
||||
const cache = Object.create(null);
|
||||
|
||||
Reference in New Issue
Block a user