mirror of
https://github.com/cloudflare/wrangler-action.git
synced 2024-11-22 18:13:24 +01:00
384 lines
11 KiB
Markdown
384 lines
11 KiB
Markdown
|
# tinybench
|
||
|
|
||
|
Benchmark your code easily with Tinybench, a simple, tiny and light-weight `7KB` (`2KB` minified and gzipped)
|
||
|
benchmarking library!
|
||
|
You can run your benchmarks in multiple JavaScript runtimes, Tinybench is
|
||
|
completely based on the Web APIs with proper timing using `process.hrtime` or
|
||
|
`performance.now`.
|
||
|
|
||
|
- Accurate and precise timing based on the environment
|
||
|
- `Event` and `EventTarget` compatible events
|
||
|
- Statistically analyzed values
|
||
|
- Calculated Percentiles
|
||
|
- Fully detailed results
|
||
|
- No dependencies
|
||
|
|
||
|
_In case you need more tiny libraries like tinypool or tinyspy, please consider submitting an [RFC](https://github.com/tinylibs/rfcs)_
|
||
|
|
||
|
## Installing
|
||
|
|
||
|
```bash
|
||
|
$ npm install -D tinybench
|
||
|
```
|
||
|
|
||
|
## Usage
|
||
|
|
||
|
You can start benchmarking by instantiating the `Bench` class and adding
|
||
|
benchmark tasks to it.
|
||
|
|
||
|
```js
|
||
|
import { Bench } from 'tinybench';
|
||
|
|
||
|
const bench = new Bench({ time: 100 });
|
||
|
|
||
|
bench
|
||
|
.add('faster task', () => {
|
||
|
console.log('I am faster')
|
||
|
})
|
||
|
.add('slower task', async () => {
|
||
|
await new Promise(r => setTimeout(r, 1)) // we wait 1ms :)
|
||
|
console.log('I am slower')
|
||
|
})
|
||
|
.todo('unimplemented bench')
|
||
|
|
||
|
await bench.run();
|
||
|
|
||
|
console.table(bench.table());
|
||
|
|
||
|
// Output:
|
||
|
// ┌─────────┬───────────────┬──────────┬────────────────────┬───────────┬─────────┐
|
||
|
// │ (index) │ Task Name │ ops/sec │ Average Time (ns) │ Margin │ Samples │
|
||
|
// ├─────────┼───────────────┼──────────┼────────────────────┼───────────┼─────────┤
|
||
|
// │ 0 │ 'faster task' │ '41,621' │ 24025.791819761525 │ '±20.50%' │ 4257 │
|
||
|
// │ 1 │ 'slower task' │ '828' │ 1207382.7838323202 │ '±7.07%' │ 83 │
|
||
|
// └─────────┴───────────────┴──────────┴────────────────────┴───────────┴─────────┘
|
||
|
|
||
|
console.table(
|
||
|
bench.todos.map(({ name }) => ({
|
||
|
'Task name': name,
|
||
|
})),
|
||
|
);
|
||
|
|
||
|
// Output:
|
||
|
// ┌─────────┬───────────────────────┐
|
||
|
// │ (index) │ Task name │
|
||
|
// ├─────────┼───────────────────────┤
|
||
|
// │ 0 │ 'unimplemented bench' │
|
||
|
// └─────────┴───────────────────────┘
|
||
|
```
|
||
|
|
||
|
The `add` method accepts a task name and a task function, so it can benchmark
|
||
|
it! This method returns a reference to the Bench instance, so it's possible to
|
||
|
use it to create an another task for that instance.
|
||
|
|
||
|
Note that the task name should always be unique in an instance, because Tinybench stores the tasks based
|
||
|
on their names in a `Map`.
|
||
|
|
||
|
Also note that `tinybench` does not log any result by default. You can extract the relevant stats
|
||
|
from `bench.tasks` or any other API after running the benchmark, and process them however you want.
|
||
|
|
||
|
## Docs
|
||
|
|
||
|
### `Bench`
|
||
|
|
||
|
The Benchmark instance for keeping track of the benchmark tasks and controlling
|
||
|
them.
|
||
|
|
||
|
Options:
|
||
|
|
||
|
```ts
|
||
|
export type Options = {
|
||
|
/**
|
||
|
* time needed for running a benchmark task (milliseconds) @default 500
|
||
|
*/
|
||
|
time?: number;
|
||
|
|
||
|
/**
|
||
|
* number of times that a task should run if even the time option is finished @default 10
|
||
|
*/
|
||
|
iterations?: number;
|
||
|
|
||
|
/**
|
||
|
* function to get the current timestamp in milliseconds
|
||
|
*/
|
||
|
now?: () => number;
|
||
|
|
||
|
/**
|
||
|
* An AbortSignal for aborting the benchmark
|
||
|
*/
|
||
|
signal?: AbortSignal;
|
||
|
|
||
|
/**
|
||
|
* warmup time (milliseconds) @default 100ms
|
||
|
*/
|
||
|
warmupTime?: number;
|
||
|
|
||
|
/**
|
||
|
* warmup iterations @default 5
|
||
|
*/
|
||
|
warmupIterations?: number;
|
||
|
|
||
|
/**
|
||
|
* setup function to run before each benchmark task (cycle)
|
||
|
*/
|
||
|
setup?: Hook;
|
||
|
|
||
|
/**
|
||
|
* teardown function to run after each benchmark task (cycle)
|
||
|
*/
|
||
|
teardown?: Hook;
|
||
|
};
|
||
|
|
||
|
export type Hook = (task: Task, mode: "warmup" | "run") => void | Promise<void>;
|
||
|
```
|
||
|
|
||
|
- `async run()`: run the added tasks that were registered using the `add` method
|
||
|
- `async warmup()`: warm up the benchmark tasks
|
||
|
- `reset()`: reset each task and remove its result
|
||
|
- `add(name: string, fn: Fn, opts?: FnOpts)`: add a benchmark task to the task map
|
||
|
- `Fn`: `() => any | Promise<any>`
|
||
|
- `FnOpts`: `{}`: a set of optional functions run during the benchmark lifecycle that can be used to set up or tear down test data or fixtures without affecting the timing of each task
|
||
|
- `beforeAll?: () => any | Promise<any>`: invoked once before iterations of `fn` begin
|
||
|
- `beforeEach?: () => any | Promise<any>`: invoked before each time `fn` is executed
|
||
|
- `afterEach?: () => any | Promise<any>`: invoked after each time `fn` is executed
|
||
|
- `afterAll?: () => any | Promise<any>`: invoked once after all iterations of `fn` have finished
|
||
|
- `remove(name: string)`: remove a benchmark task from the task map
|
||
|
- `table()`: table of the tasks results
|
||
|
- `get results(): (TaskResult | undefined)[]`: (getter) tasks results as an array
|
||
|
- `get tasks(): Task[]`: (getter) tasks as an array
|
||
|
- `getTask(name: string): Task | undefined`: get a task based on the name
|
||
|
- `todo(name: string, fn?: Fn, opts: FnOptions)`: add a benchmark todo to the todo map
|
||
|
- `get todos(): Task[]`: (getter) tasks todos as an array
|
||
|
|
||
|
### `Task`
|
||
|
|
||
|
A class that represents each benchmark task in Tinybench. It keeps track of the
|
||
|
results, name, Bench instance, the task function and the number of times the task
|
||
|
function has been executed.
|
||
|
|
||
|
- `constructor(bench: Bench, name: string, fn: Fn, opts: FnOptions = {})`
|
||
|
- `bench: Bench`
|
||
|
- `name: string`: task name
|
||
|
- `fn: Fn`: the task function
|
||
|
- `opts: FnOptions`: Task options
|
||
|
- `runs: number`: the number of times the task function has been executed
|
||
|
- `result?: TaskResult`: the result object
|
||
|
- `async run()`: run the current task and write the results in `Task.result` object
|
||
|
- `async warmup()`: warm up the current task
|
||
|
- `setResult(result: Partial<TaskResult>)`: change the result object values
|
||
|
- `reset()`: reset the task to make the `Task.runs` a zero-value and remove the `Task.result` object
|
||
|
|
||
|
```ts
|
||
|
export interface FnOptions {
|
||
|
/**
|
||
|
* An optional function that is run before iterations of this task begin
|
||
|
*/
|
||
|
beforeAll?: (this: Task) => void | Promise<void>;
|
||
|
|
||
|
/**
|
||
|
* An optional function that is run before each iteration of this task
|
||
|
*/
|
||
|
beforeEach?: (this: Task) => void | Promise<void>;
|
||
|
|
||
|
/**
|
||
|
* An optional function that is run after each iteration of this task
|
||
|
*/
|
||
|
afterEach?: (this: Task) => void | Promise<void>;
|
||
|
|
||
|
/**
|
||
|
* An optional function that is run after all iterations of this task end
|
||
|
*/
|
||
|
afterAll?: (this: Task) => void | Promise<void>;
|
||
|
}
|
||
|
```
|
||
|
|
||
|
## `TaskResult`
|
||
|
|
||
|
the benchmark task result object.
|
||
|
|
||
|
```ts
|
||
|
export type TaskResult = {
|
||
|
|
||
|
/*
|
||
|
* the last error that was thrown while running the task
|
||
|
*/
|
||
|
error?: unknown;
|
||
|
|
||
|
/**
|
||
|
* The amount of time in milliseconds to run the benchmark task (cycle).
|
||
|
*/
|
||
|
totalTime: number;
|
||
|
|
||
|
/**
|
||
|
* the minimum value in the samples
|
||
|
*/
|
||
|
min: number;
|
||
|
/**
|
||
|
* the maximum value in the samples
|
||
|
*/
|
||
|
max: number;
|
||
|
|
||
|
/**
|
||
|
* the number of operations per second
|
||
|
*/
|
||
|
hz: number;
|
||
|
|
||
|
/**
|
||
|
* how long each operation takes (ms)
|
||
|
*/
|
||
|
period: number;
|
||
|
|
||
|
/**
|
||
|
* task samples of each task iteration time (ms)
|
||
|
*/
|
||
|
samples: number[];
|
||
|
|
||
|
/**
|
||
|
* samples mean/average (estimate of the population mean)
|
||
|
*/
|
||
|
mean: number;
|
||
|
|
||
|
/**
|
||
|
* samples variance (estimate of the population variance)
|
||
|
*/
|
||
|
variance: number;
|
||
|
|
||
|
/**
|
||
|
* samples standard deviation (estimate of the population standard deviation)
|
||
|
*/
|
||
|
sd: number;
|
||
|
|
||
|
/**
|
||
|
* standard error of the mean (a.k.a. the standard deviation of the sampling distribution of the sample mean)
|
||
|
*/
|
||
|
sem: number;
|
||
|
|
||
|
/**
|
||
|
* degrees of freedom
|
||
|
*/
|
||
|
df: number;
|
||
|
|
||
|
/**
|
||
|
* critical value of the samples
|
||
|
*/
|
||
|
critical: number;
|
||
|
|
||
|
/**
|
||
|
* margin of error
|
||
|
*/
|
||
|
moe: number;
|
||
|
|
||
|
/**
|
||
|
* relative margin of error
|
||
|
*/
|
||
|
rme: number;
|
||
|
|
||
|
/**
|
||
|
* p75 percentile
|
||
|
*/
|
||
|
p75: number;
|
||
|
|
||
|
/**
|
||
|
* p99 percentile
|
||
|
*/
|
||
|
p99: number;
|
||
|
|
||
|
/**
|
||
|
* p995 percentile
|
||
|
*/
|
||
|
p995: number;
|
||
|
|
||
|
/**
|
||
|
* p999 percentile
|
||
|
*/
|
||
|
p999: number;
|
||
|
};
|
||
|
```
|
||
|
|
||
|
### `Events`
|
||
|
|
||
|
Both the `Task` and `Bench` objects extend the `EventTarget` object, so you can attach listeners to different types of events
|
||
|
in each class instance using the universal `addEventListener` and
|
||
|
`removeEventListener`.
|
||
|
|
||
|
```ts
|
||
|
/**
|
||
|
* Bench events
|
||
|
*/
|
||
|
export type BenchEvents =
|
||
|
| "abort" // when a signal aborts
|
||
|
| "complete" // when running a benchmark finishes
|
||
|
| "error" // when the benchmark task throws
|
||
|
| "reset" // when the reset function gets called
|
||
|
| "start" // when running the benchmarks gets started
|
||
|
| "warmup" // when the benchmarks start getting warmed up (before start)
|
||
|
| "cycle" // when running each benchmark task gets done (cycle)
|
||
|
| "add" // when a Task gets added to the Bench
|
||
|
| "remove" // when a Task gets removed of the Bench
|
||
|
| "todo"; // when a todo Task gets added to the Bench
|
||
|
|
||
|
/**
|
||
|
* task events
|
||
|
*/
|
||
|
export type TaskEvents =
|
||
|
| "abort"
|
||
|
| "complete"
|
||
|
| "error"
|
||
|
| "reset"
|
||
|
| "start"
|
||
|
| "warmup"
|
||
|
| "cycle";
|
||
|
```
|
||
|
|
||
|
For instance:
|
||
|
|
||
|
```js
|
||
|
// runs on each benchmark task's cycle
|
||
|
bench.addEventListener("cycle", (e) => {
|
||
|
const task = e.task!;
|
||
|
});
|
||
|
|
||
|
// runs only on this benchmark task's cycle
|
||
|
task.addEventListener("cycle", (e) => {
|
||
|
const task = e.task!;
|
||
|
});
|
||
|
```
|
||
|
|
||
|
### `BenchEvent`
|
||
|
|
||
|
```ts
|
||
|
export type BenchEvent = Event & {
|
||
|
task: Task | null;
|
||
|
};
|
||
|
```
|
||
|
|
||
|
## Prior art
|
||
|
|
||
|
- [Benchmark.js](https://github.com/bestiejs/benchmark.js)
|
||
|
- [Mitata](https://github.com/evanwashere/mitata/)
|
||
|
- [Bema](https://github.com/prisma-labs/bema)
|
||
|
|
||
|
## Authors
|
||
|
|
||
|
| <a href="https://github.com/Aslemammad"> <img width='150' src="https://avatars.githubusercontent.com/u/37929992?v=4" /><br> Mohammad Bagher </a> |
|
||
|
| ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||
|
|
||
|
## Credits
|
||
|
|
||
|
| <a href="https://github.com/uzploak"> <img width='150' src="https://avatars.githubusercontent.com/u/5059100?v=4" /><br> Uzlopak </a> | <a href="https://github.com/poyoho"> <img width='150' src="https://avatars.githubusercontent.com/u/36070057?v=4" /><br> poyoho </a> |
|
||
|
| ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||
|
|
||
|
## Contributing
|
||
|
|
||
|
Feel free to create issues/discussions and then PRs for the project!
|
||
|
|
||
|
## Sponsors
|
||
|
|
||
|
Your sponsorship can make a huge difference in continuing our work in open source!
|
||
|
|
||
|
<p align="center">
|
||
|
<a href="https://cdn.jsdelivr.net/gh/aslemammad/static/sponsors.svg">
|
||
|
<img src='https://cdn.jsdelivr.net/gh/aslemammad/static/sponsors.svg'/>
|
||
|
</a>
|
||
|
</p>
|