api-bench-runner:用于API基准测试的CLI Runner，其工作原理类似于mocha

# api-bench-runner This is a CLI tool which can be used to run [api-benchmark](https://www.npmjs.com/package/api-benchmark) tests. Its interface is inspired by [mocha](https://www.npmjs.com/package/mocha). ## Usage The idea behind this tool is to allow you to define tests in one or across multiple scripts using _easy_ to use hooks much like `mocha`. ### Example of a Test file ```js 'use strict'; suite('Status Route', () => { service('my-service', 'http://localhost:8080'); suite('Multiple requests in Parallel', () => { options({ runMode: 'parallel', minSamples: 200, maxTime: 20, }); route('status', { method: 'get', route: 'status', expectedStatusCode: 200, maxMean: 0.2, // 200ms }); }); suite('Multiple requests in Sequence', () => { options({ runMode: 'sequence', minSamples: 200, maxTime: 20, }); route('status', { method: 'get', route: 'status', expectedStatusCode: 200, maxMean: 0.2, // 200ms }); }); }); ``` In the above example we use the `suite` hook to define a test suite. This suite will result in a single `api-benchmark` test run. The `suite` hook takes a name (just for the output) and a callback function that will be executed to configure the suite. The callback will include the other hooks which lets you define the service, routes, and options. You can have multiple test files which `api-bench-runner` will aggregate together allowing you to keep you test code clear and module. By convention `api-bench-runner` will look for tests under the `bench` folder. ``` ├── bench │   ├── nested-test-example.js │   └── simple-test-example.js └── lib └── server.js ``` ### Example of Command ``` > node_modules/.bin/api-bench-runner . ``` This command will look for all the tests under the `bench` folder and run them displaying the results in the terminal. You can redirect `api-bench-runner` to a different location if you choose not to put your tests under a `bench` folder. ``` > node_modules/.bin/api-bench-runner ./benchmark ``` You can use _glob_ patterns to find test files as well ``` > node_modules/.bin/api-bench-runner ./test/**/*.bench.js ``` ## Output The output of the tool will be a print out to the terminal showing the test results. When all tests pass (or you did not define restrictions which would cause failures) you will see each test (grouped by test suite) listing info such as operations per second and a summary saying everything passed.  If however you do have restrictions and the route did not meet them or something else went wrong you will see that test run listed with an error and a summary of all test failures printed out at the bottom.  ## Reporters You can specify different reporters for `api-bench-runner` to use depending on what kind of output you need. To pick a reporter use the `--reporter` flag. ### Terminal By default it will [print to the terminal](#output) but you can make this explicit by setting the `--reporter` flag to `stdterm` or `default` ``` > node_modules/.bin/api-bench-runner ./test/**/*.bench.js --reporter default ``` ### HTML You can have `api-bench-runner` generate an HTML report. The report generated is the report generated by [api-benchmark](https://www.npmjs.com/package/api-benchmark#gethtmlresults-callback). To pick the HTML reporter set the `--reporter` flag to `html`. ``` > node_modules/.bin/api-bench-runner ./test/**/*.bench.js --reporter html ``` By default the HTML report will be generated in the working directory that `api-bench-runner` is in and will save it to the file `benchmarks.js`. To specify a different file name you can use the `--out` flag. ``` > node_modules/.bin/api-bench-runner ./test/**/*.bench.js --reporter html --out my-results.html ``` *Note* The `--out` flag has no effect on `stdterm` reporter and right now is only used by the `html` reporter. ### Multiple Reporters If you would like to see the results in the terminal _and_ get a HTML report generated you can specify both reporters via the `--reporter` flag. ``` > node_modules/.bin/api-bench-runner ./test/**/*.bench.js --reporter stdterm,html ``` ## Hooks ### Define a Suite You can define a test suite via the `suite` hook. A _suite_ equates to a single `api-benchmark` test run. The `suite` hook takes two parameters: a title (just for the output) and a callback function for configuring the suite. The callback will include all the other hooks which lets you configure this suite's services, routes, and options. ```js suite('My suite that tests the status route', () => { // Configure the suite here }); ``` You can nest suites inside each other for cases where you have shared configurations. ```js suite('My suite that tests my-service', () => { service('my-service', 'http://localhost:8080'); // All child suites use this same service. suite('My suite that tests the status route', () => { options(...); route(...); }); suite('My suite that tests the version route', () => { options(...); route(...); }); }); ``` Both the version and status suites since they don't define a service of their own will check there parent to find out what service to run the tests against. Nesting suites allows for sharable configurations. ## Defining the Service The tests defined in the suite need to run against a running HTTP server, you can tell the suite where to find the server via the `service` hook. ```js suite('My suite that tests my-service', () => { service('my-service', 'http://localhost:8080'); options(...); route(...); }); ``` The `service` hook takes two parameters: a name (must be a valid ECMAScript object property name) and the URL to the service to run the tests against. The URL is to be the base URL which we can extend with the route paths. If your hitting a development lap or production environment or a local environment you've manually started up then you can just put in the URL as a string as seen above; however, if the site is dynamic and needs to be brought up when the tests starts and thus might have a dynamic URL (random port?) you can pass in a callback function which will be called by the suite when it starts executing. ```js function findService() {...} suite('My suite that tests my-service', () => { service('my-service', findService); options(...); route(...); }); ``` The function you pass in must return a string consisting of the service URL. ## Define Suite Options Each suite can be configured with options defining how the test should be run. This includes how many samples, if the tests should be parallel or sequential, and the maximum number of concurrent requests if tests are to be done in parallel. You can define these options via the `options` hook. This hook is optional however as each option has a default value. ```js suite('Multiple requests in Sequence', () => { options({ runMode: 'sequence', minSamples: 200, maxTime: 20, }); route(...); }); ``` These options can also be nested in a parent suite if they are common. ```js suite('My suite that tests my-service', () => { // All child suites use this same service and test options. service('my-service', 'http://localhost:8080'); options({ runMode: 'sequence', minSamples: 200, maxTime: 20, }); suite('My suite that tests the status route', () => { route(...); }); suite('My suite that tests the version route', () => { route(...); }); }); ``` These options are the `api-benchmark` options, for a list of valid options see [the `api-benchmark` documentation](https://www.npmjs.com/package/api-benchmark#options-object) ## Defining a Route To define a given test for the suite you need to define the route to test. This can be