taskeduler

所属分类:其他
开发工具:Erlang
文件大小:0KB
下载次数:0
上传日期:2017-08-14 16:50:44
上 传 者sh-1993
说明:  Erlang OTP 19,rebar3,带Rest API的分布式任务调度器,大数据处理器,
(Erlang OTP 19, rebar3, distributed task scheduler with Rest API, Big data processor,)

文件列表:
LICENSE (35141, 2017-08-14)
Makefile (1529, 2017-08-14)
apidoc.json (199, 2017-08-14)
build.spec (602, 2017-08-14)
config/ (0, 2017-08-14)
config/sys.config.tmpl (523, 2017-08-14)
config/vm.args (812, 2017-08-14)
cover.spec (43, 2017-08-14)
docs/ (0, 2017-08-14)
docs/controller.html (2410, 2017-08-14)
docs/edoc-info (271, 2017-08-14)
docs/erlang.png (2109, 2017-08-14)
docs/index.html (489, 2017-08-14)
docs/modules-frame.html (1700, 2017-08-14)
docs/overview-summary.html (1101, 2017-08-14)
docs/stylesheet.css (869, 2017-08-14)
docs/task.html (2093, 2017-08-14)
docs/taskeduler.html (3795, 2017-08-14)
docs/taskeduler_api.html (9167, 2017-08-14)
docs/taskeduler_app.html (2358, 2017-08-14)
docs/taskeduler_cache.html (6078, 2017-08-14)
docs/taskeduler_service.html (3112, 2017-08-14)
docs/taskeduler_srv.html (6031, 2017-08-14)
docs/taskeduler_stats.html (5286, 2017-08-14)
docs/taskeduler_sup.html (2151, 2017-08-14)
docs/taskeduler_sync.html (9775, 2017-08-14)
docs/taskeduler_v1_http.html (3348, 2017-08-14)
docs/test_controller.html (2956, 2017-08-14)
eunit.config (90, 2017-08-14)
include/ (0, 2017-08-14)
include/taskeduler.hrl (502, 2017-08-14)
include/taskeduler_test.hrl (538, 2017-08-14)
rebar.config (992, 2017-08-14)
rebar3 (785001, 2017-08-14)
src/ (0, 2017-08-14)
src/behaviors/ (0, 2017-08-14)
src/behaviors/controller.erl (667, 2017-08-14)
src/behaviors/task.erl (379, 2017-08-14)
... ...

[Build Status / link to build job](https://github.com/elpaisa/taskeduler/blob/master/#) # taskeduler Highly Distributed Application for executing tasks in a concurrent and parallel manner ## Overview This distributed service allows to execute scheduled tasks using a task controller list, provided in the config. Using this module as a dependency of any project enables the main application to define workers and do whatever it needs to accomplish a set of tasks, this is suitable for services that need to sync or process periodically any amount of data, the best example would be obtaining a list of customers and process them using an interval of time. All modules for tasks must implement -behavior(task)., the controller module is used to obtain the list of tasks to execute. If used as a main application, see example functions test_controller.erl for testing, these are the mandatory implementations by the consumer application. If your main application requires additional endpoints, there is no need to define a new service, all you have to do is to specify the module that will contain the functions exposed as endpoints in the config as `{rest_handler, your_module}`, all functions exposed with arity of 2 will be public and searchable by the service in that module, so function_name(UrlParam1, Body), will be used if it is requested, if no request Body is received (usually `GET`), in the request and no url param, these will come as `undefined` atoms, the endpoint in this case will be `your_api/v1/function_name/param1` or `your_api/v1/function_name`, if the function needs to be async can be called as `your_api/v1/async/function_name/param1` or `your_api/v1/async/function_name`. Please see project definition [documentation][api-documentation] ## Ownership taskeduler is owned by the [elpaisa][email]. Feel free to [email me][email]." ## Dependencies This service depends on Cowboy for defining the http API ## How to consume taskeduler Endpoint | Parameters | Description ----- | ----------- | -------- start | | Starts all workers start/1 | "worker-name", {test_daily, test_regular} | Starts a specific worker stop | | Stops all workers stop/1 | "worker-name", {test_daily, test_regular} | Stops a specific worker status | | Responds the status of all workers status/1 | "worker-name", {test_daily, test_regular} | Responds status for a specific worker re-sync | | Starts all workers and runs an loop for the given interval of days defined in sys.config re-sync/1 | "worker-name", {test_daily, test_regular} | Runs re-sync for a specific worker ## How to contribute Always submit issues for discussion before creating any pull requests. ## How to report defects Open github issues ## Running locally For running locally please use. ```sh git clone git@github.com:elpaisa/taskeduler.git.git cd taskeduler make shell ``` taskeduler requires configuration parameters. Copy the `sys.config.tmpl` as `config/sys.config` and edit it accordingly: ```sh cp sys.config.tmpl config/sys.config vim sys.config ``` ```erlang %%-*- mode: erlang -*- [ {taskeduler, [ {max_workers, 12}, {max_retries, 12}, {controller_module, test_controller}, {rest_handler, test_controller}, {service_name, "/taskeduler/v1"}, {nodes, ['nonode@nohost']}, {workers, [ {test, [ {re_sync_days, 7}, {max_fetch, 2000}], [ {test_regular, 60, minutes}, {test_daily, 1, days} ] %% Worker definitions } ]} ]}, {sasl, [ {sasl_error_logger, false} ]} ]. % vim:ft=erlang ``` Once you have filled out the above changes to the `sys.config`, you can start taskeduler: ```sh make shell ``` Or ``` ./rebar3 shell ``` ### Testing To run the tests, execute: ```sh make test ``` For unit tests and for common (integration) tests, if they are different. Note that the `make test` target should generally execute whatever testing can be run by continuous integration: thus, if the common tests require some specialized setup/environment that CI is not configured for, this target should not include `ct`. #### Unit Tests To run the unit tests alone, execute: ```sh make eunit ``` OR ``` ./rebar3 eunit ``` If there is something unique about the eunit configuration, or the output of the tests or code coverage are sent somewhere different than conventional, describe here how to view those reports. #### Common tests To run the common tests alone, execute: ```sh make ct ``` The common tests for taskeduler are integration tests that require all dependencies above to be available. The node started for the common test uses the erl.config file in the project's root directory. ### HTTP API documentation In addition to viewing the [API documentation][api-documentation], you can also build it locally for testing updates to documentation prior to deploying it. To build documentation locally, you must install [apidoc][apidoc] (see instructions [here][install-apidoc]). Once you have it installed, you can build the documentation: ```sh make doc ``` To view the documentation after it is built, open the file `doc/api/index.html`: ```sh % Open in Mac OS X with default browser open docs/index.html ``` ### Doing a release This command creates a tarball in `_build/default/rel/taskeduler/`, the full application is self contained in there. ```sh make release ``` ### From Jenkins To create an rpm for installation, execute the below command, that will do test, create a self contained release and build an rpm ready for deployment, later on `./_build/default/rel/taskeduler/bin/taskeduler start` will create a service, add this to chkconfig in the server in order to do it a real centos service. ```sh make rpm ``` [design-doc]: # [email]: mailto:clientes@desarrollowebmedellin.com [api-documentation]: https://elpaisa.github.io/taskeduler/ [apidoc]: http://apidocjs.com

近期下载者

相关文件


收藏者