• A6_433434
  • 135.9KB
  • zip
  • 0
  • VIP专享
  • 0
  • 2022-04-12 18:31
Clusterluck - 以node.js编写的类似于erlang的分散式分布式系统的库
Clusterluck =========== [![Build Status](]( [![Code Coverage](]( [Documentation]( A library for writing distributed systems that use a gossip protocol to communicate state management, consistent hash rings for sharding, and vector clocks for history. ## Install ``` $ npm install clusterluck ``` ## Dependencies This module uses a native module called `node-microtime` for microsecond insert/update granularity for vector clocks, and by extension requires a C++-11 compatible compiler. The `.travis.yml` file lists g++-4.8 as an addon in response, but other compatible versions of g++ or clang should suffice. The following g++ compiler versions have been tested: - 6.*.* - 4.8 ## Test To run tests, you can use: ``` $ grunt test ``` or just: ``` $ mocha test ``` For code coverage information, you can use `istanbul` and run: ``` $ istanbul cover _mocha test ``` If `istanbul` isn't installed, just run: ``` $ npm install --global istanbul ``` ## Index - [Usage](#Usage) - [Creating Clusters](#CreatingClusters) - [Manipulating Clusters](#ManipulatingClusters) - [Example Cluster](#ExampleCluster) - [Writing GenServers](#WritingGenServers) - [Using the CLI](#UsingTheCLI) - [`inspect`](#inspect) - [`nodes`](#nodes) - [`ping`](#ping) - [`get`](#get) - [`has`](#has) - [`weight`](#weight) - [`weights`](#weights) - [`join`](#join) - [`meet`](#meet) - [`leave`](#leave) - [`insert`](#insert) - [`minsert`](#minsert) - [`update`](#update) - [`remove`](#remove) - [`mremove`](#mremove) - [Consistent Hash Ring](#ConsistentHashRing) - [Vector Clocks](#VectorClocks) - [TODO](#TODO) ### <a name="Usage" rel='nofollow' onclick='return false;'></a>Usage Clusterluck can be used as a module to write decentralized distributed systems with full-mesh IPC network topologies, or in isolation to use the consistent hash ring and vector clock data structures. #### <a name="CreatingClusters" rel='nofollow' onclick='return false;'></a>Creatings Clusters To get started, we can use the following code: ```javascript const cl = require("clusterluck"); let node = cl.createCluster("foo", "localhost", 7022); node.start("cookie", "ring", () => { console.log("Listening on port 7022!"); }); ``` This will create a single node in cluster `ring` with name `foo`, using cookie `cookie` to sign messages within the cluster. If another program tries to communicate with this node, but doesn't sign requests with this cookie, the message will be ignored with an `INVALID_CHECKSUM` error emitted in the debug logs. When nodes are added to `ring`, each node will attempt to form TCP-based IPC connections to any new node added into the cluster. Similarly, each IPC server will generate new IPC-client connections. Each external connection will queue up messages if the socket goes down, sending all queued up messages once the socket reconnects. Once nodes are removed, both ends of the connection are closed forcibly. In the background, several listeners attached to the network kernel for this node will be created, including a gossip ring listener and a command line server listener. In short, the command line server listener exists to handle requests made by the CLI tool under `bin/cli.js`, while the gossip ring listens for ring maniuplations on the cluster. On additions to the cluster, new IPC connections to external nodes will be created, and vice versa for removals from the cluster. #### <a name="ManipulatingClusters" rel='nofollow' onclick='return false;'></a>Manipulating Clusters To manipulate the cluster, we interact with the gossip property of the cluster node. For example: ``` javascript // meeting another node let gossip = node.gossip(); let kernel = node.kernel(); let nNode = new Node("bar", "localhost", 7023);; // wait some time, eventually the node will show up in this node's ring... assert.ok(gossip.ring().has(nNode)); // inserting nodes gossip.insert(nNode); assert.ok(gossip.ring().has(nNode)); // after some time, node "bar" should have "foo" in its ring... // removing nodes gossip.remove(nNode); assert.notOk(gossip.ring().has(nNode)); // after some time, ndoe "bar" should remove "foo" from its ring... // leaving a cluster gossip.leave(); gossip.once("leave", () => { assert.lengthOf(gossip.ring().nodes(), 1); assert.ok(gossip.ring.nodes()[0].equals(kernel.self())); }); // joining a cluster gossip.join("another_ring_id"); // after some time, this node will receive messages from the existing nodes in the cluster // (if any exist) ``` For documentation on available methods/inputs for cluster manipulation, visit the documentation for the `GossipRing` class. #### <a name="ExampleCluster" rel='nofollow' onclick='return false;'></a>Example Cluster In an example.js file, insert the following: ```javascript const cl = require("clusterluck"), os = require("os"); let id = process.argv[2], port = parseInt(process.argv[3]); let node = cl.createCluster(id, os.hostname(), port); node.start("cookie", "ring", () => { console.log("Listening on port %s!", port); }); ``` Then, in one terminal, run: ``` $ node example.js foo 7022 ``` And in another terminal, run: ``` $ node example.js bar 7023 ``` Now, if we spin up the CLI and connect to `foo`, we can then run: ``` // whatever os.hostname() resolves to, replace localhost with that $ meet bar localhost 7023 ``` If we then go to inspect the ring on each node, we should see both node `foo` and node `bar` in the ring. #### <a name="WritingGenServers" rel='nofollow' onclick='return false;'></a>Writing GenServers The `GenServer` class is used to create actors that send messages around and receive messages from the rest of the cluster. They're the basic unit of logic handling in clusterluck, and heavily derived off of Erlang's gen_server's, but incorporated into node.js' EventEmitter model. To start a `GenServer` with no event handling, we can use the following code: ```javascript let serve = cl.createGenServer(cluster); serve.start("name_to_listen_for"); ``` This will tell the network kernel `kernel` that any messages with id `name_to_listen_for` received on this node should be routed to `serve` for processing. Names for `GenServer`s on the same node have a uniqueness property, so trying to declare multiple instances listening on the same name will raise an error. To add some event handling to our `GenServer`, we can modify the above code as such: ```javascript let serve = cl.createGenServer(cluster); serve.on("hello", (data, from) => { serve.reply(from, "world"); }); serve.start("name_to_listen_for"); ``` With this additional logic, any "hello" event sent to this node with id `name_to_listen_for` will be responded to with "world". This includes messages sent from the local node, as well from other nodes in the cluster. Once we've declared our `GenServer` and added event handling logic, we can start sending and receiving messages to/from other nodes in the cluster. ```javascript // serve is a GenServer instance, kernel is serve's network kernel // synchronous requests // this makes a call to a GenServer listening on "server_name" locally"server_name", "event", "data", (err, out) => {...}); // this makes the same call{id: "server_name", node: kernel.self()}, "event", "data", (err, out) => {...}); // this makes the same call but to another node{id: "server_name", node: another_node}, "event", "data", (err, out) => {...}); // asynchronous requests // this makes an async call to a GenServer listening on "server_name" locally serve.cast("server_name", "event", "data"); // this makes the same call serve.cast({id: "server_name", node: kernel.self()}, "event", "data"); // this makes the same call but to another node serve.cast({id: "server_name", node: another_node}, "event", "data"); ``` Here, we see the true power of
    • Erlang生态的分布式
      py 是为 Erlang 生态系统准备的分布式 Python。该项目有两个主要特性: Python 的简单封装接口,封装了 ErlPort 调用,可方便的进行: Make module-level calls Get module-level constants Instantiate ...
    • awesome-distributed-systems:精选列表,以了解分布式系统
      构建,[免费]乔·阿姆斯特朗(Erlang的作者)博士学位论文 [Amazon Link] [Amazon Link] [paywall] [免费注册] [Amazon Link] 文件 必须阅读有关分布式系统的论文。 虽然Lamport的几乎所有工作都应放在此处,但...
    • Erlang安装包
      Erlang是一种通用的面向并发的编程语言。 其特点有:并发性、分布式、健壮性、软实时性、热代码升级、递增式代码装载、外部接口、速错等等等等
    • opencensus-erlang:统计收集和分布式跟踪框架
    • erlang programming
      Programming+Erlang.pdf+ 面对软件错误构建可靠的分布式系统.pdf+ Concurrent Programming in ERLANG
    • erlang资源
    • EdpBench:Erlang 分布式协议 (EDP) 基准
      Erlang分布式协议基准 这是一个基准生成器,用于测量 Erlang 和 Java 节点之间的消息和字节传输速率。 它使用网络上的和包来创建 Java 节点,该节点知道 Erlang 端口映射器( )、数据类型及其分布式协议。 基准 ...
    • epmdpxy:使用 Erlang Port Mapper Deamon 模拟 Netsplits
      EPMDPXY - Erlang 端口映射器守护进程的代理 EPMDPXY 模拟了 EPMD 的基本功能,足以让本地 ... 分布式 Erlang 节点连接到 EPMD 并将其节点名称与随机侦听器端口一起注册,该端口用于接受来自集群中其他节点的连接 ( A
    • Erlang-PARADIS课程
      Erlang-PARADIS课程 SU DSV PARADIS(并行和分布式编程)课程中用于Erlang分配的代码。
    • erlang_面对软件错误构建可靠的分布式系统