Node.js-Clusterluck-以node.js编写的类似于erlang的分散式分布式系统的库

Clusterluck =========== [](https://travis-ci.org/azuqua/clusterluck) [](https://coveralls.io/github/azuqua/clusterluck?branch=master) [Documentation](https://azuqua.github.io/clusterluck) A library for writing distributed systems that use a gossip protocol to communicate state management, consistent hash rings for sharding, and vector clocks for history. ## Install ``` $ npm install clusterluck ``` ## Dependencies This module uses a native module called `node-microtime` for microsecond insert/update granularity for vector clocks, and by extension requires a C++-11 compatible compiler. The `.travis.yml` file lists g++-4.8 as an addon in response, but other compatible versions of g++ or clang should suffice. The following g++ compiler versions have been tested: - 6.*.* - 4.8 ## Test To run tests, you can use: ``` $ grunt test ``` or just: ``` $ mocha test ``` For code coverage information, you can use `istanbul` and run: ``` $ istanbul cover _mocha test ``` If `istanbul` isn't installed, just run: ``` $ npm install --global istanbul ``` ## Index - [Usage](#Usage) - [Creating Clusters](#CreatingClusters) - [Manipulating Clusters](#ManipulatingClusters) - [Example Cluster](#ExampleCluster) - [Writing GenServers](#WritingGenServers) - [Using the CLI](#UsingTheCLI) - [`inspect`](#inspect) - [`nodes`](#nodes) - [`ping`](#ping) - [`get`](#get) - [`has`](#has) - [`weight`](#weight) - [`weights`](#weights) - [`join`](#join) - [`meet`](#meet) - [`leave`](#leave) - [`insert`](#insert) - [`minsert`](#minsert) - [`update`](#update) - [`remove`](#remove) - [`mremove`](#mremove) - [Consistent Hash Ring](#ConsistentHashRing) - [Vector Clocks](#VectorClocks) - [TODO](#TODO) ### <a name="Usage" rel='nofollow' onclick='return false;'></a>Usage Clusterluck can be used as a module to write decentralized distributed systems with full-mesh IPC network topologies, or in isolation to use the consistent hash ring and vector clock data structures. #### <a name="CreatingClusters" rel='nofollow' onclick='return false;'></a>Creatings Clusters To get started, we can use the following code: ```javascript const cl = require("clusterluck"); let node = cl.createCluster("foo", "localhost", 7022); node.start("cookie", "ring", () => { console.log("Listening on port 7022!"); }); ``` This will create a single node in cluster `ring` with name `foo`, using cookie `cookie` to sign messages within the cluster. If another program tries to communicate with this node, but doesn't sign requests with this cookie, the message will be ignored with an `INVALID_CHECKSUM` error emitted in the debug logs. When nodes are added to `ring`, each node will attempt to form TCP-based IPC connections to any new node added into the cluster. Similarly, each IPC server will generate new IPC-client connections. Each external connection will queue up messages if the socket goes down, sending all queued up messages once the socket reconnects. Once nodes are removed, both ends of the connection are closed forcibly. In the background, several listeners attached to the network kernel for this node will be created, including a gossip ring listener and a command line server listener. In short, the command line server listener exists to handle requests made by the CLI tool under `bin/cli.js`, while the gossip ring listens for ring maniuplations on the cluster. On additions to the cluster, new IPC connections to external nodes will be created, and vice versa for removals from the cluster. #### <a name="ManipulatingClusters" rel='nofollow' onclick='return false;'></a>Manipulating Clusters To manipulate the cluster, we interact with the gossip property of the cluster node. For example: ``` javascript // meeting another node let gossip = node.gossip(); let kernel = node.kernel(); let nNode = new Node("bar", "localhost", 7023); gossip.meet(nNode); // wait some time, eventually the node will show up in this node's ring... assert.ok(gossip.ring().has(nNode)); // inserting nodes gossip.insert(nNode); assert.ok(gossip.ring().has(nNode)); // after some time, node "bar" should have "foo" in its ring... // removing nodes gossip.remove(nNode); assert.notOk(gossip.ring().has(nNode)); // after some time, ndoe "bar" should remove "foo" from its ring... // leaving a cluster gossip.leave(); gossip.once("leave", () => { assert.lengthOf(gossip.ring().nodes(), 1); assert.ok(gossip.ring.nodes()[0].equals(kernel.self())); }); // joining a cluster gossip.join("another_ring_id"); // after some time, this node will receive messages from the existing nodes in the cluster // (if any exist) ``` For documentation on available methods/inputs for cluster manipulation, visit the documentation for the `GossipRing` class. #### <a name="ExampleCluster" rel='nofollow' onclick='return false;'></a>Example Cluster In an example.js file, insert the following: ```javascript const cl = require("clusterluck"), os = require("os"); let id = process.argv[2], port = parseInt(process.argv[3]); let node = cl.createCluster(id, os.hostname(), port); node.start("cookie", "ring", () => { console.log("Listening on port %s!", port); }); ``` Then, in one terminal, run: ``` $ node example.js foo 7022 ``` And in another terminal, run: ``` $ node example.js bar 7023 ``` Now, if we spin up the CLI and connect to `foo`, we can then run: ``` // whatever os.hostname() resolves to, replace localhost with that $ meet bar localhost 7023 ``` If we then go to inspect the ring on each node, we should see both node `foo` and node `bar` in the ring. #### <a name="WritingGenServers" rel='nofollow' onclick='return false;'></a>Writing GenServers The `GenServer` class is used to create actors that send messages around and receive messages from the rest of the cluster. They're the basic unit of logic handling in clusterluck, and heavily derived off of Erlang's gen_server's, but incorporated into node.js' EventEmitter model. To start a `GenServer` with no event handling, we can use the following code: ```javascript let serve = cl.createGenServer(cluster); serve.start("name_to_listen_for"); ``` This will tell the network kernel `kernel` that any messages with id `name_to_listen_for` received on this node should be routed to `serve` for processing. Names for `GenServer`s on the same node have a uniqueness property, so trying to declare multiple instances listening on the same name will raise an error. To add some event handling to our `GenServer`, we can modify the above code as such: ```javascript let serve = cl.createGenServer(cluster); serve.on("hello", (data, from) => { serve.reply(from, "world"); }); serve.start("name_to_listen_for"); ``` With this additional logic, any "hello" event sent to this node with id `name_to_listen_for` will be responded to with "world". This includes messages sent from the local node, as well from other nodes in the cluster. Once we've declared our `GenServer` and added event handling logic, we can start sending and receiving messages to/from other nodes in the cluster. ```javascript // serve is a GenServer instance, kernel is serve's network kernel // synchronous requests // this makes a call to a GenServer listening on "server_name" locally serve.call("server_name", "event", "data", (err, out) => {...}); // this makes the same call serve.call({id: "server_name", node: kernel.self()}, "event", "data", (err, out) => {...}); // this makes the same call but to another node serve.call({id: "server_name", node: another_node}, "event", "data", (err, out) => {...}); // asynchronous requests // this makes an async call to a GenServer listening on "server_name" locally serve.cast("server_name", "event", "data"); // this makes the same call serve.cast({id: "server_name", node: kernel.self()}, "event", "data"); // this makes the same call but to another node serve.cast({id: "server_name", node: another_node}, "event", "data"); ``` Here, we see the true power of