文件列表:

CMakeLists.txt (691, 2023-11-06)
Doxyfile (112776, 2023-11-06)
HdrHistogram_c/ (0, 2023-11-06)
LICENSE (11380, 2023-11-06)
RDMA/ (0, 2023-11-06)
install-deps.sh (530, 2023-11-06)
src/ (0, 2023-11-06)
src/CMakeLists.txt (1401, 2023-11-06)
src/client.cpp (1090, 2023-11-06)
src/client_pool.cc (2691, 2023-11-06)
src/client_pool.h (1465, 2023-11-06)
src/csl.cc (18068, 2023-11-06)
src/csl.h (147, 2023-11-06)
src/csl_config.h (443, 2023-11-06)
src/ncl_stat.cc (8775, 2023-11-06)
src/ncl_stat.h (1185, 2023-11-06)
src/posix_client.cpp (3215, 2023-11-06)
src/properties.cc (3422, 2023-11-06)
src/properties.h (2868, 2023-11-06)
src/rdma/ (0, 2023-11-06)
src/rdma/client.cc (25666, 2023-11-06)
src/rdma/client.h (9903, 2023-11-06)
src/rdma/common.h (584, 2023-11-06)
src/rdma/mr_pool.cc (1171, 2023-11-06)
src/rdma/mr_pool.h (1245, 2023-11-06)
src/rdma/qp_pool.cc (1210, 2023-11-06)
src/rdma/qp_pool.h (1341, 2023-11-06)
src/rdma/server.cc (11736, 2023-11-06)
src/rdma/server.h (2712, 2023-11-06)
src/server.cpp (1036, 2023-11-06)
src/syscall_type.h (1292, 2023-11-06)
src/util.h (2575, 2023-11-06)
tst/ (0, 2023-11-06)
tst/CMakeLists.txt (293, 2023-11-06)
... ...

# Near-Compute-Log (compute-side-log) [](https://opensource.org/licenses/Apache-2.0) ## Publication This repo is the artifact of: > [Xuhao Luo, Ramnatthan Alagappan, and Aishwarya Ganesan. **SplitFT: Fault Tolerance for Disaggregated Datacenters via Remote Memory Logging.** In *European Conference on Computer Systems (EuroSys ’24), April 22–25, 2024, Athens, Greece.*](https://doi.org/10.1145/3627703.3629561) ## Organization This repo is organized as follow: ``` Near-Compute-Log/ ├─ RDMA/ (a C++ RDMA library) ├─ src/ (source code for NCL) │ ├─ rdma/ (rdma related code) │ ├─ csl.cc (top level code for NCL library) │ ├─ csl_config.h (configurations) │ ├─ server.cpp (NCL replication peer code) │ ├─ client.cpp (an example client that uses NCL internal api) │ ├─ posix_client.cpp (an example client that uses POSIX api) │ ├─ ... ├─ tst/ (some test code) ``` ## Download ```bash git clone https://github.com/dassl-uiuc/compute-side-log.git --recurse-submodules cd compute-side-log ``` ## Install Dep ### Install submodules and dependencies ```bash git submodule init git submodule update ./install-deps.sh sudo ldconfig ``` ### Install RDMA Driver For Mellanox NIC, use `RDMA/install.sh` to install RDMA driver. You may change the exact version of MLNX_OFED to match your own operating system. See https://network.nvidia.com/products/infiniband-drivers/linux/mlnx_ofed/ for detail. For other RDMA NIC, please refer to vendor for installation instructions. ### Install zookeeper server Zookeeper can be installed on any server that is accessible to all client and servers. You may install it on the client machine for convenience. ```bash # install jvm if you haven't sudo apt install -y openjdk-11-jre-headless # Download zookeeper binary cd .. wget https://archive.apache.org/dist/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3-bin.tar.gz tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz cd apache-zookeeper-3.6.3-bin ``` Edit zookeeper config ```bash cd conf cp zoo_sample.cfg zoo.cfg # Edit line 12 of zoo.cfg to # dataDir=${PATH_TO_ZOOKEEPER_FOLDER}/zookeeper cd .. ``` Start zookeeper ```bash ./bin/zkServer.sh start cd ../compute-side-log ``` ## Configure ### Config RDMA library Edit `RDMA/src/infinity/core/Configuration.h` ```c++ // line 47: edit this to be the device name of your RNIC (can be obtained from `ifconfig`) static constexpr const char* DEFAULT_NET_DEVICE = "enp0s31f6"; // line 49: edit this to be the hca_id of your IB device (can be obtained from `ibv_devinfo`) static constexpr const char* DEFAULT_IB_DEVICE = "rxe_0" // line 51: edit this to be the physical port number of IB device (can be obtained from `ibv_devinfo`) static constexpr const uint16_t DEFAULT_IB_PHY_PORT = 1; ``` ### Config NCL library Edit `src/csl-config.h` ```c++ // line 7: edit this to be the ip address of the server running zookeeper (the previous step) const std::string ZK_DEFAULT_HOST = "127.0.0.1:2181"; // line 10: edit this to be the number of replicas const int DEFAULT_REP_FACTOR = 1; // line 11: edit this to be the size (in bytes) of memory region to be registered for each file on each replica const size_t MR_SIZE = 1024 * 1024 * 100; ``` ## Build ```bash cd compute-side-log sudo cp src/csl.h /usr/include # install NCL header file cmake -S . -B build -DCMAKE_BUILD_TYPE=Release cmake --build build ``` The binaries will be in `./build/src/`, which contains: - `libcsl.so`: The NCL library - `server`: The NCL replication peer - `posix_client`: The example POSIX client ## Run test First, download, install deps, configure, and build NCL on all client and servers. You may also only build it on one machine, and mount the src code folder to all other machines via NFS. But zookeeper client library and gflags need to be installed on all machines ```bash # after mounting cd ${NCL_DIR}/zookeeper/zookeeper-client/zookeeper-client-c && sudo make install sudo apt install -y libgflags-dev sudo ldconfig ``` First start server process on every server machine, then run client process. ```bash # start server process on every server machine. The number of server machines needed is specified in DEFAULT_REP_FACTOR in csl_config.h ./build/src/server # start client ./build/src/posix_client 128 w ./test.txt ncl ``` What the posix client does is to keep writing a fixed value (42 here) until the memory region is filled up. The server process prints the first 128 bytes of each memory region every second. You should see it prints 128 '*' (ASCII code 42) in this test. Press Ctrl+C to exit the server process. ## General Usage To make a file backed by NCL, just add the NCL flag `O_CSL` when creating the file. ```c #include int fd = open("test.txt", O_RDWR | O_CREAT | O_CSL, 0644); ``` The file should not have content in it. Currently NCL does not support backing a file that has existed content. Developer should ensure that the file size will not exceed the configured `MR_SIZE`. Typically, write-ahead-log files are small and have a configurable size limit. Then preload the NCL library when running the process (assume NCL servers are already running on replication peers). ```bash LD_PRELOAD=${PATH_TO_LIB}/libcsl.so ./app ``` ## Support Platform Near-Compute-Log is tested on the following platform: - OS: Ubuntu 20.04 LTS - NIC: Mellanox MT27710 Family ConnectX-4 Lx (25Gb RoCE) - RDMA Driver: MLNX_OFED-5.4-3.6.8.1

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：