上传   管理   搜索   留言
Pudn.com > 下载中心  > 其他  > A1_Distributed_Systems
A1_Distributed_Systems
去下载(0) 赞(1) 踩(0) 评论(0) 收藏(0)
所属分类：其他
开发工具：Python
文件大小：0KB
下载次数：0
上传日期：2024-01-24 20:20:10
上 传 者：sh-1993
说明：  A1分布式系统
(A1 Distributed Systems)
文件列表:
analysis/
images/
load_balancer/
server/
DS_assign1_LB_2024.pdf
Makefile
client.sh
docker-compose.yml
# Assignment-1 Distributed Systems This repository contains the code for the Assignment-1 of Distributed Systems(CS60002) course of Spring, 2024. # Implementing a Customizable Load Balancer


Fig.1: Overview

# Table of Contents - [Prerequisites](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#prerequisites) - [Getting Started](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#getting-started) - [Building Docker Images](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#building-docker-images) - [Running Docker Containers](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#running-docker-containers) - [Interact With System](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#interact-with-system) - [Remove Existing Container](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#interact-with-system) - [Clear Existing Images](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#clear-existing-images) - [Design Choices](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#design-choices) - [Troubleshooting](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#troubleshooting) - [Docker Exit with Code 137](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#docker-exit-with-code-137) - [Removing Docker Containers](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#removing-docker-containers) - [Evaluation](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#evaluation) - [Server Load Analysis](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#server-load-analysis) - [Server Avg Load Analysis](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#server-avg-load-analysis) - [Server Failure Testing](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#server-failure-testing) - [Hashing Function Variation Analysis](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#hashing-function-variation-analysis) - [Group Details](https://github.com/PranavMehrotra/A1_Distributed_Systems/blob/master/#group-details) # Prerequisite ## 1. Docker: latest sudo apt-get update sudo apt-get install \ ca-certificates \ curl \ gnupg \ lsb-release sudo mkdir -p /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io ## 2. Docker-compose standalone sudo curl -SL https://github.com/docker/compose/releases/download/v2.15.1/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose # Getting Started ## Building Docker Images To create the necessary Docker images for the load balancer and servers, execute the following command: ```bash make install ``` ## Running Docker Containers To initiate the deployment of load balancer containers, execute the following command: ```bash make deploy ``` This command will launch the load balancer container, which, in turn, will spawn the initial N server Docker containers along with their heartbeat probing threads. Ensure that the necessary configurations are in place for a seamless deployment. The command also clears any existing containers using server or load balancer image (i.e. execute make clean). **Note:** The deployment command launches Docker in the background mode. Therefore, users are advised to check the docker-compose logs to view load-balancer logs. ## Interact with System To interact with the load balancer and send GET/POST requests, launch the interactive terminal using the following command: ```bash bash client.sh ``` ## Remove Existing Container To stop and remove all containers using the server image and load balancer, run the following command: ```bash make clean ``` Executing this command is recommended before running the main code to ensure there are no conflicting container instances already running. This helps prevent potential errors and ensures a clean environment for the code execution. ## Clear Existing Images To remove previously created server and load balancer images, execute the following command: ```bash make deepclean ``` It is advisable to run this command before executing the main code to eliminate any pre-existing images with the same name. This ensures a clean slate and avoids potential conflicts during the code execution. # Design Choices
  1. When executing the /add endpoint, users may provide existing server hostnames as part of the request. In such cases, the load balancer takes a proactive approach to ensure that the specified num_add parameter is honored. Even if the user supplies hostnames that already exist in the system, the load balancer will ignore already existing hostnames and generate new hostnames for additional servers to fulfill the exact count specified by num_add.
  2. When executing the /rm endpoint, users may provide hostnames for removal. To ensure the specified number of servers to be removed is consistently achieved, the load balancer employs a strategy wherein, if the user-provided hostname doesn't exist in the system, it randomly selects and removes a server hostname from the existing set.
  3. Every server is equipped with a heartbeat thread that sends a heartbeat message every 0.2 seconds. If no heartbeat is detected for two consecutive attempts, the server is declared dead, triggering the spawning of a new server. This mechanism prevents premature declarations of server death due to network fluctuations, ensuring stability in the system.
# Troubleshooting ## Docker Exit with Code 137 Code 137 indicated RAM memory related issue. Stop and remove already runining container to free up space. ## Removing Docker Containers ### 1. Stop docker container ``` Particular container: docker stop container_id Stop all runing container: docker stop $(docker ps -a -q) ``` ### 2. Remove docker container ``` Particular container: docker rm container_id Stop all runing container: docker rm $(docker ps -a -q) ``` # Evaluation ## Server Load Analysis To analyze the distribution load of 10,000 asynchronous requests on N=3 servers, follow these commands: #### Change to the 'analysis' directory: ```bash cd analysis ``` #### Run the Python script for analysis: ```bash python analysis.py ``` The analysis.py script sends 10,000 asynchronous requests (/home requests) to the load balancer. Based on the responses received, it plots a frequency map illustrating how many responses come from each server. Ensure that the necessary dependencies are installed and the system is properly configured before executing the analysis script. ## Server Avg Load Analysis Modify the NUM_INITIAL_SERVERS = 3 line number 9 of `load_balancer/client_handler.py` with 2 to 6 and run analysis.py to get average load and standard distribution of load distribution. Follow folder `analysis/old_hash/` for exact distribution bar plots. ## Server Failure Testing To analyze the server failure scenario execute the following: #### Change to the 'analysis' directory: ```bash cd analysis ``` #### Run the Python script for analysis: ```bash python kill_server.py ``` The kill_server.py script is designed to simulate the process of killing a server, validating the functionality of all endpoints in the event of server failure, and assessing the prompt respawn of the server. The effectiveness of server respawn is verified through the analysis of load distribution and the count of dropped requests(failed) during the time it takes for the heartbeat mechanism to detect and respawn the server. ## Hashing Function Variation Analysis After testing all the 9 combinations of **SHA-256, SHA-1, and MD5** hashing functions for request and server hashings, we found that the most effective combination is: - Server Hashing: SHA-1 - Request Hashing: MD5 To implement this new hash function configuration, follow these steps: 1. Open the file `load_balancer/consistent_hashing.py` and locate the `server_hash_func` and `request_hash_func` definition. 2. Comment the alternative hash function provided. 3. Save the changes. 4. Re-run the analysis script after deploying the load balancer again. 5. Ensure that you have cleared previous containers and images, and rebuild the project before executing the analysis script. This ensures that the new hash function is applied to the load balancer and the analysis is based on the updated configuration. The evaluation results of new hash function on variable number of servers are present in `analysis/md5_hash` # Group Details 1. Pranav Mehrotra (20CS10085) 2. Saransh Sharma (20CS30065) 3. Pranav Nyati (20CS30037) 4. Shreyas Jena (20CS30049)
近期下载者：
相关文件：
收藏者：

© 联合开发网 from 2004 | 联系站长 | 湘ICP备2023001425号 | 网安备43010502000604 |