Distributed-File-Storage-System

所属分类:云数据库/云存储
开发工具:Python
文件大小:5321KB
下载次数:0
上传日期:2019-04-12 01:54:01
上 传 者sh-1993
说明:  一种开源、可扩展、去中心化、健壮的异构文件存储解决方案,具有容错、副本...
(An open-source, scalable, decentralized, robust, heterogeneous file storage solution which is fault tolerant, replicated, distributed and lets you upload, download, and see the catalog of other cluster with low latency and LRU cache capabilities.)

文件列表:
Project Report.pdf (982227, 2019-04-12)
SuperNode (0, 2019-04-12)
SuperNode\ClusterStatus.py (1892, 2019-04-12)
SuperNode\build.sh (108, 2019-04-12)
SuperNode\db.py (832, 2019-04-12)
SuperNode\proto (0, 2019-04-12)
SuperNode\proto\fileService.proto (1139, 2019-04-12)
SuperNode\proto\fileService_pb2.py (15788, 2019-04-12)
SuperNode\proto\fileService_pb2_grpc.py (7917, 2019-04-12)
SuperNode\requirements.txt (30, 2019-04-12)
SuperNode\superNode.py (10053, 2019-04-12)
__pycache__ (0, 2019-04-12)
__pycache__\FluffyServer.cpython-36.pyc (3305, 2019-04-12)
__pycache__\HeartbeatService.cpython-36.pyc (2029, 2019-04-12)
__pycache__\database.cpython-36.pyc (2326, 2019-04-12)
__pycache__\database.cpython-37.pyc (2289, 2019-04-12)
__pycache__\db.cpython-36.pyc (1088, 2019-04-12)
__pycache__\db.cpython-37.pyc (831, 2019-04-12)
__pycache__\fluffy_pb2.cpython-36.pyc (3819, 2019-04-12)
__pycache__\fluffy_pb2.cpython-37.pyc (3808, 2019-04-12)
__pycache__\fluffy_pb2_grpc.cpython-36.pyc (2993, 2019-04-12)
__pycache__\fluffy_pb2_grpc.cpython-37.pyc (2984, 2019-04-12)
__pycache__\heartbeat_pb2.cpython-36.pyc (2628, 2019-04-12)
__pycache__\heartbeat_pb2_grpc.cpython-36.pyc (1681, 2019-04-12)
build_pb.sh (108, 2019-04-12)
client.py (5149, 2019-04-12)
config.yaml (880, 2019-04-12)
files (0, 2019-04-12)
files\flix.gif (3620438, 2019-04-12)
files\wow2.jpg (36899, 2019-04-12)
generated (0, 2019-04-12)
generated\__pycache__ (0, 2019-04-12)
generated\__pycache__\fileService_pb2.cpython-37.pyc (4985, 2019-04-12)
generated\__pycache__\fileService_pb2_grpc.cpython-37.pyc (4102, 2019-04-12)
generated\__pycache__\fluffy_pb2.cpython-36.pyc (3771, 2019-04-12)
generated\__pycache__\fluffy_pb2_grpc.cpython-36.pyc (2945, 2019-04-12)
... ...

# Distributed File Storage Project Fluffy is an open-source, scalable, decentralized, robust, heterogeneous file storage solution which ensures that multiple servers can inter-operate to form a dynamic ‘overlay’ fabric. The system supports some important design aspects such as - - Language Agnostic - System failure & recovery - Work-stealing algorithms - Scalability - Robustness - Queues to serve multiple requests simultaneously - Cluster Consensus (RAFT) - Data Replication - Caching Optimization - Efficient Searching ##### The following user services are currently supported (with an option to modularly add new services at any time) - 1. File Upload (supports any file type - pdf, img, avi, txt, mp4, xml, json, m4v, etc.) 2. File Download 3. File Search 4. File List (Lists all files on the system that belong to a specific user) 5. File Delete 6. File Update This system allows many clients to have access to data and supports operations (create, delete, modify, read, write) on that data. Each data file is partitioned into several parts called chunks. Each chunk is stored on different remote machines, facilitating the parallel execution of applications. **************************************************************************************************************** ### Architecture Diagram ![Link to Architecture Diagram](https://github.com/shubhamsawantsjsu/Distributed-File-Storage-System/blob/master/images/ArchitectureDiagram.png) **************************************************************************************************************** ### Technology Stack ![Link to Technology Stack Diagram](https://github.com/shubhamsawantsjsu/Distributed-File-Storage-System/blob/master/images/TechStack.png) **************************************************************************************************************** ### Usage - install all depenedencies as mentioned in requirements.txt - Check config.yaml to configure the server IPs and other parameters. - Update iptable.txt to incude all the IPs in your cluster (for each individual cluster) #### Starting supernode ``` python3 supernode.py ``` #### Strating cluster nodes ``` python3 server.py four ``` ``` python3 server.py five ``` ``` python3 server.py six ``` #### Starting client ``` python3 client.py ```

近期下载者

相关文件


收藏者