PythonDjango支持像PostgresCitus这样的分布式多租户数据库

  • w2_941130
    了解作者
  • 64.4KB
    文件大小
  • zip
    文件格式
  • 0
    收藏次数
  • VIP专享
    资源类型
  • 0
    下载次数
  • 2022-04-22 12:19
    上传日期
Python/Django支持像Postgres Citus这样的分布式多租户数据库
PythonDjango支持像PostgresCitus这样的分布式多租户数据库.zip
内容介绍
# django-multitenant [![Build Status](https://travis-ci.org/citusdata/django-multitenant.svg?branch=unit-test)](https://travis-ci.org/citusdata/django-multitenant) Python/Django support for distributed multi-tenant databases like Postgres+Citus Enables easy scale-out by adding the tenant context to your queries, enabling the database (e.g. Citus) to efficiently route queries to the right database node. There are architecures for building multi-tenant databases viz. **Create one database per tenant**, **Create one schema per tenant** and **Have all tenants share the same table(s)**. This library is based on the 3rd design i.e **Have all tenants share the same table(s)**, it assumes that all the tenant relates models/tables have a tenant_id column for representing a tenant. The following link talks more about the trade-offs on when and how to choose the right architecture for your multi-tenat database: https://www.citusdata.com/blog/2016/10/03/designing-your-saas-database-for-high-scalability/ **Other useful links on multi-tenancy**: 1. https://www.citusdata.com/blog/2017/03/09/multi-tenant-sharding-tutorial/ 1. https://www.citusdata.com/blog/2017/06/02/scaling-complex-sql-transactions/ ## Installation: 1. pip install --no-cache-dir django_multitenant ## Supported Django versions/Pre-requisites. | Python | Django | | ------------- | -------------:| | 2.7 | 1.9 | | 2.7 | 1.10 | | 2.7 | 1.11 | | 3.6 | 2.0 | ## Usage: In order to use this library you can either use Mixins or have your models inherit from our custom model class. ### Changes in Models: 1. In whichever files you want to use the library import it: ```python from django_multitenant.fields import * from django_multitenant.models import * ``` 1. All models should inherit the TenantModel class. `Ex: class Product(TenantModel):` 1. Define a static variable named tenant_id and specify the tenant column using this variable. `Ex: tenant_id='store_id'` 1. All foreign keys to TenantModel subclasses should use TenantForeignKey in place of models.ForeignKey 1. A sample model implementing the above 2 steps: ```python class Product(TenantModel): store = models.ForeignKey(Store) tenant_id='store_id' name = models.CharField(max_length=255) description = models.TextField() class Meta(object): unique_together = ["id", "store"] class Purchase(TenantModel): store = models.ForeignKey(Store) tenant_id='store_id' product_purchased = TenantForeignKey(Product) ``` ### Changes in Models using mixins: 1. In whichever files you want to use the library import it by just saying ```python from django_multitenant.mixins import * ``` 1. All models should use the `TenantModelMixin` and the django `models.Model` or your customer Model class `Ex: class Product(TenantModelMixin, models.Model):` 1. Define a static variable named tenant_id and specify the tenant column using this variable. `Ex: tenant_id='store_id'` 1. All foreign keys to TenantModel subclasses should use TenantForeignKey in place of models.ForeignKey 1. A sample model implementing the above 2 steps: ```python class ProductManager(TenantManagerMixin, models.Manager): pass class Product(TenantModelMixin, models.Model): store = models.ForeignKey(Store) tenant_id='store_id' name = models.CharField(max_length=255) description = models.TextField() objects = ProductManager() class Meta(object): unique_together = ["id", "store"] class PurchaseManager(TenantManagerMixin, models.Manager): pass class Purchase(TenantModelMixin, models.Model): store = models.ForeignKey(Store) tenant_id='store_id' product_purchased = TenantForeignKey(Product) objects = PurchaseManager() ``` ### Automating composite foreign keys at db layer: 1. Creating foreign keys between tenant related models using TenantForeignKey would automate adding tenant_id to reference queries (ex. product.purchases) and join queries (ex. product__name). If you want to ensure to create composite foreign keys (with tenant_id) at the db layer, you should change the database ENGINE in the settings.py to `django_multitenant.backends.postgresql`. ```python 'default': { 'ENGINE': 'django_multitenant.backends.postgresql', ...... ...... ...... } ``` ### Where to Set the Tenant? 1. Write authentication logic using a middleware which also sets/unsets a tenant for each session/request. This way developers need not worry about setting a tenant on a per view basis. Just set it while authentication and the library would ensure the rest (adding tenant_id filters to the queries). A sample implementation of the above is as follows: ```python class SetCurrentTenantFromUser(object): def process_request(self, request): if not hasattr(self, 'authenticator'): from rest_framework_jwt.authentication import JSONWebTokenAuthentication self.authenticator = JSONWebTokenAuthentication() try: user, _ = self.authenticator.authenticate(request) except: # TODO: handle failure return try: #Assuming your app has a function to get the tenant associated for a user current_tenant = get_tenant_for_user(user) except: # TODO: handle failure return set_current_tenant(current_tenant) def process_response(self, request, response): set_current_tenant(None) return response ``` ```python MIDDLEWARE_CLASSES = ( 'our_app.utils.multitenancy.SetCurrentTenantFromUser', ) ``` 1. Set the tenant using set_current_tenant(t) api in all the views which you want to be scoped based on tenant. This would scope all the django API calls automatically(without specifying explicit filters) to a single tenant. If the current_tenant is not set, then the default/native API without tenant scoping is used. ```python def application_function: # current_tenant can be stored as a SESSION variable when a user logs in. # This should be done by the app t = current_tenant #set the tenant set_current_tenant(t); #Django ORM API calls; #Command 1; #Command 2; #Command 3; #Command 4; #Command 5; ``` ## Supported APIs: 1. Most of the APIs under Model.objects.*. 1. Model.save() injects tenant_id for tenant inherited models. ```python s=Store.objects.all()[0] set_current_tenant(s) #All the below API calls would add suitable tenant filters. #Simple get_queryset() Product.objects.get_queryset() #Simple join Purchase.objects.filter(id=1).filter(store__name='The Awesome Store').filter(product__description='All products are awesome') #Update Purchase.objects.filter(id=1).update(id=1) #Save p=Product(8,1,'Awesome Shoe','These shoes are awesome') p.save() #Simple aggregates Product.objects.count() Product.objects.filter(store__name='The Awesome Store').count() #Subqueries Product.objects.filter(name='Awesome Shoe'); Purchase.objects.filter(product__in=p); ``` ## Credits This library uses similar logic of setting/getting tenant object as in [django-simple-multitenant](https://github.com/pombredanne/django-simple-multitenant). We thank the authors for their efforts. ## License Copyright (C) 2018, Citus Data Licensed under the MIT license, see LICENSE file for details. This library contains portions of code from django-simple-multitenant: Copyright (C) 2011, Daniel Romaniuk Licensed under the AGPL 3.0 license, see LICENSE file for details.
评论
    相关推荐
    • askreduce:分布式问题
      分布式问题 仅使用intro_python材料进行初始设置 intro_python.md是 CSQ(冒号分隔的问题) csq2fix.py以 JSON 格式转换为 Django 固定装置格式 该输出进入要加载的shufflesort/fixtures 设置新鲜: python ...
    • django-q:Django的多处理分布式任务队列
      Django的多处理分布式任务队列 特征 多处理工作者池 异步任务 计划的,cron和重复的任务 签名并压缩的软件包 失败和成功数据库或缓存 结果挂钩,组和链 Django Admin集成 PaaS与多个实例兼容 多集群监控器 Redis,...
    • python-django:Django 框架的 OpenTracing 检测
      这个包通过在 Django 项目中启用分布式跟踪。 一旦生产系统应对真正的并发或拆分成许多服务,关键(以前很容易)的任务就会变得困难:面向用户的延迟优化、后端错误的根本原因分析、关于现在分布式系统的不同部分的...
    • leopardv:分布式爬虫管理系统
      SpiderMAN是基于Django + scrapyd + bootstrap的分布式爬虫管理系统,能实现分布式任务调度,对数据的监控等 功能 用户登录 管理服务器节点 管理爬虫,爬虫部署 查看数据统计 容器快速部署 用户登录 官员管理 查看...
    • django-stard-packages软件包
      Django的多处理分布式任务队列 7.7千 5 一组可配置的面板,可显示各种调试信息,并提供... 6.3千 6 集成的Django应用程序集,可解决身份验证,注册... 6.2千 7 Django扩展 5.2千 8 域驱动的Django电子商务框架 ...
    • JustDownlink:基于Scrapy+Elasticsearch+Django搭建的分布式电影搜索
      基于 scrapy + elasticsearch + django 搭建的分布式电影搜索 利用 scrapy 爬取知名电影网站的下载链接 利用 elasticsearch 存储数据 利用 django 搭建电影搜索界面 数据采集 支持同步、异步数据存储至 Mysql 数据库...
    • Gerapy:基于Scrapy,Scrapyd,Django和Vue.js的分布式爬网程序管理框架
      基于Scrapy,Scrapyd,Scrapyd-Client,Scrapyd-API,Django和Vue.js的分布式爬虫管理框架。 文献资料 可从和在线获取文档。 支持 Gerapy是基于Python 3.x开发的。 稍后可能会支持Python2.x。 用法 通过pip安装...
    • celery——cmd命令.zip
      在iis环境中使用django框架,在里面再用celery分布式,在window环境下如何无窗口式运行celery
    • django-celery
      celery是一个异步任务队列/基于分布式消息传递的作业队列
    • GaussDB_100_1.0.1-DATABASE-REDHAT-64bit.tar.gz
      guassdb100在redhat上安装包,单机部署的包,安装步骤请看我的文中介绍,经过大量实验搭建总结出来的文档