datahub-cloudformation

所属分类:云计算
开发工具:GO
文件大小:417053KB
下载次数:0
上传日期:2023-03-10 21:32:02
上 传 者sh-1993
说明:  使用AWS CloudFormation部署数据中心堆栈的存储库
(Repository for deploying datahub stack using AWS CloudFormation)

文件列表:
Ingestion (0, 2023-06-23)
Ingestion\acryl-executor-ecs-stack.sh (1438, 2023-06-23)
Ingestion\fargate-login.sh (816, 2023-06-23)
Ingestion\templates (0, 2023-06-23)
Ingestion\templates\python.ecs.template.yaml (25221, 2023-06-23)
license (0, 2023-06-23)
s3upload.sh (907, 2023-06-23)
scripts (0, 2023-06-23)
scripts\check_extra_tags-v2.sh (866, 2023-06-23)
scripts\check_extra_tags-v3.sh (939, 2023-06-23)
scripts\check_extra_tags.sh (857, 2023-06-23)
scripts\python-jwt.tar.gz (716, 2023-06-23)
scripts\python-jwt (0, 2023-06-23)
scripts\python-jwt\create_jwt.py (660, 2023-06-23)
scripts\python-jwt\requirements.txt (13, 2023-06-23)
scripts\remove_namespace_finalizer.sh (548, 2023-06-23)
templates (0, 2023-06-23)
templates\datahub-deployment-cleanup.yaml (26874, 2023-06-23)
templates\datahub-deployment-existing-vpc-2-subnets.yaml (35177, 2023-06-23)
templates\datahub-deployment-existing-vpc-3-subnets.yaml (32247, 2023-06-23)
templates\datahub-deployment-v2-existing-vpc-2-subnets.yaml (41884, 2023-06-23)
templates\datahub-deployment-v2-existing-vpc-3-subnets.yaml (42055, 2023-06-23)
templates\datahub-deployment-v2.yaml (46362, 2023-06-23)
templates\datahub-deployment.yaml (37412, 2023-06-23)
templates\nested (0, 2023-06-23)
templates\nested\2-subnets (0, 2023-06-23)
templates\nested\2-subnets\admin-v2.yaml (37541, 2023-06-23)
templates\nested\2-subnets\admin.yaml (33349, 2023-06-23)
templates\nested\2-subnets\eks-cluster.yaml (27174, 2023-06-23)
templates\nested\2-subnets\eks-nodegroup.yaml (7781, 2023-06-23)
templates\nested\2-subnets\elasticsearch-v2.yaml (5843, 2023-06-23)
templates\nested\2-subnets\elasticsearch.yaml (6060, 2023-06-23)
templates\nested\2-subnets\msk-v2.yaml (5123, 2023-06-23)
templates\nested\2-subnets\msk.yaml (5462, 2023-06-23)
templates\nested\2-subnets\mysql-v2.yaml (6083, 2023-06-23)
... ...

# Cloudformation Demo ![AcryDatahubCFN](https://user-images.githubusercontent.com/1105928/138394072-c86ddffa-5b6d-433f-95c8-37***842445d4.png) ## step 1 to 4 runs on customer AWS account 1. upload templates/scripts/license to S3 - get license file from Acryl, for example: <> - clone this repo - set AWS access, then upload needed files to your S3 bucket (for example,cf-templates-xxx-us-west-2), under folder 'dev' ```console cd cloudformation export AWS_PROFILE=*** ./s3upload.sh cf-templates-xxx-us-west-2 dev xxx-customer1.yaml ``` 2. create stack to deploy datahub platform in AWS - choose region -> Cloudformation -> Create stack - choose template based on your use case - Template Amazon S3 URL: https://cf-templates-xxx-us-west-2.s3.us-west-2.amazonaws.com/dev/templates/datahub-deployment-v2.yaml (will create new VPC with 3 Subnets first, then deploy datahub to the new VPC) - Template Amazon S3 URL: https://cf-templates-xxx-us-west-2.s3.us-west-2.amazonaws.com/dev/templates/datahub-deployment-v2-existing-vpc-3-subnets.yaml (will use existing VPC with 3 Subnets to deploy datahub) - Template Amazon S3 URL: https://cf-templates-xxx-us-west-2.s3.us-west-2.amazonaws.com/dev/templates/datahub-deployment-v2-existing-vpc-2-subnets.yaml (will use existing VPC with 2 Subnets deploy datahub) - StackName: dev-datahub - TemplateBucketName: cf-templates-xxx-us-west-2 - Environment: dev - VPCID: vpc-0xxxxxxxxxxxxxxxx - The AZ's to deploy to: choose 3 or 2 AZs per your use case - The Existing Private Subnet 1 ID: subnet-1xxxxxxxxxxxxxxxx - The Existing Private Subnet 2 ID: subnet-2xxxxxxxxxxxxxxxx - The Existing Private Subnet 3 ID: subnet-3xxxxxxxxxxxxxxxx, or leave empty for 2-subnet setup - Enable Creation of ElasticSearch Service Role: set to true if ServiceLinked Role for ES doesn't exists - The AWS IAM Role arn that will be allowed to manage EKS, for example: aws:iam::AccountID:role/admin-role - DataHub Domain Name: datahub.a.b.c - ELB cert arn: arn: arn:aws:acm:REGION:AccountID:certificate/xx-xx-xx-xx-xx (ssl cert for datahub.a.b.c) - The Elastic Load Balancer Inbound CIDRs: comma seperated CIDR list that can access DataHub ALB - Kots Admin Domain Name: kotsadm.e.f.g - Kots ELB cert arn: arn:aws:acm:REGION:AccountID:certificate/xx-xx-xx-xx-xx (ssl cert for kotsadm.e.f.g) - Applicaiton: Kots application name - ApplicationReleaseChannel: choose Kots Application Release Channel - CreatePrivateLink: Choose 'true' to create private link endpoint service - RemoveTempResources: if you want to keep admin provision host, choose 'false' - Tags, can have customized tags here, no space allowed in either Key or Value - Stack failure options: choose 'rollback on failure' - for troubleshooting, choose 'Preserve successfully provisioned resources' (useful when working on development of cloudformation) - check: - "I acknowledge that AWS CloudFormation might create IAM resources with custom names." - "I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND" - click "Create stack" - you will see a stack 'dev-datahub' (this is master stack), and it will invoke nested stacks in order 3. find datahub platform info - after Stack Info show Status 'CREATE_COMPLETE', you can find needed info from nested stack <>'s Outputs 4. manually create DNS record - create datahub.xxx.xxx.com to point to the datahub-frontend ALB - create new routes to reach this new VPC - access https://datahub.xxx.xxx.com to use datahub ## step 5 runs on Acryl AWS account 5. manually create VPC endpoint - under Acryl AWS account, go to same region as customer, find service by service name, for example com.amazonaws.vpce.us-west-2.vpce-svc-*** (get service name from step 4.), select shared vpc, choose 3 private subnets, attach default security group - access https://{vpc_endpoint_dns} to use manage datahub release

近期下载者

相关文件


收藏者