monster

所属分类:大数据
开发工具:Shell
文件大小:0KB
下载次数:0
上传日期:2021-07-06 16:50:58
上 传 者sh-1993
说明:  DSP数据工程中Monster团队的中心
(Hub for the Monster team in DSP Data Engineering)

文件列表:
archive/ (0, 2021-07-06)
getting-started/ (0, 2021-07-06)
getting-started/monster.png (68243, 2021-07-06)
getting-started/pages/ (0, 2021-07-06)
getting-started/pages/dev-setup.md (4638, 2021-07-06)
getting-started/pages/group-team-setup.md (4949, 2021-07-06)
getting-started/pages/new-broadies.md (2197, 2021-07-06)
getting-started/pages/reading-list.md (1089, 2021-07-06)
getting-started/pages/toolbox-icon.png (1453, 2021-07-06)
getting-started/pages/vault-github-token.png (271385, 2021-07-06)
getting-started/scripts/ (0, 2021-07-06)
getting-started/scripts/clone-repositories (1209, 2021-07-06)
getting-started/scripts/install-tools (4349, 2021-07-06)
getting-started/scripts/login (532, 2021-07-06)
getting-started/scripts/setup-graal (1283, 2021-07-06)
getting-started/scripts/setup-vault (1227, 2021-07-06)
tech-docs/ (0, 2021-07-06)
tech-docs/devops/ (0, 2021-07-06)
tech-docs/devops/motivation.md (997, 2021-07-06)
tech-docs/devops/requirements.md (1914, 2021-07-06)
tech-docs/devops/strategy.md (7326, 2021-07-06)
tech-docs/stack/ (0, 2021-07-06)
tech-docs/stack/languages.md (6528, 2021-07-06)
tech-docs/stack/systems.md (8871, 2021-07-06)
tech-docs/stack/tools.md (4498, 2021-07-06)
templates/ (0, 2021-07-06)
templates/python-project/ (0, 2021-07-06)
templates/python-project/.pre-commit-config.yaml (102, 2021-07-06)
... ...

# The Monster Team [![Monster Slack](https://img.shields.io/badge/Slack%20Channel-%23monster-blue.svg?style=flat)](https://broadinstitute.slack.com/messages/CCAU5L6LV/) [![Monster CI Slack](https://img.shields.io/badge/Slack%20Channel-%23monster--ci-blue.svg?style=flat)](https://broadinstitute.slack.com/messages/CFXEDUUP5/) New to the team? [Start here](./getting-started/README.md). ## People | Name | Role | GitHub | | --- | --- | --- | | Jeff Korte | Product Owner | @JeffKorte | | Quazi Hoque | Software Engineer | @quazi-broad | | Drew Herbst | Tech Lead | @aherbst-broad | ## GitHub Teams * [DSP Monsters](https://github.com/orgs/broadinstitute/teams/dsp-monsters) - Team for repositories under the `broadinstitute` org * [Monster](https://github.com/orgs/DataBiosphere/teams/monster) - Team for repositories under the `DataBiosphere` org ## Projects ### Data Modeling Linked Data definitions for the Terra Core Data Model, with extensions for unmodeled datasets. #### Documentation * [Google Docs](https://drive.google.com/drive/folders/1n8TP4Q_4n2pCysjQz2Hkn2kpHGEILLCj) * [Confluence - DSP Core Data Model](https://broadinstitute.atlassian.net/wiki/spaces/DSPCDM/overview) * [Confluence - FAIR Community of Practice](https://broadinstitute.atlassian.net/wiki/spaces/FairCoP/overview) #### GitHub repos * [TerraCore Data Model](https://github.com/DataBiosphere/terra-core-data-model) - Data Model definitions and examples ### Data Ingest Pipelines for moving data into the [Jade Data Repository](https://github.com/databiosphere/jade-data-repo). #### Documentation * [Google Docs](https://drive.google.com/drive/folders/1LjtBbMZs5-FqTGcRjw80ZBlHhfd_LT2z) #### GitHub repos * [ClinVar](https://github.com/DataBiosphere/clinvar-ingest) - ETL pipeline for the ClinVar dataset * [ENCODE](https://github.com/DataBiosphere/encode-ingest) - ETL pipeline for the ENCODE dataset * [Dog Aging](https://github.com/DataBiosphere/dog-aging-ingest) - ETL pipeline for the Dog Aging Project dataset * [HCA](https://github.com/DataBiosphere/hca-ingest) - ETL pipeline for the HCA ### Ingest Utilities Tools and libraries used to support the top-level ingest pipelines. #### GitHub repos * [Base utilities](https://github.com/DataBiosphere/ingest-utils) - Common utilities shared across our batch ETL projects * [XML-to-JSON-list](https://github.com/broadinstitute/monster-xml-to-json-list) - Command-line tool for mechanical conversion of XML into Beam-friendly JSON ### Operations Infrastructure, configuration, and shared code used to manage developing and deploying our services. ### GitHub repos * [Helm charts](https://github.com/broadinstitute/monster-helm) - Custom Helm charts for pieces of Monster infrastructure * [Core deployments](https://github.com/broadinstitute/monster-deploy) - Terraform modules, Helm releases, and deploy scripts for Monster's GCP environments * [setup-chart-releaser](https://github.com/broadinstitute/setup-chart-releaser) - GitHub Action to install [Chart Releaser](https://github.com/helm/chart-releaser) ## Semi-Archived The repositories in this section are still being used, but we're trying to move away from them. ### Data Ingest Framework Our first stabs at data ingest envisioned a framework of dataset-agnostic services. We shifted away from that pattern because it introduced significant overhead vs. custom pipelines using common command-line tools. #### GitHub repos * [Transporter](https://github.com/databiosphere/transporter) - Bulk file-transfer system * [Storage Libs](https://github.com/broadinstitute/monster-storage-libs) - Utility libraries for I/O against external storage systems

近期下载者

相关文件


收藏者