Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master' into feat/new-es-7
Browse files Browse the repository at this point in the history
  • Loading branch information
thanh-nguyen-dang committed Nov 6, 2023
2 parents e3f75a5 + 9265477 commit f6ec8c2
Show file tree
Hide file tree
Showing 599 changed files with 8,438 additions and 109,337 deletions.
10 changes: 3 additions & 7 deletions .github/workflows/image_build_and_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,11 +67,6 @@ jobs:
ls ..
pwd
- name: Extract branch name
shell: bash
run: echo "branch=$(echo $(echo ${GITHUB_REF#refs/heads/} | tr / _))" >> $GITHUB_OUTPUT
id: extract_branch

- name: Copy configuration files
run: |
cp -f tests/integrated_tests/gen3/tube/etlMapping.yaml ../compose-etl/configs/etlMapping.yaml
Expand All @@ -87,8 +82,9 @@ jobs:
- name: Replace tube branch
working-directory: ../compose-etl
run: |
BRANCH=${{ steps.extract_branch.outputs.branch }}
sed -i "s/tube:master/tube:${BRANCH//\//_}/g" docker-compose.yml
BRANCH=${{ github.head_ref || github.ref_name }}
BRANCH=$(echo $BRANCH | tr / _)
sed -i "s/tube:master/tube:$BRANCH/g" docker-compose.yml
- name: change max_map_count (needed for running Elasticsearch in container)
working-directory: ../compose-etl
Expand Down
1,017 changes: 425 additions & 592 deletions poetry.lock

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ gen3datamodel = "3.0.2"
hdfs = "2.7.0"
psycopg2-binary="2.9.3"
psqlgraph = "3.0.1"
py2neo = "2021.2.3"
pyspark = "3.3.0"
urllib3 = "1.26.16"
requests = "^2.28.1"
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,6 @@
"2020-01-17 06:02:21.920928+00","{phs-jenkins,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""9a44404f79dc6f4a703a3c920cb8168d"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""3d9cba6023"", ""file_size"": 82, ""object_id"": ""dg.XXXX/96a8bdbb-ffe6-4b73-85d8-077113ba4cc8"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""IDAT"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_089cfcb3b9"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-17T06:02:21.920928+00:00"", ""updated_datetime"": ""2020-01-17T06:02:21.920928+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 v2.0""}","6ee014e3-08fd-4793-be6d-efceea9f1c85"
"2020-01-17 06:02:25.486043+00","{phs-jenkins,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""35cc1c62ae485d5fd84d76efe1f1038f"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""374cc8f429"", ""file_size"": 45, ""object_id"": ""dg.XXXX/293ab724-5952-4894-b073-8bf8cdef7187"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_0f7903f8b1"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-17T06:02:25.486043+00:00"", ""updated_datetime"": ""2020-01-17T06:02:25.486043+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 v2.0""}","9b43176f-a5ff-449d-8809-98b009e223c6"
"2020-01-17 06:02:25.996377+00","{phs-jenkins,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""2b08d27ca9f8c701e1683b6cc35bca56"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""6a9bec74b7"", ""file_size"": 73, ""object_id"": ""dg.XXXX/414fc12f-07aa-44a2-a636-5f1b8b7a4d1d"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""IDAT"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_0e4a6da4cf"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-17T06:02:25.996377+00:00"", ""updated_datetime"": ""2020-01-17T06:02:25.996377+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 v2.0""}","b1fe7a37-253b-46d5-8498-17fa2acc34f6"
"2020-01-17 06:02:27.622894+00","{phs-jenkins,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""1417d7dd12491d2fa2df5dfddf0695f1"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""98454cd64e"", ""file_size"": 43, ""object_id"": ""dg.XXXX/420f68e3-dfc5-40f7-a488-429855128d57"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_bae932c9b9"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-17T06:02:27.622894+00:00"", ""updated_datetime"": ""2020-01-17T06:02:27.622894+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 v2.0""}","07ce47b3-e3cf-4449-95e2-cd0f97f63231"
"2020-02-17 22:11:08.545392+00","{phs-jenkins,jnkns}","{}","{""type"": ""raw_snp_genotype"", ""state"": ""validated"", ""md5sum"": ""61a2dc1880f911805d52873c6d97c793"", ""samples"": [], ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""zoe"", ""file_size"": 21153942804, ""object_id"": ""dg.XXXX/420f68e3-dfc5-40f7-a488-429855128888"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_bae932ddd9"", ""data_category"": ""SNP Genotypes"", ""state_comment"": ""zoe"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-13T06:02:27.622894+00:00"", ""updated_datetime"": ""2020-01-13T06:02:27.622894+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 v2.0"", ""core_metadata_collections"": [{""id"": ""df2fcfd1-b591-46af-85fb-3973e2915c60"", ""submitter_id"": ""core_metadata_collection_9f28aa40cc""}]}","07ce47b3-e3cf-4449-95e2-cd0f97f63333"
"2021-01-08 19:06:21.543558+00","{phs-jenkins2,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""b77ec4ef33b0194bfff84fa84fd1ba17"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""ab3514fb8d"", ""file_size"": 47, ""object_id"": ""dg.XXXX/2540642a-1f7a-429e-941b-01179b46c987"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins2"", ""data_format"": ""PLINK BED File"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_6c8561c54d"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2021-01-08T19:06:21.543558+00:00"", ""updated_datetime"": ""2021-01-08T19:06:21.543558+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v2.0 BeadChip""}","db5edc3a-9d40-4f0c-98cd-fa81340a800d"
"2021-01-08 19:06:21.953907+00","{phs-jenkins2,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""ad7d1b0fc5b2d142a49708f5e5e255bb"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""b9473dc19b"", ""file_size"": 38, ""object_id"": ""dg.XXXX/d3f6e0fb-9b14-406d-b7ec-8cfbde1e5f26"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins2"", ""data_format"": ""IDAT"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_87a3cdfdaa"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2021-01-08T19:06:21.953907+00:00"", ""updated_datetime"": ""2021-01-08T19:06:21.953907+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v2.0 BeadChip""}","a4685a40-9e93-41ca-8282-55ddf0406148"
Expand Down Expand Up @@ -159,3 +158,4 @@
"2021-12-03 17:29:46.819802+00","{phs-jenkins2,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""1934980c70028286e3d0e1dc93fc52c2"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""ab4a7dd26d"", ""file_size"": 38, ""object_id"": ""dg.XXXX/d91d5320-1361-47b5-a621-682a68942017"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins2"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_94ec1ee267"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2021-12-03T17:29:46.819802+00:00"", ""updated_datetime"": ""2021-12-03T17:29:46.819802+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v3.0 BeadChip""}","2a771ed5-65f6-4fe3-a495-50a8ba1c3215"
"2021-12-03 17:29:47.2528+00","{phs-jenkins2,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""7fcab28b2175fe30b96f403fb1e8de67"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""906409e0cd"", ""file_size"": 98, ""object_id"": ""dg.XXXX/4949a7af-45bb-435f-90dd-c1d9f64c5e93"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins2"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_4c198ea272"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": null, ""created_datetime"": ""2021-12-03T17:29:47.252800+00:00"", ""updated_datetime"": ""2021-12-03T17:29:47.252800+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v3.0 BeadChip""}","10db8aa1-d1ce-49c8-9b6d-f4d8eaf9a13e"
"2021-12-03 17:29:47.627038+00","{phs-jenkins2,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""0539888381dc2b0710b5d98a5629c3a3"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""eb846b8e74"", ""file_size"": 91000000000, ""object_id"": ""dg.XXXX/36aa80ae-f382-45a1-9749-bd7f97b0d7c5"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins2"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_95b1f96f48"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2021-12-03T17:29:47.627038+00:00"", ""updated_datetime"": ""2021-12-03T17:29:47.627038+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v2.0 BeadChip""}","e6e8f885-e18e-40e5-9602-66b0dea0d09f"
"2020-01-17 06:02:27.622894+00","{phs-jenkins,jnkns}","{}","{""state"": ""validated"", ""md5sum"": ""1417d7dd12491d2fa2df5dfddf0695f1"", ""data_type"": ""SNP Intensities and/or Called Genotypes"", ""file_name"": ""98454cd64e"", ""file_size"": 43, ""object_id"": ""dg.XXXX/420f68e3-dfc5-40f7-a488-429855128d58"", ""file_state"": ""registered"", ""project_id"": ""jnkns-jenkins"", ""data_format"": ""Illumina Final Report"", ""assay_method"": ""SNP Array"", ""submitter_id"": ""raw_snp_genotype_bae932c9b9"", ""data_category"": ""SNP Genotypes"", ""assay_instrument"": ""Illumina"", ""created_datetime"": ""2020-01-17T06:02:27.622894+00:00"", ""updated_datetime"": ""2023-05-22T18:49:47.758824+00:00"", ""assay_instrument_model"": ""Illumina Infinium Global Screening Array-24 Kit v2.0 BeadChip""}","07ce47b3-e3cf-4449-95e2-cd0f97f63231"
Original file line number Diff line number Diff line change
@@ -1 +1 @@
"2020-01-16 19:05:12.953264+00","{}","{}","{""schema_version"": ""-7780338601903350209""}","root"
"2020-01-16 19:05:12.953264+00","{}","{}","{""schema_version"": ""1879278627024622741""}","root"
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading

0 comments on commit f6ec8c2

Please sign in to comment.