Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INF-597: Terraform config files not updating and Flower docker container not building successfully. #598

Conversation

alexmassen-hane
Copy link
Contributor

@alexmassen-hane alexmassen-hane commented Feb 14, 2023

  1. When using the observatory terraform build-image config.yaml command, it was not copying/overwriting terraform config files to the observatory home directory and thus when deploying a new version of the platform to the cloud, it was using the old terraform config. I have now forced the platform to delete the old terraform config files before copying the new ones across from the codebase.

  2. After updating the terraform files a strange permissions issue with docker arose, causing the flower container to fail when building the main VM. Have added the appropriate permissions to the /home/packer folder for the airflow user so the Flower container can build successfully.

Waiting on results from the unit tests.

@codecov
Copy link

codecov bot commented Feb 14, 2023

Codecov Report

Base: 92.26% // Head: 92.25% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (bef78b6) compared to base (9724174).
Patch coverage: 83.33% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #598      +/-   ##
===========================================
- Coverage    92.26%   92.25%   -0.02%     
===========================================
  Files           50       50              
  Lines         6829     6831       +2     
  Branches       779      780       +1     
===========================================
+ Hits          6301     6302       +1     
  Misses         409      409              
- Partials       119      120       +1     
Impacted Files Coverage Δ
...bservatory/platform/terraform/terraform_builder.py 96.77% <83.33%> (-1.03%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@jdddog jdddog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix Alex, that's great. Just a small comment about one of the functions.

def copy_dir(source_path: str, destination_path: str, ignore):
distutils.dir_util.copy_tree(source_path, destination_path)
def copy_dir(source_path: str, destination_path: str, ignore, update: int = 0):
distutils.dir_util.copy_tree(source_path, destination_path, update=update)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the update parameter is already set to zero by default in distutils.dir_util.copy_tree, so we shouldn't need to set it.

I seem to have messed up some of this code, because the ignore parameter doesn't get passed to anything. Maybe we were originally using shutil.copytree as that has an ignore parameter? We should probably either remove the ignore parameter or figure out if we could use shutil.copytree which can ignore files with patterns.

@jdddog jdddog force-pushed the INF-597-observatory-platform-main-vm-image-fails-to-build-flower-container branch from 87d89cc to bef78b6 Compare February 15, 2023 21:09
@jdddog jdddog self-requested a review February 15, 2023 21:09
Copy link
Contributor

@jdddog jdddog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Alex, looks good.

@jdddog jdddog merged commit 5f15c22 into develop Feb 16, 2023
@jdddog jdddog deleted the INF-597-observatory-platform-main-vm-image-fails-to-build-flower-container branch February 16, 2023 00:19
jdddog pushed a commit that referenced this pull request Feb 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants