Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Suggested change to AmazonEbsSurrogate to fix unknown configuration key error and expected map got slice error #181

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Suggested change to AmazonEbsSurrogate to fix unknown configuration key error and expected map got slice error #181

wants to merge 2 commits into from

Conversation

Pixxle
Copy link

@Pixxle Pixxle commented Oct 2, 2020

Issue

https://www.packer.io/docs/builders/amazon-ebs
Issue 1:
Amazon EBS has no valid parameter "source_device_name" and packer will fail to build any template generated with AmazonEbsSurrogate.

Packer validate output from Packerlicious.AmazonEbsSurrogate fails with
unknown configuration key: '"source_device_name"'

Issue 2:
Packer expects a single ami_root_device and not a list of them. AmazonEbsSurrogate requires the ami_root_device to be a list of BlockDeviceMapping, when outputing packer template from packerlicious AmazonEbsSurrogate, the template fails with:

* 'ami_root_device' expected a map, got 'slice'

List of Changes Proposed

Change prop of AmazonEbsSurrogate to only expect single BlockDeviceMapping on the key ami_root_device and remove the prop source_device_name completely from AmazonEbsSurrogate.

Testing Evidence

packer validate before removing source_device_name

Before changes:

(venv) {12:54}~/Documents/packerlicious/tests:master ✗ ➭ packer validate output.json
Error: Failed to prepare build: "amazon-ebssurrogate"

1 error occurred:
        * unknown configuration key: '"source_device_name"'

(venv) {12:42}~/Documents/packerlicious/tests:master ✗ ➭ packer validate output.json 
Error: Failed to prepare build: "amazon-ebssurrogate"

1 error(s) decoding:

* 'ami_root_device' expected a map, got 'slice'

After changes:


(venv) {12:58}~/Documents/el-kamino:master ✗ ➭ packer validate output.json
(venv) {12:58}~/Documents/el-kamino:master ✗ ➭ echo $?
0


(venv) {13:02}~/Documents/packerlicious/tests:master ✗ ➭ python3 -m pytest     
================================================= test session starts ==================================================
platform darwin -- Python 3.8.5, pytest-6.1.0, py-1.9.0, pluggy-0.13.1 -- /Users/dennis.hedlund/Documents/packerlicious/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/dennis.hedlund/Documents/packerlicious, configfile: setup.cfg
collected 165 items                                                                                                    

packerlicious/test_builder.py::TestBuilderAttributes::test_support_named_builds PASSED                           [  0%]
packerlicious/test_builder_alicloud.py::TestAliCloudBuilder::test_required_fields_missing PASSED                 [  1%]
packerlicious/test_builder_amazon.py::TestAmazonInstanceBuilder::test_required_fields_missing PASSED             [  1%]
packerlicious/test_builder_amazon.py::TestAmazonInstanceBuilder::test_exactly_one_source_ami PASSED              [  2%]
packerlicious/test_builder_amazon.py::TestAmazonInstanceBuilder::test_mutually_exclusive_security_group_ami PASSED [  3%]
packerlicious/test_builder_amazon.py::TestAmazonInstanceBuilder::test_mutually_exclusive_subnet_id PASSED        [  3%]
packerlicious/test_builder_amazon.py::TestAmazonInstanceBuilder::test_mutually_exclusive_vpc_id PASSED           [  4%]
packerlicious/test_builder_amazon.py::TestAmazonEbsBuilder::test_required_fields_missing PASSED                  [  4%]
packerlicious/test_builder_amazon.py::TestAmazonEbsBuilder::test_exactly_one_source_ami PASSED                   [  5%]
packerlicious/test_builder_amazon.py::TestAmazonEbsBuilder::test_mutually_exclusive_security_group_id PASSED     [  6%]
packerlicious/test_builder_amazon.py::TestAmazonEbsBuilder::test_mutually_exclusive_subnet_id PASSED             [  6%]
packerlicious/test_builder_amazon.py::TestAmazonEbsBuilder::test_mutually_exclusive_vpc_id PASSED                [  7%]
packerlicious/test_builder_amazon.py::TestAmazonEbsSurrogate::test_required_fields_missing PASSED                [  7%]
packerlicious/test_builder_amazon.py::TestAmazonEbsSurrogate::test_exactly_one_source_ami PASSED                 [  8%]
packerlicious/test_builder_amazon.py::TestAmazonEbsSurrogate::test_mutually_exclusive_security_group_ami PASSED  [  9%]
packerlicious/test_builder_amazon.py::TestAmazonEbsSurrogate::test_mutually_exclusive_subnet_id PASSED           [  9%]
packerlicious/test_builder_amazon.py::TestAmazonEbsSurrogate::test_mutually_exclusive_vpc_id PASSED              [ 10%]
packerlicious/test_builder_amazon.py::TestAmazonEbsVolume::test_required_fields_missing PASSED                   [ 10%]
packerlicious/test_builder_amazon.py::TestAmazonEbsVolume::test_exactly_one_source_ami PASSED                    [ 11%]
packerlicious/test_builder_amazon.py::TestAmazonEbsVolume::test_mutually_exclusive_security_group_ami PASSED     [ 12%]
packerlicious/test_builder_amazon.py::TestAmazonEbsVolume::test_mutually_exclusive_subnet_id PASSED              [ 12%]
packerlicious/test_builder_amazon.py::TestAmazonEbsVolume::test_mutually_exclusive_vpc_id PASSED                 [ 13%]
packerlicious/test_builder_amazon.py::TestAmazonChroot::test_required_fields_missing PASSED                      [ 13%]
packerlicious/test_builder_amazon.py::TestAmazonChroot::test_exactly_one_source_ami PASSED                       [ 14%]
packerlicious/test_builder_amazon.py::TestAmazonChroot::test_exactly_one_source_ami_when_from_scratch_is_false PASSED [ 15%]
packerlicious/test_builder_amazon.py::TestAmazonChroot::test_required_fields_when_from_scratch_is_true PASSED    [ 15%]
packerlicious/test_builder_azure.py::TestAzureBuilder::test_required_fields_missing PASSED                       [ 16%]
packerlicious/test_builder_cloudstack.py::TestCloudStackBuilder::test_required_fields_missing PASSED             [ 16%]
packerlicious/test_builder_cloudstack.py::TestCloudStackBuilder::test_validate PASSED                            [ 17%]
packerlicious/test_builder_digitalocean.py::TestDigitalOceanBuilder::test_required_fields_missing PASSED         [ 18%]
packerlicious/test_builder_docker.py::TestDockerBuilder::test_required_fields_missing PASSED                     [ 18%]
packerlicious/test_builder_docker.py::TestDockerBuilder::test_source_and_content_mutually_exclusive PASSED       [ 19%]
packerlicious/test_builder_file.py::TestFileBuilder::test_required_fields_missing PASSED                         [ 20%]
packerlicious/test_builder_file.py::TestFileBuilder::test_source_and_content_mutually_exclusive PASSED           [ 20%]
packerlicious/test_builder_file.py::TestFileBuilder::test_source_and_content_missing PASSED                      [ 21%]
packerlicious/test_builder_googlecompute.py::TestGoogleComputeBuilder::test_required_fields_missing PASSED       [ 21%]
packerlicious/test_builder_googlecompute.py::TestGoogleComputeBuilder::test_validate PASSED                      [ 22%]
packerlicious/test_builder_hetznercloud.py::TestHetznerCloudBuilder::test_required_fields_missing PASSED         [ 23%]
packerlicious/test_builder_hyperone.py::TestHyperOneBuilder::test_no_required_fields PASSED                      [ 23%]
packerlicious/test_builder_hyperv.py::TestHypervIsoBuilder::test_required_fields_missing PASSED                  [ 24%]
packerlicious/test_builder_hyperv.py::TestHypervVmcxBuilder::test_required_fields_missing PASSED                 [ 24%]
packerlicious/test_builder_hyperv.py::TestHypervVmcxBuilder::test_exactly_one_clone_from_required PASSED         [ 25%]
packerlicious/test_builder_hyperv.py::TestHypervVmcxBuilder::test_exactly_one_clone_from_specified PASSED        [ 26%]
packerlicious/test_builder_linode.py::TestLinodeBuilder::test_required_fields_missing PASSED                     [ 26%]
packerlicious/test_builder_lxc.py::TestLxcBuilder::test_required_fields_missing PASSED                           [ 27%]
packerlicious/test_builder_lxd.py::TestLxdBuilder::test_required_fields_missing PASSED                           [ 27%]
packerlicious/test_builder_navercloud.py::TestNaverCloudBuilder::test_required_fields_missing PASSED             [ 28%]
packerlicious/test_builder_null.py::TestNullBuilder::test_no_required_fields PASSED                              [ 29%]
packerlicious/test_builder_oneandone.py::TestOneAndOneBuilder::test_required_fields_missing PASSED               [ 29%]
packerlicious/test_builder_openstack.py::TestOpenStackBuilder::test_required_fields_missing PASSED               [ 30%]
packerlicious/test_builder_openstack.py::TestOpenStackBuilder::test_validate PASSED                              [ 30%]
packerlicious/test_builder_oracle.py::TestOracleClassicBuilder::test_required_fields_missing PASSED              [ 31%]
packerlicious/test_builder_oracle.py::TestOracleOCIBuilder::test_required_fields_missing PASSED                  [ 32%]
packerlicious/test_builder_parallels.py::TestParallelsIsoBuilder::test_required_fields_missing PASSED            [ 32%]
packerlicious/test_builder_parallels.py::TestParallelsIsoBuilder::test_iso_checksum_mutually_exclusive PASSED    [ 33%]
packerlicious/test_builder_parallels.py::TestParallelsPvmBuilder::test_required_fields_missing PASSED            [ 33%]
packerlicious/test_builder_profitbricks.py::TestProfitBricksBuilder::test_required_fields_missing PASSED         [ 34%]
packerlicious/test_builder_proxmox.py::TestProxmoxBuilder::test_required_fields_missing PASSED                   [ 35%]
packerlicious/test_builder_qemu.py::TestQemuBuilder::test_required_fields_missing PASSED                         [ 35%]
packerlicious/test_builder_qemu.py::TestQemuBuilder::test_iso_checksum_mutually_exclusive PASSED                 [ 36%]
packerlicious/test_builder_scaleway.py::TestScalewayBuilder::test_required_fields_missing PASSED                 [ 36%]
packerlicious/test_builder_tencentcloud.py::TestTencentCloudCvmBuilder::test_required_fields_missing PASSED      [ 37%]
packerlicious/test_builder_triton.py::TestTritonBuilder::test_required_fields_missing PASSED                     [ 38%]
packerlicious/test_builder_triton.py::TestTritonBuilder::test_validate PASSED                                    [ 38%]
packerlicious/test_builder_vagrant.py::TestVagrantBuilder::test_required_fields_missing PASSED                   [ 39%]
packerlicious/test_builder_vagrant.py::TestVagrantBuilder::test_source_path_global_id_mutually_exclusive PASSED  [ 40%]
packerlicious/test_builder_virtualbox.py::TestVirtualBoxOvfBuilder::test_required_fields_missing PASSED          [ 40%]
packerlicious/test_builder_virtualbox.py::TestVirtualBoxIsoBuilder::test_required_fields_missing PASSED          [ 41%]
packerlicious/test_builder_virtualbox.py::TestVirtualBoxIsoBuilder::test_iso_checksum_mutually_exclusive PASSED  [ 41%]
packerlicious/test_builder_vmware.py::TestVMwareIsoBuilder::test_required_fields_missing PASSED                  [ 42%]
packerlicious/test_builder_vmware.py::TestVMwareIsoBuilder::test_iso_checksum_mutually_exclusive PASSED          [ 43%]
packerlicious/test_builder_vmware.py::TestVMwareVmxBuilder::test_required_fields_missing PASSED                  [ 43%]
packerlicious/test_builder_yandex.py::TestYandexBuilder::test_no_required_fields PASSED                          [ 44%]
packerlicious/test_community_builder_arm.py::TestCommunityArmImageBuilder::test_required_fields_missing PASSED   [ 44%]
packerlicious/test_community_provisioner_inspec.py::TestCommunityInspecProvisioner::test_required_fields_missing PASSED [ 45%]
packerlicious/test_examples.py::TestFileExamples::test_examples[/Users/dennis.hedlund/Documents/packerlicious/tests/packerlicious/../../examples/getting_started_example.py] PASSED [ 46%]
packerlicious/test_post_processor_alicloud.py::TestAliCloudImportPostProcessor::test_required_fields PASSED      [ 46%]
packerlicious/test_post_processor_amazon.py::TestAmazonImportPostProcessor::test_required_fields PASSED          [ 47%]
packerlicious/test_post_processor_artifice.py::TestArtificePostProcessor::test_required_fields PASSED            [ 47%]
packerlicious/test_post_processor_atlas.py::TestAtlasPostProcessor::test_required_fields PASSED                  [ 48%]
packerlicious/test_post_processor_checksum.py::TestChecksumPostProcessor::test_no_required_fields PASSED         [ 49%]
packerlicious/test_post_processor_checksum.py::TestChecksumPostProcessor::test_checksum_types_valid PASSED       [ 49%]
packerlicious/test_post_processor_checksum.py::TestChecksumPostProcessor::test_checksum_types_invalid PASSED     [ 50%]
packerlicious/test_post_processor_compress.py::TestChecksumPostProcessor::test_no_required_fields PASSED         [ 50%]
packerlicious/test_post_processor_digital_ocean.py::TestDigitalOceanImporterPostProcessor::test_required_fields_missing PASSED [ 51%]
packerlicious/test_post_processor_docker.py::TestDockerImportPostProcessor::test_required_fields_missing PASSED  [ 52%]
packerlicious/test_post_processor_docker.py::TestDockerPushPostProcessor::test_no_required_fields PASSED         [ 52%]
packerlicious/test_post_processor_docker.py::TestDockerSavePostProcessor::test_required_fields_missing PASSED    [ 53%]
packerlicious/test_post_processor_docker.py::TestDockerTagPostProcessor::test_required_fields_missing PASSED     [ 53%]
packerlicious/test_post_processor_google_compute.py::TestGoogleComputeImportPostProcessor::test_required_fields_missing PASSED [ 54%]
packerlicious/test_post_processor_google_compute.py::TestGoogleComputeExportPostProcessor::test_required_fields_missing PASSED [ 55%]
packerlicious/test_post_processor_manifest.py::TestManifestPostProcessor::test_no_required_fields PASSED         [ 55%]
packerlicious/test_post_processor_manifest.py::TestManifestPostProcessor::test_custom_data PASSED                [ 56%]
packerlicious/test_post_processor_shell_local.py::TestShellLocalPostProcessor::test_required_fields PASSED       [ 56%]
packerlicious/test_post_processor_vagrant.py::TestVagrantPostProcessor::test_no_required_fields PASSED           [ 57%]
packerlicious/test_post_processor_vagrant.py::TestVagrantCloudPostProcessor::test_required_fields_missing PASSED [ 58%]
packerlicious/test_post_processor_vsphere.py::TestVSphereProcessor::test_required_fields_missing PASSED          [ 58%]
packerlicious/test_post_processor_vsphere_template.py::TestVSphereTemplateProcessor::test_required_fields_missing PASSED [ 59%]
packerlicious/test_project_infrastructure.py::test_travis_with_coveralls PASSED                                  [ 60%]
packerlicious/test_provisioner.py::TestProvisionerAttributes::test_support_only PASSED                           [ 60%]
packerlicious/test_provisioner.py::TestProvisionerAttributes::test_support_pause_before PASSED                   [ 61%]
packerlicious/test_provisioner_ansible.py::TestAnsibleLocalProvisioner::test_required_fields_missing PASSED      [ 61%]
packerlicious/test_provisioner_ansible.py::TestAnsibleProvisioner::test_required_fields_missing PASSED           [ 62%]
packerlicious/test_provisioner_breakpoint.py::TestBreakpointProvisioner::test_no_required_fields PASSED          [ 63%]
packerlicious/test_provisioner_chef.py::TestChefClientProvisioner::test_no_required_fields PASSED                [ 63%]
packerlicious/test_provisioner_chef.py::TestChefSoloProvisioner::test_no_required_fields PASSED                  [ 64%]
packerlicious/test_provisioner_converge.py::TestConvergeProvisioner::test_required_fields_missing PASSED         [ 64%]
packerlicious/test_provisioner_converge.py::TestConvergeProvisioner::test_module_directory_required_fields_missing PASSED [ 65%]
packerlicious/test_provisioner_file.py::TestFileProvisioner::test_required_fields_missing PASSED                 [ 66%]
packerlicious/test_provisioner_file.py::TestFileProvisioner::test_direction_valid[download] PASSED               [ 66%]
packerlicious/test_provisioner_file.py::TestFileProvisioner::test_direction_valid[upload] PASSED                 [ 67%]
packerlicious/test_provisioner_file.py::TestFileProvisioner::test_direction_invalid PASSED                       [ 67%]
packerlicious/test_provisioner_inspec.py::TestInspecProvisioner::test_required_fields_missing PASSED             [ 68%]
packerlicious/test_provisioner_inspec.py::TestInspecProvisioner::test_property_attributes_renders PASSED         [ 69%]
packerlicious/test_provisioner_packer.py::TestPuppetMasterlessProvisioner::test_required_fields_missing PASSED   [ 69%]
packerlicious/test_provisioner_packer.py::TestPuppetServerProvisioner::test_no_required_fields PASSED            [ 70%]
packerlicious/test_provisioner_powershell.py::TestPowerShellProvisioner::test_required_fields_missing PASSED     [ 70%]
packerlicious/test_provisioner_salt.py::TestSaltMasterlessProvisioner::test_required_fields_missing PASSED       [ 71%]
packerlicious/test_provisioner_salt.py::TestSaltMasterlessProvisioner::test_warning_minion_config PASSED         [ 72%]
packerlicious/test_provisioner_salt.py::TestSaltMasterlessProvisioner::test_no_warning_minion_config PASSED      [ 72%]
packerlicious/test_provisioner_shell.py::TestShellProvisioner::test_required_fields_missing PASSED               [ 73%]
packerlicious/test_provisioner_shell.py::TestShellLocalProvisioner::test_required_fields_missing PASSED          [ 73%]
packerlicious/test_provisioner_windows.py::TestWindowsShellProvisioner::test_required_fields_missing PASSED      [ 74%]
packerlicious/test_provisioner_windows.py::TestWindowsRestartProvisioner::test_no_required_fields PASSED         [ 75%]
packerlicious/test_template.py::TestPackerTemplate::test_template PASSED                                         [ 75%]
packerlicious/test_template.py::TestPackerTemplate::test_template_variables PASSED                               [ 76%]
packerlicious/test_template.py::TestPackerTemplate::test_template_builders PASSED                                [ 76%]
packerlicious/test_template.py::TestPackerTemplate::test_template_provisioners PASSED                            [ 77%]
packerlicious/test_template.py::TestPackerTemplate::test_template_post_processors PASSED                         [ 78%]
packerlicious/test_template.py::TestPackerTemplate::test_variable_duplicate_entries PASSED                       [ 78%]
packerlicious/test_template.py::TestPackerTemplate::test_variable_no_duplicate_entries PASSED                    [ 79%]
packerlicious/test_template.py::TestPackerTemplate::test_sensitve_variables PASSED                               [ 80%]
packerlicious/test_template.py::TestPackerTemplate::test_jagged_array_render PASSED                              [ 80%]
packerlicious/test_validator.py::TestValidator::test_boolean PASSED                                              [ 81%]
packerlicious/test_validator.py::TestValidator::test_integer PASSED                                              [ 81%]
packerlicious/test_validator.py::TestValidator::test_integer_range PASSED                                        [ 82%]
packerlicious/test_validator.py::TestValidator::test_network_port PASSED                                         [ 83%]
packerlicious/test_validator.py::TestValidator::test_network_port_ref PASSED                                     [ 83%]
packerlicious/test_validator.py::TestValidator::test_tg_healthcheck_port PASSED                                  [ 84%]
packerlicious/test_validator.py::TestValidator::test_tg_healthcheck_port_ref PASSED                              [ 84%]
packerlicious/test_validator.py::TestValidator::test_s3_bucket_name PASSED                                       [ 85%]
packerlicious/test_validator.py::TestValidator::test_elb_name PASSED                                             [ 86%]
packerlicious/test_validator.py::TestValidator::test_encoding PASSED                                             [ 86%]
packerlicious/test_validator.py::TestValidator::test_status PASSED                                               [ 87%]
packerlicious/test_validator.py::TestValidator::test_iam_names PASSED                                            [ 87%]
packerlicious/test_validator.py::TestValidator::test_iam_path PASSED                                             [ 88%]
packerlicious/test_validator.py::TestValidator::test_iam_role_name PASSED                                        [ 89%]
packerlicious/test_validator.py::TestValidator::test_iam_group_name PASSED                                       [ 89%]
packerlicious/test_validator.py::TestValidator::test_iam_user_name PASSED                                        [ 90%]
packerlicious/test_validator.py::TestValidator::test_all_or_nothing PASSED                                       [ 90%]
packerlicious/test_validator.py::TestValidator::test_mutually_exclusive PASSED                                   [ 91%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_failure[a_string_value] PASSED                 [ 92%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_failure[1234] PASSED                           [ 92%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_failure[test_value2] PASSED                    [ 93%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_failure[test_value3] PASSED                    [ 93%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_failure[test_value4] PASSED                    [ 94%]
packerlicious/test_validator.py::TestValidator::test_jagged_array_success[test_value0] PASSED                    [ 95%]
packerlicious/test_variables.py::TestPackerVariables::test_environment_variable PASSED                           [ 95%]
packerlicious/test_variables.py::TestPackerVariables::test_environment_variable_name_specified PASSED            [ 96%]
packerlicious/test_variables.py::TestPackerVariables::test_template_variable PASSED                              [ 96%]
packerlicious/test_variables.py::TestPackerVariables::test_user_variable PASSED                                  [ 97%]
packerlicious/test_variables.py::TestPackerVariables::test_packer_variable PASSED                                [ 98%]
packerlicious/community/test_builder_vsphere.py::TestVSphereIsoBuilder::test_required_fields_missing PASSED      [ 98%]
packerlicious/community/test_builder_vsphere.py::TestVSphereIsoBuilder::test_iso_checksum_mutually_exclusive PASSED [ 99%]
packerlicious/community/test_builder_vsphere.py::TestVSphereCloneBuilder::test_required_fields_missing PASSED    [100%]

Dennis Vinterfjärd added 2 commits October 2, 2020 12:26
Fixed ami_root_device expected map found slice
b945a31 
Packer expects ami_root_device to be map of values and not slice,
expecting a list of BlockDeviceMapping forces the user to do workarounds
like populate the ami_root_device with slice, pop it of and set it to be
map afterwards.
Removed invalid parameter in AmazonEbsSurrogate
00151c3 
AmazonEbsSurrogate was expecting prop source_device_name, running packer
validate on output generated with source_device_name configured in
AmazonEbsSurrogate returns unknown configuration key:
"source_device_name"

source_device_name is not a valid parameter for amazon ebs builder
according to https://www.packer.io/docs/builders/amazon-ebs
@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 00151c3 on Pixxle:master into 0d91aa3 on mayn:develop.

@Pixxle
Copy link
Author

Pixxle commented Oct 2, 2020

Appveyor python3.4 build failed due to missing dependency (typing library)

@mayn
Copy link
Owner

mayn commented Oct 2, 2020

thanks will take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Pixxle @coveralls @mayn