route53 module fails with TypeError: a bytes-like object is required, not 'str' #313

t-readyroc · 2020-12-02T14:50:46Z

SUMMARY

This issue has been reported elsewhere, but is 2yrs old now & I'm hoping that the maintainers here can possibly help get the fix merged. The issue was reported in 2018, here, and there is a patch available, which actually fixes the issue, here, but the patch is only a branch, which is now behind master by several minor versions.

I actually attempted to run the playbook without boto, using only boto3, but that also failed ("boto required for this module"). As a side not, your README states that only boto3, not boto is required.

Anyway, I'm sure this report will be closed, but I wanted to make the maintainers aware that it's something that's affecting the users of your module, and am hoping that you can help to get the fix merged.

ISSUE TYPE

Bug Report

COMPONENT NAME

route53

ANSIBLE VERSION

ansible 2.10.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/organization_virtualenvs/sales_eng/lib64/python3.6/site-packages/ansible
  executable location = /opt/organization_virtualenvs/sales_eng/bin/ansible
  python version = 3.6.8 (default, Dec  5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

CONFIGURATION

Empty output

OS / ENVIRONMENT

Red Hat 8.3.1. Ansible running on Tower 3.7.3

STEPS TO REPRODUCE

- name: create records
  community.aws.route53:
    aws_access_key: "{{ svc_route53_key }}"
    aws_secret_key: "{{ svc_route53_secret }}"
    zone: "{{ new_record_zone }}"
    record: "{{ new_record_name }}.{{ new_record_zone }}"
    type: "{{ new_record_type }}"
    ttl: "{{ new_record_ttl }}"
    value: "{{ new_record_value }}"
    state: present
  when: 
    - new_record_zone is defined
    - new_record_name is defined
    - new_record_value not in existing_records
  tags: new_records

EXPECTED RESULTS

Records created

ACTUAL RESULTS

Type Error from subject

{
    "module_stdout": "",
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 708, in <module>\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 601, in main\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 393, in get_zone_id_by_name\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 453, in invoke_with_throttling_retries\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 559, in get_zones\n    zones = self.get_all_hosted_zones()\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 120, in get_all_hosted_zones\n    params=params)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 103, in make_request\n    retry_handler=self._retry_handler)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 1071, in make_request\n    retry_handler=retry_handler)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 913, in _mexe\n    self.is_secure)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 705, in get_http_connection\n    return self.new_http_connection(host, port, is_secure)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 747, in new_http_connection\n    connection = self.proxy_ssl(host, is_secure and 443 or 80)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 796, in proxy_ssl\n    sock.sendall(\"CONNECT %s HTTP/1.0\\r\\n\" % host)\nTypeError: a bytes-like object is required, not 'str'\n",
    "exception": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1606918842.2402813-12-124109579850519/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 708, in <module>\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 601, in main\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 393, in get_zone_id_by_name\n  File \"/tmp/ansible_community.aws.route53_payload_b740_vmh/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 453, in invoke_with_throttling_retries\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 559, in get_zones\n    zones = self.get_all_hosted_zones()\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 120, in get_all_hosted_zones\n    params=params)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/route53/connection.py\", line 103, in make_request\n    retry_handler=self._retry_handler)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 1071, in make_request\n    retry_handler=retry_handler)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 913, in _mexe\n    self.is_secure)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 705, in get_http_connection\n    return self.new_http_connection(host, port, is_secure)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 747, in new_http_connection\n    connection = self.proxy_ssl(host, is_secure and 443 or 80)\n  File \"/opt/organization_virtualenvs/sales_eng/lib/python3.6/site-packages/boto/connection.py\", line 796, in proxy_ssl\n    sock.sendall(\"CONNECT %s HTTP/1.0\\r\\n\" % host)\nTypeError: a bytes-like object is required, not 'str'\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1,
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "_ansible_no_log": false,
    "changed": false
}

The text was updated successfully, but these errors were encountered:

ansibullbot · 2021-01-13T00:26:55Z

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

gravesm · 2021-03-05T20:26:02Z

@t-readyroc thank you for raising this issue. the route53 module was recently refactored in #405. These changes should be in collection version 1.4. Could you try that out and let us know if that fixes your issue?

ansibullbot · 2021-04-20T09:39:32Z

Files identified in the description:

[plugins/modules/route53.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/route53.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2021-04-20T09:39:33Z

cc @jillr @jimbydamonk @s-hertel @tremble @wimnat
click here for bot help

BitPan · 2021-05-04T07:14:09Z

Still getting same error after upgrading to 1.5.0 version(with ansible 2.10.7 and python 3.6.12)

The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/local/lib/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 708, in <module>
  File "/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 601, in main
  File "/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 393, in get_zone_id_by_name
  File "/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 453, in invoke_with_throttling_retries
  File "/usr/local/lib/python3.6/site-packages/boto/route53/connection.py", line 559, in get_zones
    zones = self.get_all_hosted_zones()
  File "/usr/local/lib/python3.6/site-packages/boto/route53/connection.py", line 120, in get_all_hosted_zones
    params=params)
  File "/usr/local/lib/python3.6/site-packages/boto/route53/connection.py", line 103, in make_request
    retry_handler=self._retry_handler)
  File "/usr/local/lib/python3.6/site-packages/boto/connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "/usr/local/lib/python3.6/site-packages/boto/connection.py", line 913, in _mexe
    self.is_secure)
  File "/usr/local/lib/python3.6/site-packages/boto/connection.py", line 705, in get_http_connection
    return self.new_http_connection(host, port, is_secure)
  File "/usr/local/lib/python3.6/site-packages/boto/connection.py", line 747, in new_http_connection
    connection = self.proxy_ssl(host, is_secure and 443 or 80)
  File "/usr/local/lib/python3.6/site-packages/boto/connection.py", line 796, in proxy_ssl
    sock.sendall("CONNECT %s HTTP/1.0\r\n" % host)
TypeError: a bytes-like object is required, not 'str'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1620112377.2238576-2632-124122034154179/AnsiballZ_route53.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/local/lib/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/local/lib/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/local/lib/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 708, in <module>\n  File \"/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 601, in main\n  File \"/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 393, in get_zone_id_by_name\n  File \"/tmp/ansible_community.aws.route53_payload_yc8g0_60/ansible_community.aws.route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py\", line 453, in invoke_with_throttling_retries\n  File \"/usr/local/lib/python3.6/site-packages/boto/route53/connection.py\", line 559, in get_zones\n    zones = self.get_all_hosted_zones()\n  File \"/usr/local/lib/python3.6/site-packages/boto/route53/connection.py\", line 120, in get_all_hosted_zones\n    params=params)\n  File \"/usr/local/lib/python3.6/site-packages/boto/route53/connection.py\", line 103, in make_request\n    retry_handler=self._retry_handler)\n  File \"/usr/local/lib/python3.6/site-packages/boto/connection.py\", line 1071, in make_request\n    retry_handler=retry_handler)\n  File \"/usr/local/lib/python3.6/site-packages/boto/connection.py\", line 913, in _mexe\n    self.is_secure)\n  File \"/usr/local/lib/python3.6/site-packages/boto/connection.py\", line 705, in get_http_connection\n    return self.new_http_connection(host, port, is_secure)\n  File \"/usr/local/lib/python3.6/site-packages/boto/connection.py\", line 747, in new_http_connection\n    connection = self.proxy_ssl(host, is_secure and 443 or 80)\n  File \"/usr/local/lib/python3.6/site-packages/boto/connection.py\", line 796, in proxy_ssl\n    sock.sendall(\"CONNECT %s HTTP/1.0\\r\\n\" % host)\nTypeError: a bytes-like object is required, not 'str'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

tremble · 2021-05-04T07:37:10Z

The traceback includes calls to the original boto rather than boto3/botocore

"site-packages/boto/route53/connection.py"

This implies you're still using the version from 1.3.0 or earlier. Please would you double check that you don't have an old copy of the collection installed.

BitPan · 2021-05-04T09:14:53Z

It worked after explicitly specifying the collection in the requirements.yml file. It seems that the collection community.aws 1.3.0 module is built into the ansible 2.10.7 installable?

I could see v1.3.0 is installed even without specifying the community.aws in my requirements.yml file.

root@5f1e3ceff3fb:/opt/code# ansible-galaxy collection list | grep aws
amazon.aws        1.5.0
amazon.aws                1.4.0
**community.aws             1.3.0**
netapp.aws                20.9.0

Fix:

  - name: community.aws
    version: 1.5.0

Thanks @tremble for your help!

tremble · 2021-05-04T09:27:33Z

@BitPan Thanks for the confirmation here.

There are a number of boto (rather than boto3) based modules which remain and will likely see a similar issue. Migrating all of them to boto3 is something we'd like to see, however it's a non-trival amount of work.

Since this is ultimately a bug in the underlying library rather than our consumption of the library I'm going to close this issue.

* fix public_access example

goneri added the needs_triage label Dec 18, 2020

ansibullbot added bug This issue/PR relates to a bug python3 traceback labels Jan 13, 2021

gravesm removed the needs_triage label Mar 5, 2021

rituparnop mentioned this issue Apr 20, 2021

module fails with TypeError: a bytes-like object is required, not 'str' ansible-collections/amazon.aws#332

Closed

ansibullbot added the module module label Apr 20, 2021

tremble closed this as completed May 4, 2021

alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this issue May 25, 2022

s3_bucket - (docs) fix public_access example (ansible-collections#313)

b2e478a

* fix public_access example

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

route53 module fails with TypeError: a bytes-like object is required, not 'str' #313

route53 module fails with TypeError: a bytes-like object is required, not 'str' #313

t-readyroc commented Dec 2, 2020 •

edited by tremble

Loading

ansibullbot commented Jan 13, 2021

gravesm commented Mar 5, 2021

ansibullbot commented Apr 20, 2021

ansibullbot commented Apr 20, 2021

BitPan commented May 4, 2021

tremble commented May 4, 2021

BitPan commented May 4, 2021 •

edited

Loading

tremble commented May 4, 2021

route53 module fails with TypeError: a bytes-like object is required, not 'str' #313

route53 module fails with TypeError: a bytes-like object is required, not 'str' #313

Comments

t-readyroc commented Dec 2, 2020 • edited by tremble Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

ansibullbot commented Jan 13, 2021

gravesm commented Mar 5, 2021

ansibullbot commented Apr 20, 2021

ansibullbot commented Apr 20, 2021

BitPan commented May 4, 2021

tremble commented May 4, 2021

BitPan commented May 4, 2021 • edited Loading

tremble commented May 4, 2021

t-readyroc commented Dec 2, 2020 •

edited by tremble

Loading

BitPan commented May 4, 2021 •

edited

Loading