proxmox_template 'Connection aborted.'

[einsibjarni]

Summary

Proxmox cluster, all nodes running Proxmox 7.2-11, installed on Debian Bullseye

We have a playbook that uploads a custom ISO file using proxmox_template module and creates a new VM. We don't run it that often, yesterday we discovered it not working. The error from the task is:

fatal: [ht-dev-eh01.stg.isnic.is -> localhost]: FAILED! => changed=false 
  msg: 'uploading/downloading of template deploy-ht-dev-eh01.stg.isnic.is.iso failed with exception: (''Connection aborted.'', ConnectionResetError(54, ''Connection reset by peer''))'

This line appears in syslog on the Proxmox node:

pveproxy[1534092]: problem with client 2001:67c:6c:xxxx; No space left on device

There's plenty of space on all drives, both on the ansible controller and on the proxmox host. I've tracked the error to AnyEvent https://metacpan.org/release/MLEHMANN/AnyEvent-7.17/source/lib/AnyEvent/Handle.pm#L1360

It seems AnyEvent receives more than rbuf_max in it's read buffer, so it emits the confusing Errno::ENOSPC error.
Uploading ISO through Proxmox web UI works with the same user/pass as used in ansible playbook.

Issue Type

Bug Report

Component Name

proxmox_template

Ansible Version

$ ansible --version
ansible [core 2.13.2]
  config file = /srv/devops/ansible/ansible.cfg
  configured module search path = ['/home/einar/.ansible/plugins/modules', '/usr/local/share/py39-ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/einar/.ansible/collections:/usr/local/share/py39-ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.15 (main, Nov 14 2022, 06:06:09) [Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a
  jinja version = 3.1.2
  libyaml = True

Community.general Version

$ ansible-galaxy collection list community.general
# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 5.3.0

Configuration

$ ansible-config dump --only-changed

OS / Environment

Ansible controller is running FreeBSD 13.1-RELEASE p4
Remote host is running Proxmox 7.2-11 on Debian bullseye

Steps to Reproduce

- name: "Upload generated deployment ISO to Proxmox (always returns 400 due to proxmox bug, failure ignored)"
  proxmox_template:
    api_host: "{{ proxmox_vm_api_name }}"
    api_user: "{{ proxmox_vm_api_user }}"
    api_password: "{{ proxmox_vm_api_pass }}"
    validate_certs: yes
    state: present
    node: "{{ machine_config['node'] }}"
    src: "{{ deployiso_dir }}/iso_generated/deploy-{{ inventory_hostname }}.iso"
    storage: "{{ proxmox_vm_storage_iso }}"
    content_type: "iso"
    force: yes # Overwrite old ISO
  when: proxmox_vm_secrets_available|default(false) and (proxmox_vm_iso_non_standard or config_result['changed'] or isoscript_result['changed'])
  retries: 3 # Proxmox sometimes fails with HTTP 400 error, workaround with retry. See is/nic/devops#13496.
  delay: 3
  ignore_errors: True # UGLY HACK: Proxmox returns 400 errors, but the ISO has been successfully uploaded. Just assume success for now. FIXME: Find out why this is.
  delegate_to: "localhost"

Expected Results

I assumed the ISO would be uploaded to Proxmox template storage

Actual Results

fatal: [ht-dev-eh01.stg.isnic.is -> localhost]: FAILED! => changed=false 
  msg: 'uploading/downloading of template deploy-ht-dev-eh01.stg.isnic.is.iso failed with exception: (''Connection aborted.'', ConnectionResetError(54, ''Connection reset by peer''))'

This line appears in syslog on the Proxmox node:

pveproxy[1534092]: problem with client 2001:67c:6c:xxxx; No space left on device

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/modules/proxmox_template.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @Ajpantuso @Thulium-Drake @UnderGreen @joshainglis @karmab @tleguern
click here for bot help