MAC OSX Monterey build failed with gnutls 3.7.3

[pkarashchenko]

Run ../configure --target-list=xtensa-softmmu --enable-debug --enable-sanitizers --disable-strip --disable-capstone --disable-vnc with macOS Monterey.
Crypto config:

  Crypto
    TLS priority                 : "NORMAL"
    GNUTLS support               : YES 3.7.3
      GNUTLS crypto              : YES
    libgcrypt                    : NO
    nettle                       : NO
    crypto afalg                 : NO
    rng-none                     : NO
    Linux keyring                : NO

Run make vga=no -j$(nproc). Make ends with:

Undefined symbols for architecture x86_64:
  "_gcry_mpi_invm", referenced from:
      _esp32_rsa_mod_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_mul", referenced from:
      _esp32_rsa_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_mulm", referenced from:
      _esp32_rsa_mod_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_new", referenced from:
      _esp32_rsa_exp_mod in hw_misc_esp32_rsa.c.o
      _esp32_rsa_mod_mul_op in hw_misc_esp32_rsa.c.o
      _esp32_rsa_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_powm", referenced from:
      _esp32_rsa_exp_mod in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_print", referenced from:
      _mpi_gcrypt_to_block in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_release", referenced from:
      _esp32_rsa_exp_mod in hw_misc_esp32_rsa.c.o
      _esp32_rsa_mod_mul_op in hw_misc_esp32_rsa.c.o
      _esp32_rsa_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_scan", referenced from:
      _mpi_block_to_gcrypt in hw_misc_esp32_rsa.c.o
  "_gcry_mpi_set_bit", referenced from:
      _esp32_rsa_mod_mul_op in hw_misc_esp32_rsa.c.o
  "_gcry_strerror", referenced from:
      _mpi_block_to_gcrypt in hw_misc_esp32_rsa.c.o
      _mpi_gcrypt_to_block in hw_misc_esp32_rsa.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
make: *** [run-ninja] Error 1

Currently compiled with workaround: ../configure --target-list=xtensa-softmmu --enable-debug --enable-sanitizers --disable-strip --disable-capstone --disable-vnc --enable-gcrypt.

[igrr]

--enable-gcrypt flag is now mentioned in the wiki: https://github.com/espressif/qemu/wiki#configure

I'm not sure what should be the behavior if libgcrypt isn't available. We could disable emulation of the cryptographic peripherals or we could fail earlier with a clear error message that libgcrypt is required when compiling with ESP32 support.

[pkarashchenko]

The case is that currently build system selects one of GNU TLS, nettle or libcrypt. I think the solution can be to migrate to some lib agnostic APIs or make a selection of crypto APIs that are called based on crypto library selected. What do you think?

[igrr]

QEMU does provide some library-agnostic crypto APIs which can work with either of those libraries (see include/crypto directory). However for emulation of ESP crypto peripherals these functions aren't sufficient, so we decided to use libgcrypt directly.

It's possible to propose some additional library-agnostic crypto APIs but then we would have to:

• implement these APIs for all 3 libraries
• make sure emulation works correctly with all 3 libraries
• maintain a much larger patch set (ESP peripherals + changes to library-agnostic crypto part)

That's quite a lot of extra work which we aren't likely to do near term.

[pkarashchenko]

Ok. It is fine that --enable-gcrypt flag is now mentioned in the wiki. Let's close the issue.