Key format PKCS8 and RSA Key error with Java clients

[Angelote2022]

Vault version: 1.11.3
Java version: openjdk 11.0.13

Hello, I have an issue with PKCS8 key formating and PEM for my Kafka clients. If I convert the resulting Vault PEM file to pk12 and import into a JKS file kafka clients can connect without any issues. Apparently this issue was solved in the version 0.9.0.1

• secret/pki: Private keys can now be marshalled as PKCS#8 [Add PKCS8 marshaling to PKI #3518]

vault write -field="certificate" kafka-int-ca/issue/kafka-client common_name="producer.clients.kafka.acme.com" format="pem_bundle" private_key_fo rmat="pkcs8" > pkcs8.pem

But it definetely gives me a java libcrypto error:

Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs Caused by: java.io.IOException: overrun, bytes = 924 at java.base/javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:95) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:512) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:462) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:435) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.load(DefaultSslEngineFactory.java:412) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:349) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.<init>(DefaultSslEngineFactory.java:405) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:293) at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161) at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:140) at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:97) at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:73) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192) at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105) at org.apache.kafka.clients.producer.KafkaProducer.newSender(KafkaProducer.java:450) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:431) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:292) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:319) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:304) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:45) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)

If I extract the private key from the pem_bundle and convert to PKCS8 and concatenate the root and int certs it works flawless:

openssl pkcs8 \ -topk8 \ -in producer.key \ -inform pem \ -v1 PBE-SHA1-RC4-128 \ -out pkcs8.key \ -outform pem \ -passin pass:changeit \ -passout pass:changeit

[stevendpclark]

Hello @Angelote2022!

Thanks for the report for the pkcs8 encoding issue. We've looked at it and while the functionality is working as we expect the documentation wasn't as clear as it should have been. I've updated it within PR#17080 to make things clearer hopefully.

Basically the private_key_format argument solely applies to the private_key response field not to the private key within a PEM bundle as changing that might break expectations/backwards compatibility, so we are reluctant to change it.

Closing this issue for now.