[OS X] can't connect: error importing private key with OSStatus -25257 / failed to import certificate and private key with error 1038

[coderbyheart]

• I've searched for previous similar issues and didn't find any solution

Known Issue

• I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Platform/OS/Hardware/Device
What are you running the sdk on?

Mac OS X 10.15 on GitHub Actions

Describe the question
When trying to connect using the connection cannot be established. These errors are printed:

Error:  [2021-09-01T11:40:25Z] [0000000110907dc0] [pki-utils] - static: error importing private key with OSStatus -25257
113
Error:  [2021-09-01T11:40:25Z] [0000000110907dc0] [tls-handler] - static: failed to import certificate and private key with error 1038.

This happens in a project where we run the same tests on Ubuntu and Mac OS using GitHub actions. The Ubuntu configuration works.

[bretambrose]

What are the properties of the key? ECC key import has not been implemented (in the SDK) for OsX yet.

[coderbyheart]

Oh, yes. Good point.
The key is an ECC key, generated using this command: openssl ecparam -out device.key -name prime256v1 -genkey

[bretambrose]

Well if you can use an RSA one temporarily, I'll see if we can't get ecc key import bumped in priority. It bugs me that it's not supported but I'd need to get someone else to do it since I don't have a Mac.

[coderbyheart]

For this scenario it's OK. I using the Node.js client to verify that JITP configuration works, so the key type does not matter in that case.

[coderbyheart]

Using an RSA key I see a different error:

   ℹ  IoT (cert) Generating key for device <deviceId> ⏱ +3ms
   ℹ  IoT (cert) openssl genrsa -out device.key 2048 ⏱ +0ms
   ℹ  IoT (cert) Generating CSR for device <deviceId> ⏱ +91ms
   ℹ  IoT (cert) openssl req -new -key device.key -out device.csr -subj /CN=<deviceId> ⏱ +0ms
   ℹ  IoT (cert) Generating certificate for device <deviceId> ⏱ +14ms
   ℹ  IoT (cert) openssl x509 -req -in device.csr -CAkey rootCA.key -CA rootCA.pem -CAcreateserial -out device.pem -days 1 -sha256 ⏱ +0ms
   ℹ  step I connect the tracker "{agpsDevice}" ⏱ +32ms
   ℹ  IoT > connect Connecting <deviceId> to xxx-ats.iot.eu-west-1.amazonaws.com ... ⏱ +4ms

Error:  [2021-09-01T18:04:42Z] [0000700004e25000] [socket] - id=0x7fdee5e8d470 fd=26: connect failed with error code 65.
Error:  [2021-09-01T18:04:42Z] [0000700004e25000] [channel-bootstrap] - id=0x7fdee2c50de0: failed to create socket with error 1049
Error:  [2021-09-01T18:04:43Z] [0000700004e25000] [tls-handler] - id=0x7fdee5cbdd70: error reported during SSLRead. OSStatus code -9805

CrtError: Failed to connect: libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.
Failed to connect.
    at /Users/runner/work/asset-tracker-cloud-aws-js/asset-tracker-cloud-aws-js/node_modules/aws-crt/dist/native/mqtt.js:333:36
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {
  error: 'Failed to connect: libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.',
  error_code: undefined,
  error_name: undefined
}
Failed to connect: libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.

[jmklix]

Can you provide more details about how you have this setup including a code snip-it?

Also a common error that causes UNEXPECTED_HANGUP is when the permissions policy is not set up correctly, so you might want to check that as well.

[coderbyheart]

This is running in a GitHub action, and the same run works for a runner using Linux.

The source is here, and it works with the v1 SDK (which the saga branch of that repo uses).

[coderbyheart]

I can probably create a smaller example, tomorrow that reproduces this problem.

[coderbyheart]

Given that there is a CA certificate, and it is registered for JITP, these are the steps that happen for a new device:

# Key
openssl genrsa -out device.key
# CSR
openssl req -new -key device.key -out device.csr -subj /CN=c43e4843-c950-43a0-975a-88903403fd52
# Certificate
openssl x509 -req -in device.csr -CAkey rootCA.key -CA rootCA.pem -CAcreateserial -out device.pem -days 1 -sha256

const certWithCa = (
    [
        fs.readFileSync(`device.pem`, 'utf-8'),
        fs.readFileSync(`rootCA.pem`, 'utf-8'),
    ]
).join(os.EOL)

const cfg = iot.AwsIotMqttConnectionConfigBuilder.new_mtls_builder(
    fs.readFileSync(`device.key`, 'utf-8'),
    certWithCa,
)
cfg.with_clean_session(true)
cfg.with_client_id(clientId)
cfg.with_endpoint(mqttEndpoint)
const client = new mqtt.MqttClient(clientBootstrap)
const connection = client.new_connection(cfg.build())
connection.on('error', (err) => {
    console.error(err)
    reject(err)
})
connection.on('connect', () => {
    console.log('CONNECTED')
    resolve(connection)
})
connection.connect().catch((err) => {
    console.debug(`Failed to connect.`)
    console.error(err)
})

[bretambrose]

I think concatenating device and root CAs is a feature we are currently missing support for. I believe we have an open internal ticket on the matter.

[TwistedTwigleg]

I just checked this and confirmed that Mac will import ECC keys and certificates to connect to AWS IoT Core. I tried importing a ECC certificate and key pair, and was successfully able to connect using the PubSub MQTT sample. ECC support for Mac was added in October 2022 in this PR, so now there shouldn't be issues with ECC keys on Mac.

I also tested certificate and CA pairs that are combined together using the following code in a slightly modified PubSub sample:

const certWithCa = (
    [
        readFileSync(argv.cert, 'utf-8'),
        readFileSync(argv.ca_file, 'utf-8'),
    ]
).join(EOL)
let config_builder = iot.AwsIotMqttConnectionConfigBuilder.new_mtls_builder(
    certWithCa,
    readFileSync(argv.key, 'utf-8')
);
config_builder.with_clean_session(false);
config_builder.with_client_id(argv.client_id || "test-" + Math.floor(Math.random() * 100000000));
config_builder.with_endpoint(argv.endpoint);
const config = config_builder.build();

const client = new mqtt.MqttClient();
const connection = client.new_connection(config);

const timer = setInterval(() => { }, 60 * 1000);

await connection.connect()
await execute_session(connection, argv)
await connection.disconnect()

clearTimeout(timer);

And I was able to connect successfully on Mac with both RSA and ECC keys. Based on these results, I think this issue is now fixed.

---

One thing I noticed looking at the code snippet shared, the certificate and key order is flipped. In the code example provided, the order is:

const cfg = iot.AwsIotMqttConnectionConfigBuilder.new_mtls_builder(
    fs.readFileSync(`device.key`, 'utf-8'), // key
    certWithCa, // cert
)

But the order the new_mtls_builder function expects is:

const cfg = iot.AwsIotMqttConnectionConfigBuilder.new_mtls_builder(
    certWithCa, // cert
    fs.readFileSync(`device.key`, 'utf-8'), // key
)

However the order of the functions may have changed since the issue was opened and the code example shared. If you are still having issues though, it might be something to change and see if it fixes the issue.

---

I am going to close this issue, but please do not hesitate to reply or make a new issue if you are still having issues with concatenated certificate/key pairs or ECC keys on MacOS.
Thanks! Closing this issue...

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.