OpenSSL exception when reading/writing at line-rate

[carlhoerberg]

When reading from a OpenSSL::SSL::Socket::Server at full speed I very quickly get

OpenSSL::SSL::Error:SSL_read: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac

However, if i phase the write side I don't get it.

Will add an example but opening the issue as an placeholder.

[carlhoerberg]

This is a minimal example that reproduces the problem consistently:

require "socket"
require "openssl"

def server
  socket = TCPServer.new(5555)
  context = OpenSSL::SSL::Context::Server.new
  context.private_key = "localhost.key"
  context.certificate_chain = "localhost.crt"

  loop do
    client = socket.accept? || break
    ssl_socket = OpenSSL::SSL::Socket::Server.new(client, context, sync_close: true)
    spawn echo(ssl_socket)
  end
end

def echo(ssl_socket)
  puts "Client connected"
  i = 0
  buf = uninitialized UInt8[4096]
  loop do
    i += 1
    ssl_socket.read(buf.to_slice)
    ssl_socket.write(buf.to_slice)
  end
rescue ex
  puts "Failed at the #{i} read"
  raise ex
end

def client
  socket = TCPSocket.new("127.0.0.1", 5555)
  context = OpenSSL::SSL::Context::Client.new
  context.verify_mode = OpenSSL::SSL::VerifyMode::NONE
  ssl_socket = OpenSSL::SSL::Socket::Client.new(socket, context)
  spawn do
    buf = StaticArray(UInt8, 1000).new(0)
    loop do
      ssl_socket.write buf.to_slice
    end
  end
  buf = uninitialized UInt8[4096]
  loop do
    ssl_socket.read buf.to_slice
  end
end

unless File.exists? "localhost.key"
  system "openssl req -x509 -out localhost.crt -keyout localhost.key -newkey rsa:2048 -nodes -sha256 -subj '/CN=localhost'"
end

spawn client
server

Output:

Client connected
Failed at the 137 read
Unhandled exception in spawn: SSL_read: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac (OpenSSL::SSL::Error)
  from /usr/local/Cellar/crystal/0.27.2/src/openssl/ssl/socket.cr:110:9 in 'unbuffered_read'
  from /usr/local/Cellar/crystal/0.27.2/src/io/buffered.cr:67:16 in 'read'
  from tls_bug.cr:23:5 in 'echo'
  from tls_bug.cr:13:5 in '->'
  from /usr/local/Cellar/crystal/0.27.2/src/fiber.cr:255:3 in 'run'
  from /usr/local/Cellar/crystal/0.27.2/src/fiber.cr:72:34 in '->'

Now if you tweak the StaticArray we're sending, and use 1024 or 4096 I don't not get the same problem. 1022 works too, but not 1020. Fails after 137 iterations each time.

Curiously, if i decrease the OpenSSL:SSL::Socket::Server#send_buffer_size to 8192 it doesn't happen either, but with anything higher than that it fails after 137 reads.

[carlhoerberg]

Also noticed that if I disable write buffering on the client socket it doesn't crash either:

  ssl_socket = OpenSSL::SSL::Socket::Client.new(socket, context)
  socket.sync = true
...

Related to #7458 ?

[asterite]

@carlhoerberg Could you add a spec to this in spec/openssl/ssl/server_spec.cr? There's a helper ssl_context_pair, check the other specs.

I tried to write it but I don't understand the code above well enough to put it in a spec.

(Or if someone else wants to do it, please go ahead. Thank you!)