SocketClosedUnexpectedlyError: Socket closed unexpectedly #2032

wyvasi · 2022-03-11T09:55:52Z

Good day!
The socket connection closes unexpectedly and doesn't reconnect, my older version of node-redis(2.8.0) was reconnecting fine and still does.
After reconnect it doesn't print ready anymore.
We are using AWS ElastiCache.
I can only reproduce this on production env.
We have a package that wraps node-redis and is used in all our apps, so I don't think is related to version difference.
I tried to comment out next code and it doesn't close the connection at all (from @socket.io/redis-adapter).

this.subClient.pSubscribe(this.channel + "*", (msg, channel) => {
    this.onmessage(null, channel, msg);
}, true);

Redis connected
Redis ready!
Redis connected
Redis ready!
fetch error Error: timeout reached while waiting for fetchSockets response
    at Timeout._onTimeout (C:\work\test-socket\node_modules\@socket.io\redis-adapter\dist\index.js:615:28)
    at listOnTimeout (node:internal/timers:568:17)
    at processTimers (node:internal/timers:510:7)
List with sockets connected to hello room: 0
List with sockets connected to hello room: 5
SocketClosedUnexpectedlyError: Socket closed unexpectedly
    at Socket.<anonymous> (C:\work\test-socket\node_modules\@node-redis\client\dist\lib\client\socket.js:195:118)
    at Object.onceWrapper (node:events:510:26)
    at Socket.emit (node:events:390:28)
    at TCP.<anonymous> (node:net:687:12)
Redis reconnecting
Redis connected
ParserError: Protocol error, got "n" as reply type byte
    at handleError (C:\work\test-socket\node_modules\redis-parser\lib\parser.js:190:15)
    at parseType (C:\work\test-socket\node_modules\redis-parser\lib\parser.js:304:14) {
  offset: 304,
  buffer: '....'

Here is where I catch the parser error:
client/socket.ts

if (this.#initiator) {
            try {
                await this.#initiator();
            } catch (err) {

List with buffers and offsets:

[{
        offset: 405,
        buffer: '{"type":"Buffer","data":[42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,52,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,114,104,122,78,49,49,74,80,98,111,113,107,54,120,78,103,80,52,100,121,71,102,106,84,120,75,86,115,121,98,88,106,107,108,56,56,45,50,66,119,76,65,112,73,99,35,13,10,36,49,53,53,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,205,5,184,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,52,104,105,115,82,111,111,109,114,104,122,78,49,49,74,80,98,111,113,107,54,120,78,103,80,52,100,121,71,102,106,84,120,75,86,115,121,98,88,106,107,108,56,56,45,50,66,119,76,65,112,73,99,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10,42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,52,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,100,76,57,116,118,98,120,51,84,118,52,114,56,87,114,109,56,83,78,89,50,71,53,110,98,80,114,97,112,85,70,116,71,50,107,74,76,116,90,45,50,66,57,105,85,35,13,10,36,49,53,51,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,110,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,52,104,105,115,82,111,111,109,100,76,57,116,118,98,120,51,84,118,52,114,56,87,114,109,56,83,78,89,50,71,53,110,98,80,114,97,112,85,70,116,71,50,107,74,76,116,90,45,50,66,57,105,85,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10,42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,50,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,120,89,112,113,90,54,101,111,89,48,107,120,90,105,81,86,113,97,111,89,90,99,69,119,121,72,50,100,70,122,78,115,108,52,82,50,114,122,111,109,85,113,107,35,13,10,36,49,53,49,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,113,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,50,104,105,115,82,111,111,109,120,89,112,113,90,54,101,111,89,48,107,120,90,105,81,86,113,97,111,89,90,99,69,119,121,72,50,100,70,122,78,115,108,52,82,50,114,122,111,109,85,113,107,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10,42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,50,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,53,111,74,87,66,49,70,85,50,108,109,86,73,54,55,70,106,68,99,83,103,71,118,121,68,48,76,66,83,90,67,83,97,67,113,103,97,69,115,86,119,120,85,35,13,10,36,49,53,50,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,204,203,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,50,104,105,115,82,111,111,109,53,111,74,87,66,49,70,85,50,108,109,86,73,54,55,70,106,68,99,83,103,71,118,121,68,48,76,66,83,90,67,83,97,67,113,103,97,69,115,86,119,120,85,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10,42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,52,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,53,85,74,71,49,53,87,110,116,103,79,88,115,116,106,115,49,45,50,66,82,66,76,106,72,102,82,68,65,109,115,122,112,98,114,82,86,82,69,102,73,50,105,53,85,35,13,10,36,49,53,52,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,204,155,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,52,104,105,115,82,111,111,109,53,85,74,71,49,53,87,110,116,103,79,88,115,116,106,115,49,45,50,66,82,66,76,106,72,102,82,68,65,109,115,122,112,98,114,82,86,82,69,102,73,50,105,53,85,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10]}'
    },
    {
        offset: 136,
        buffer: '{"type":"Buffer","data":[42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,49,57,13,10,105,110,116,101,114,97,99,116,45,114,101,113,117,101,115,116,35,47,35,13,10,58,49,13,10,42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,50,48,13,10,105,110,116,101,114,97,99,116,45,114,101,115,112,111,110,115,101,35,47,35,13,10,58,50,13,10,42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,50,55,13,10,105,110,116,101,114,97,99,116,45,114,101,115,112,111,110,115,101,35,47,35,88,115,52,75,106,79,35,13,10,58,51,13,10,42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,50,52,13,10,105,110,116,101,114,97,99,116,45,114,101,113,117,101,115,116,35,47,97,100,109,105,110,35,13,10,58,52,13,10,42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,50,53,13,10,105,110,116,101,114,97,99,116,45,114,101,115,112,111,110,115,101,35,47,97,100,109,105,110,35,13,10,58,53,13,10,42,51,13,10,36,57,13,10,115,117,98,115,99,114,105,98,101,13,10,36,51,50,13,10,105,110,116,101,114,97,99,116,45,114,101,115,112,111,110,115,101,35,47,97,100,109,105,110,35,104,65,112,66,90,76,35,13,10,58,54,13,10,42,51,13,10,36,49,48,13,10,112,115,117,98,115,99,114,105,98,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,58,55,13,10,42,51,13,10,36,49,48,13,10,112,115,117,98,115,99,114,105,98,101,13,10,36,49,55,13,10,105,110,116,101,114,97,99,116,35,47,97,100,109,105,110,35,42,13,10,58,56,13,10]}'
    },
    {
        offset: 12,
        buffer: '{"type":"Buffer","data":[42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,56,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,90,71,86,107,121,121,84,115,98,77,84,45,50,66,111,76,53,69,104,82,106,69,121,81,78,116,48,100,69,57,99,66,52,45,50,70,54,67,90,76,113,81,113,45,50,66,117,81,119,35,13,10,36,49,53,56,13,10,147,166,119,65,102,98,82,66,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,204,131,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,56,104,105,115,82,111,111,109,90,71,86,107,121,121,84,115,98,77,84,45,50,66,111,76,53,69,104,82,106,69,121,81,78,116,48,100,69,57,99,66,52,45,50,70,54,67,90,76,113,81,113,45,50,66,117,81,119,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10]}'
    },
    {
        offset: 88,
        buffer: '{"type":"Buffer","data":[42,52,13,10,36,56,13,10,112,109,101,115,115,97,103,101,13,10,36,49,50,13,10,105,110,116,101,114,97,99,116,35,47,35,42,13,10,36,54,56,13,10,105,110,116,101,114,97,99,116,35,47,35,104,105,115,82,111,111,109,89,111,97,115,90,54,72,53,80,45,50,66,71,45,50,66,80,45,50,66,80,49,90,106,82,87,74,87,87,83,54,113,72,74,50,86,120,80,77,68,78,51,117,81,101,120,75,87,81,35,13,10,36,49,53,56,13,10,147,166,104,55,52,78,66,121,131,164,116,121,112,101,2,164,100,97,116,97,146,173,110,111,116,105,102,105,99,97,116,105,111,110,115,130,165,99,111,117,110,116,204,194,164,116,121,112,101,181,110,111,116,105,102,105,99,97,116,105,111,110,115,95,99,111,117,110,116,101,114,163,110,115,112,161,47,131,165,114,111,111,109,115,145,217,56,104,105,115,82,111,111,109,89,111,97,115,90,54,72,53,80,45,50,66,71,45,50,66,80,45,50,66,80,49,90,106,82,87,74,87,87,83,54,113,72,74,50,86,120,80,77,68,78,51,117,81,101,120,75,87,81,166,101,120,99,101,112,116,144,165,102,108,97,103,115,128,13,10]}'
    }]

Code for creating the client

const Redis = require('redis');
Redis.debug_mode = true;

const getClient = async (url) => {

    const client = Redis.createClient({
        socket: {
            host: url
            port: 6379,
        }
    });
    client.on('error', (err) => {
        console.error(err);
    });
    client.on('connect', () => {
        console.log('Redis connected');
    });
    client.on('reconnecting', () => {
        console.log('Redis reconnecting');
    });
    client.on('ready', () => {
        console.log('Redis ready!');
    });

    await client.connect();
    return client;
};

module.exports = {
    getClient,
};

Environment:

Node.js Version: v17.3.1
Redis Server Version: 6.2
Node Redis Version: 4.0.3
Platform: Ubuntu and Windows 10

The text was updated successfully, but these errors were encountered:

wyvasi · 2022-03-15T16:09:36Z

After disconnect/reconnect it stops receiving messages. With the older version of node-redis it was reconnecting without any problems. I added some code to print the parser error.

Code:

const { getClient } = require('./libs/redis');
const { redisUrl, key } = require('./config');

const start = async () => {
    const client = await getClient(redisUrl);
    client.pSubscribe(key + '#/#*', (msg, channel) => {
        onmessage();
    }, true);
    client.pSubscribe( key + '#/admin#*', (msg, channel) => {
        onmessage();
    }, true);
};

start().catch(err => console.log(err));

let x = 0;
const onmessage = (channel, msg) => {
    x++;
};

setInterval(() => {console.log(`Got ${x} messages!`)}, 5000);

const Redis = require('redis');

const onRedisError = (err) => {console.error(err)};
const onRedisConnect = () => {console.log('Redis connected')};
const onRedisReconnecting = () => {console.log('Redis reconnecting')};
const onRedisReady = () => {console.log('Redis ready!')};

const getClient = async (url) => {
    const client = Redis.createClient({
        socket: {
            host: url.replace('redis://', '').replace(':6379', ''),
            port: 6379,
        }
    });
    client.on('error', onRedisError);
    client.on('connect', onRedisConnect);
    client.on('reconnecting', onRedisReconnecting);
    client.on('ready', onRedisReady);

    await client.connect();
    return client;
};

module.exports = {
    getClient,
};

Output:

Redis connected
Redis ready!
Got 261 messages!
Got 1751 messages!
Got 3573 messages!
Got 5616 messages!
Got 8532 messages!
Got 9348 messages!
Got 9623 messages!
Got 10069 messages!
Got 11630 messages!
Got 11828 messages!
Got 12844 messages!
Got 14737 messages!
SocketClosedUnexpectedlyError: Socket closed unexpectedly
    at Socket.<anonymous> (C:\work\test-socket\node_modules\@node-redis\client\dist\lib\client\socket.js:196:118)
    at Object.onceWrapper (node:events:510:26)
    at Socket.emit (node:events:390:28)
    at TCP.<anonymous> (node:net:687:12)
Redis reconnecting
Redis connected
ParserError: Protocol error, got "e" as reply type byte
    at handleError (C:\work\test-socket\node_modules\redis-parser\lib\parser.js:190:15)
    at parseType (C:\work\test-socket\node_modules\redis-parser\lib\parser.js:304:14) {
  offset: 358,
  buffer: '{"type":"Buffer","data":[42,52,13,10,36,56,13,10,112,109,101,11.......
}
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!
Got 16116 messages!

rpong · 2022-03-16T03:28:21Z

We are currently stuck with v3 for months now because this issue seems to be still unresolved with v4.

erajanraja24 · 2022-03-27T12:51:58Z

@rpong Could you please update the tentative ETA for this issue as it is a blocker for v4?

rpong · 2022-04-03T17:30:38Z

@rpong Could you please update the tentative ETA for this issue as it is a blocker for v4?

Hi @erajanraja24, @leibale would be the best person to ask, I am not part of the team..

1nstinct · 2022-04-13T20:00:00Z

Hey guys @leibale. Any updates on this fix for this issue?

leibale · 2022-04-14T16:41:17Z

This code does not reproduce the issue (I'm running docker restart while the code is running)

import { createClient } from '@node-redis/client';

async function client() {
    const client = createClient();

    client.on('error', err => console.error('client error', err));
    client.on('connect', () => console.log('client is connect'));
    client.on('reconnecting', () => console.log('client is reconnecting'));
    client.on('ready', () => console.log('client is ready'));

    await client.connect();

    return client;
}

const [subscriber, publisher] = await Promise.all([
    client(),
    client()
]);

let counter = 0;
subscriber.pSubscribe('*', () => counter++);
setInterval(() => console.log(`Got ${counter} messages!`), 1000);

setInterval(async () => {
    try {
        await publisher.publish('channel', 'message');
    } catch (err) {
        console.error('publish error', err);
    }
}, 5);

client error Error: connect ECONNREFUSED 127.0.0.1:6379
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1146:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 6379
}
client is reconnecting
client is reconnecting
client is connect
client is connect
client is ready
Got 484 messages!
Got 484 messages!
client is ready
Got 677 messages!
Got 868 messages!

rpong · 2022-05-04T01:50:06Z

@leibale i just tested 4.1.0 and it seems this issue still persists, reconnecting still doesn't work like it used to like in v3. after "Socket closed unexpectedly", it just doesn't reconnect without restarting the client application. We simulate this by restarting redis (docker) while the application is running.

leibale · 2022-05-04T02:11:53Z

#1985 (comment)

rpong · 2022-05-04T03:02:56Z

    client.on('error', err => console.error('client error', err));
    client.on('connect', () => console.log('client is connect'));
    client.on('reconnecting', () => console.log('client is reconnecting'));
    client.on('ready', () => console.log('client is ready'));

Thanks @leibale , i have discovered that this IS working if we do following before the "await client.connect();"

client.on('error', err => console.log('client error', err));
client.on('connect', () => console.log('client is connect'));
client.on('reconnecting', () => console.log('client is reconnecting'));
client.on('ready', () => console.log('client is ready'));

but without the above, it doesn't reconnect like v3 does. We can upgrade now and just pass debug info for the above, but would like to confirm if this is the intended behavior.

leibale · 2022-05-04T03:21:37Z

client.on('error') was required in v3 as well (see https://nodejs.org/api/events.html#error-events)

rpong · 2022-05-04T03:26:09Z

https://nodejs.org/api/events.html#error-events

In our case, we did not have this at all for v3 but reconnecting was working.

Thank you for the info, we can finally upgrade to v4 after many months.

cheesecakeassassin · 2022-05-16T04:38:48Z

I can confirm that the code above that @leibale provided fixes the issue! My issue in particular was that I had deployed my app using Redis to Heroku and upon the socket closing unexpectedly, the Heroku-deployed app would crash until I redeployed it. The problem was driving me nuts and this was the only solution on the web that fixed it. Thank you!

KunalBurangi · 2022-05-16T13:29:36Z

@leibale , The issue persists in v4 and yes it does reconnect. but is there any particular reason why is it crashing unexpectedly?
yes we do have load-balancer and proxies like gateway and Nginx

derikb · 2022-05-17T14:56:10Z

Just had this occur for us with 4.1.0 running...

Code:

const pubClient = createClient(config);
    const subClient = pubClient.duplicate();

    pubClient.on('connect', () => { chatlogger.error('Redis pub adapter connect'); });
    pubClient.on('ready', () => { chatlogger.error('Redis pub adapter ready'); });
    pubClient.on('end', () => { chatlogger.error('Redis pub adapter end'); });
    pubClient.on('reconnecting', () => { chatlogger.error('Redis pub adapter reconnecting'); });
    pubClient.on('error', (err) => {
        chatlogger.error('Redis pub adapter error', err);
    });

    subClient.on('connect', () => { chatlogger.error('Redis sub adapter connect'); });
    subClient.on('ready', () => { chatlogger.error('Redis sub adapter ready'); });
    subClient.on('end', () => { chatlogger.error('Redis sub adapter end'); });
    subClient.on('reconnecting', () => { chatlogger.error('Redis sub adapter reconnecting'); });
    subClient.on('error', (err) => {
        chatlogger.error('Redis sub adapter error', err);
    });

    await pubClient.connect();
    await subClient.connect();

Our logging:

2022-05-17 13:58:01 error: Redis sub adapter error: SocketClosedUnexpectedlyError: Socket closed unexpectedly
2022-05-17 13:58:01 error: Redis sub adapter reconnecting: ''
2022-05-17 13:58:01 error: Redis sub adapter connect: ''

The ready event didn't ever happen. When I restarted node it did connect/ready.

Any other ideas/tips? Something I'm missing? This is for a socket.io adaptor so the redis connection going down really messes with the rest of our app.

derikb · 2022-05-17T17:11:23Z

I should add I was getting the ready event in dev testing if I just stopped/started redis. It seems to still be a problem for whatever is happening when the socket closes unexpectedly. (In my case via an AWS Elasticache.

ryanmeador · 2022-08-01T18:43:28Z

I believe I'm experiencing this issue on 4.2.0. I set my event listeners like this:

  client.on('error', err => log.error(`Redis error: ${err}`));
  client.on('reconnecting', params => log.info(`Redis reconnecting, attempt ${params.attempt}`));
  client.on('connect', () => log.info('Redis connected'));
  client.on('ready', () => log.info('Redis ready'));
  client.on('end', () => log.info('Redis connection closed'));

When it starts up, I see this in the log:

INFO redis: Redis connected
INFO redis: Redis ready

After a while, it disconnects. I'm using Heroku's Redis server, which seems to drop connections after a few minutes, I think in an attempt to clean up stale connections. Before upgrading from 3.x.x, it would reconnect seamlessly. Now, all I see is this:

ERROR redis: Redis error: Error: Socket closed unexpectedly

No reconnect, no ready. I have to restart my Node process to get it working again. Can you offer any advice?

ryanmeador · 2022-08-01T19:20:19Z

I think I have discovered the cause of my problem. I was expecting a params object to be passed to my reconnect event handler, but this was apparently removed in v4. Since my handler was now accessing an undefined property, it was throwing an error at runtime. This error was silently swallowed, but still prevented the reconnection. Could you please log errors of this nature, and still continue with the reconnect? No doubt it will help others in the future who face a similar problem.

leibale · 2022-08-01T21:21:53Z

@ryanmeador can you please open a new issue with reproduction?

redis/node-redis#2032

inapeace0 · 2022-08-05T16:14:19Z

Hi, I got this same issue on heroku and I changed to the latest version(4.2.0), and still got the same issue after some minutes.

This is my code.

import { createClient } from 'redis';
import config from 'config'

const redisUrl = config.get<string>("redisUrl");

const redisClient = createClient({
  url: redisUrl,
});

const connectRedis = async () => {
  try {
    await redisClient.connect();
    console.log('Redis client connect successfully');
    redisClient.set('try', 'Hello Welcome to Express with TypeORM');
  } catch (error) {
    console.log(error);
    setTimeout(connectRedis, 5000);
  }
};

connectRedis();

export default redisClient;

Is there a solution for this? Or maybe I rewrite the code?

leibale · 2022-08-05T20:16:28Z

@CaCaBlocker the only known bug that can cause this issue is the one @ryanmeador pointed out in this comment, and I'm already working to fix it (see this branch).

Other than that, this error could be caused by some kind of a proxy or a load balancer between the redis-server and the node server.

Without a reproduction I won't be able to debug and hopefully solve it.

fabioselau077 · 2022-12-20T00:54:53Z

@leibale

Seeing the same on a managed redis db on digitalocean after upgrading.

Redis 6 node-redis: 4.5.1

same, using redis db on digitalocean
How to fixe?

leibale · 2022-12-20T00:58:46Z

@fabioselau077 @jorenvandeweyer https://www.digitalocean.com/community/questions/idle-redis-connection-reconnects-every-five-minutes
try to set pingInterval to a value lower than 5 minutes (4 * 60 * 1000 should do it)

ilhmyz · 2022-12-22T10:43:19Z

@fabioselau077
using redis db on digitalocean try to set TLS to true
see https://github.com/redis/node-redis/blob/master/docs/client-configuration.md#TLS

If certain event handlers aren't defined on the client before connecting, the app will crash on disconnect event (e.g., timeout). Issue comment regarding issue details from node-redis repo: redis/node-redis#2032 (comment) Fixes #129, #160

HenonoaH · 2023-04-24T05:47:52Z

Hi, I got this same issue on heroku and I changed to the latest version(4.2.0), and still got the same issue after some minutes.

This is my code.

import { createClient } from 'redis';
import config from 'config'

const redisUrl = config.get<string>("redisUrl");

const redisClient = createClient({
  url: redisUrl,
});

const connectRedis = async () => {
  try {
    await redisClient.connect();
    console.log('Redis client connect successfully');
    redisClient.set('try', 'Hello Welcome to Express with TypeORM');
  } catch (error) {
    console.log(error);
    setTimeout(connectRedis, 5000);
  }
};

connectRedis();

export default redisClient;

Is there a solution for this? Or maybe I rewrite the code?

@CaCaBlocker I having the same issue from Redis hosted in Heroku. Did you get any solution or workaround for this?

polearnik · 2023-04-24T06:54:55Z

@HenonoaH try to add listeners to all available events like ready reconnect error etc.

HenonoaH · 2023-04-24T09:14:13Z

@polearnik I have already added all the event listeners. I like to know if that exception is standard. I have configured Rollbar for my app and getting an exception log every five minutes!! This happens after the upgrade.

If certain event handlers aren't defined on the client before connecting, the app will crash on disconnect event (e.g., timeout). Issue comment regarding issue details from node-redis repo: redis/node-redis#2032 (comment) Fixes #129, #160

leonardovenoso · 2023-05-02T09:45:56Z

Changing to redis-server version 6 on Ubuntu and Nodejs Redis 4.6.6 it works fine.
NestJS example:

import { Module } from '@nestjs/common';
import { createClient } from 'redis';
import { ConfigService } from '@nestjs/config';

@module({
providers: [
{
inject: [ConfigService],
provide: 'REDIS_CLIENT',
useFactory: async (configService: ConfigService) => {
const redisCli = createClient({
url: redis://${configService.get('REDIS_USER')}:${configService.get('REDIS_PASSWORD')}@${configService.get('REDIS_HOST')}:${configService.get('REDIS_PORT')},
});

    redisCli.on('connect', () => {
      console.log('Connected to Redis');
    });
    
    redisCli.on('error', (err) => {
      console.log(err.message);
    });
    
    redisCli.on('ready', () => {
      console.log('Redis is ready');
    });
    
    redisCli.on('reconnecting', () => {
      console.log('client is reconnecting')
    });

    redisCli.on('end', () => {
      console.log('Redis connection ended');
    })
    
    process.on('SIGINT', () => {
      redisCli.quit();
    });

    return redisCli;
  }
}

],
exports: ['REDIS_CLIENT'],
})
export class RedisCliModule {}

hieuhuynhh · 2023-05-10T04:41:21Z

@fabioselau077 @jorenvandeweyer https://www.digitalocean.com/community/questions/idle-redis-connection-reconnects-every-five-minutes try to set pingInterval to a value lower than 5 minutes (4 * 60 * 1000 should do it)

Hi @leibale, I'm using ioredis and i don't see pingInterval option. How can i solve it with ioredis? Thanks.

leibale · 2023-05-10T07:46:59Z

@hieuhuynhh a simple setInterval should fix it:

const timer = setInterval(async () => {
  try {
    await client.ping();
  } catch (err) {
    console.error('Ping Interval Error', err);
  }
}, 1000 * 60 * 4);

// make sure to `clearInterval` when you close the client
clearInterval(timer);
client.disconnect();

niksy · 2023-05-19T11:21:19Z

Although documentation mentions setting handler for error event, it is not clear that this is actually needed to have this issue fixed. There weren’t any errors of this kind in v3.

Is there a reason why this happens in v4? Can we somehow check for that error inside error handler besides it’s message content? There isn’t any error code for that, and instanceof check is not inituitive since errors modules is not exposed publicly.

This could also be related to comment in #1993 (comment).

leibale · 2023-05-21T09:31:41Z

@niksy https://github.com/redis/node-redis#events

":warning: You MUST listen to error events. If a client doesn't have at least one error listener registered and an error occurs, that error will be thrown and the Node.js process will exit. See the EventEmitter docs for more details."

niksy · 2023-05-22T07:30:56Z

Yeah, I understand that, sorry I missed that part. It makes sense that this is needed, but it doesn’t answer the question why this type of error doesn’t happen on v3. I will see if it’s related to pinging you mentioned.

leibale · 2023-05-23T08:25:23Z

@niksy #1993 (comment)

ishaarawy · 2023-05-31T12:44:58Z

changing from socket connection to url solved the issue on local machine connecting to docker container

redisClient = redis.createClient({
          url: 'redis://127.0.0.1:6379'
        });

djdabs · 2023-06-02T21:53:27Z

not sure if related but I'm getting similar errors. All event listeners are configured. Application is a nestjs server dockerized on node:18-alpine, talking to memorystore redis in GCP (idle connections are not automatically closed).

client initially connects -> socket gets closed -> client tries reconnecting -> uncaught exception occurs for socket closing

    "redis": "^4.6.7",

 async connectToRedis(): Promise<void>{
    try{
      const pubClient = createClient({
          socket: {
                host: process.env.REDIS_HOST,
                port: Number(process.env.REDIS_PORT)
         },
      });
  
      const subClient = pubClient.duplicate();
  
      pubClient.on('error', (error) => {
        console.error('pubClient redis error:', error);
      });
      pubClient.on('connect', () => console.log('pubClient redis is connected'));
      pubClient.on('reconnecting', () => console.log('pubClient redis is reconnecting'));
      pubClient.on('ready', () => console.log('pubClient redis is ready'));
      
      subClient.on('error', (error) => {
        console.error('subClient redis error:', error);
      });
      subClient.on('connect', () => console.log('subClient redis is connected'));
      subClient.on('reconnecting', () => console.log('subClient redis is reconnecting'));
      subClient.on('ready', () => console.log('subClient redis is ready'));
  
      await pubClient.connect()
      await subClient.connect()
     
      this.adapterConstructor = createAdapter(pubClient, subClient);
    } catch(err){
      console.log(err)
    }
    
  }

niksy · 2023-06-03T09:20:21Z

@djdabs try this: #2443 (comment)

djdabs · 2023-06-05T18:39:11Z

@djdabs try this: #2443 (comment)

Still getting a triggerUncaugthException after client connects -> attempts to reconnect, but I'm now getting a different error.

fwiw, i don't think i need a pingInterval with GCP. Their docs say by default no idle timeout is configured. https://cloud.google.com/memorystore/docs/redis/supported-redis-configurations#modifiable_configuration_parameters

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:930:11)
    at Socket._write (node:net:942:8)
    at doWrite (node:internal/streams/writable:411:12)
    at clearBuffer (node:internal/streams/writable:572:7)
    at Writable.uncork (node:internal/streams/writable:351:7)
    at Immediate._onImmediate (/usr/src/app/node_modules/@redis/client/dist/lib/client/socket.js:84:69)
    at process.processImmediate (node:internal/timers:476:21) {

djdabs · 2023-06-06T01:14:25Z

resolved my issue.. after switching to ioredis the error message got more specific and I was able to track down the underlying problem (redis required TLS). Disabled TLS on memorystore and it was able to work now.

redis error: ParserError: Protocol error, got "\u0015" as reply type byte. Please report this

redis/node-redis#2032

stefanodecillis · 2023-08-28T09:14:47Z

Are there any updates on this issue? I'm currently facing the same problem while using a Redis instance served by Digital Ocean. TLS is in place and the job is listening on "error" - each time the issue occurs, I reset the connection. I don't know if it is the best solution but it kind of works even if it throws the error every five minutes.

In the previous comments, there is a suggestion that pings the server. Do you think that is the best way to handle it at the moment? Or should we wait the patch?

jorenvandeweyer · 2023-08-28T09:32:08Z

@stefanodecillis You can just use pingInterval since #2524 is fixed in version redis@4.6.8

shobhitbehl98 · 2023-12-29T20:23:25Z

Getting the same error at v4.6.12
`const redis = require('redis');
require('dotenv').config();
const client = redis.createClient({
url: process.env.KV_URL
});
(async () => {
await client.connect();
})();
client.on('connect', () => {
console.log('Connected to Redis');
});

client.on('error', (err) => {
console.error(Redis Error: ${err});
});

client.on('reconnecting', () => console.log('client is reconnecting'));
client.on('ready', () => console.log('client is ready'));

module.exports = client;`

This is the error im gettting

ERROR Redis Error: Error: Socket closed unexpectedly INFO client is reconnecting INFO Connected to Redis ERROR Redis Error: Error: Socket closed unexpectedly ERROR Redis Error: Error: Socket closed unexpectedly INFO client is reconnecting INFO Connected to Redis ERROR Redis Error: Error: Socket closed unexpectedly ERROR Redis Error: Error: Socket closed unexpectedly Task timed out after 10.02 seconds

It runs well on local but not when deployed on vercel

jake-chambers · 2024-01-06T19:57:28Z

We are experiencing the same issue. We're listening error events explicitly...our connection logic is below @leibale :

export const createRedisClient = (options: RedisClientOptions) => {
    const client = createClient(options);
    client.on('error', async error => {
        console.error(`Redis client error : ${error}`);
        Sentry.captureException(error); // Capture the error with Sentry

        if (
            error.code === 'ECONNRESET' ||
            error.code === 'EPIPE' ||
            error.code === 'ECONNREFUSED'
        ) {
            // Disconnect before attempting to reconnect
            await client.disconnect();

            client.connect().catch(err => {
                console.error(`Failed to reconnect redis client: ${err}`);
                Sentry.captureException(err); // Capture the reconnection error with Sentry
            });
        }
    });
    client.connect().catch(err => {
        console.error(`Failed to connect redis client: ${err}`);
        Sentry.captureException(err); // Capture the reconnection error with Sentry
    });
    return client;
};

arpit-absyadav · 2024-02-01T05:22:58Z

The error message indicates that Redis is running in protected mode due to enabled protected mode and the absence of a password for the default user. In this mode, connections are only accepted from the loopback interface. To enable connections from external computers, you can choose one of the following solutions:

Disable protected mode by executing the command: 'CONFIG SET protected-mode no'.

wyvasi added the Bug label Mar 11, 2022

rpong mentioned this issue Mar 16, 2022

socket.reconnectStrategy short circuits if no error listener is added to redis client. #1993

Closed

leibale mentioned this issue Apr 21, 2022

fix socket error handlers #2092

Merged

leibale linked a pull request Apr 21, 2022 that will close this issue

fix socket error handlers #2092

Merged

leibale closed this as completed in #2092 Apr 25, 2022

sampolahtinen mentioned this issue Jun 28, 2022

fix: revert to redis 3.1.2 sampolahtinen/bigbuttonmix#41

Merged

mifi added a commit to transloadit/uppy that referenced this issue Aug 4, 2022

fix crash if redis disconnects

e340d84

redis/node-redis#2032

mifi mentioned this issue Aug 4, 2022

fix crash if redis disconnects transloadit/uppy#3954

Merged

mifi added a commit to transloadit/uppy that referenced this issue Aug 4, 2022

fix crash if redis disconnects (#3954)

88b08e3

redis/node-redis#2032

leibale mentioned this issue Aug 8, 2022

Handle unhandled errors in socket.reconnectStrategry #2226

Merged

sadsciencee mentioned this issue Feb 23, 2023

Improve RedisSessionStorage for Heroku-hosted apps Shopify/shopify-app-js#129

Closed

mkevinosullivan mentioned this issue Apr 6, 2023

Add event handlers to redis client Shopify/shopify-app-js#194

Merged

3 tasks

niksy mentioned this issue May 19, 2023

SocketClosedUnexpectedlyError while awaiting #initiator results in multiple connecting sockets / connections #2443

Closed

HeavenFox pushed a commit to docsend/uppy that referenced this issue Jun 27, 2023

fix crash if redis disconnects (transloadit#3954)

36bc872

redis/node-redis#2032

Vortec4800 mentioned this issue Aug 3, 2023

fix: Redis 4 does not reconnect after unhandled error parse-community/parse-server#8706

Merged

2 tasks

SocketClosedUnexpectedlyError: Socket closed unexpectedly #2032

SocketClosedUnexpectedlyError: Socket closed unexpectedly #2032

Comments

wyvasi commented Mar 11, 2022 • edited Loading

wyvasi commented Mar 15, 2022 • edited Loading

rpong commented Mar 16, 2022

erajanraja24 commented Mar 27, 2022

rpong commented Apr 3, 2022

1nstinct commented Apr 13, 2022

leibale commented Apr 14, 2022 • edited Loading

rpong commented May 4, 2022 • edited Loading

leibale commented May 4, 2022

rpong commented May 4, 2022

leibale commented May 4, 2022

rpong commented May 4, 2022

cheesecakeassassin commented May 16, 2022 • edited Loading

KunalBurangi commented May 16, 2022

derikb commented May 17, 2022

derikb commented May 17, 2022

ryanmeador commented Aug 1, 2022

ryanmeador commented Aug 1, 2022

leibale commented Aug 1, 2022

inapeace0 commented Aug 5, 2022 • edited Loading

leibale commented Aug 5, 2022 • edited Loading

fabioselau077 commented Dec 20, 2022 • edited Loading

leibale commented Dec 20, 2022 • edited Loading

ilhmyz commented Dec 22, 2022

HenonoaH commented Apr 24, 2023

polearnik commented Apr 24, 2023

HenonoaH commented Apr 24, 2023

leonardovenoso commented May 2, 2023

hieuhuynhh commented May 10, 2023

leibale commented May 10, 2023

niksy commented May 19, 2023 • edited Loading

leibale commented May 21, 2023

niksy commented May 22, 2023

leibale commented May 23, 2023

ishaarawy commented May 31, 2023 • edited Loading

djdabs commented Jun 2, 2023 • edited Loading

niksy commented Jun 3, 2023

djdabs commented Jun 5, 2023 • edited Loading

djdabs commented Jun 6, 2023

stefanodecillis commented Aug 28, 2023

jorenvandeweyer commented Aug 28, 2023

shobhitbehl98 commented Dec 29, 2023 • edited Loading

jake-chambers commented Jan 6, 2024 • edited Loading

arpit-absyadav commented Feb 1, 2024

wyvasi commented Mar 11, 2022 •

edited

Loading

wyvasi commented Mar 15, 2022 •

edited

Loading

leibale commented Apr 14, 2022 •

edited

Loading

rpong commented May 4, 2022 •

edited

Loading

cheesecakeassassin commented May 16, 2022 •

edited

Loading

inapeace0 commented Aug 5, 2022 •

edited

Loading

leibale commented Aug 5, 2022 •

edited

Loading

fabioselau077 commented Dec 20, 2022 •

edited

Loading

leibale commented Dec 20, 2022 •

edited

Loading

niksy commented May 19, 2023 •

edited

Loading

ishaarawy commented May 31, 2023 •

edited

Loading

djdabs commented Jun 2, 2023 •

edited

Loading

djdabs commented Jun 5, 2023 •

edited

Loading

shobhitbehl98 commented Dec 29, 2023 •

edited

Loading

jake-chambers commented Jan 6, 2024 •

edited

Loading