[1.x] Fatal (internal) error in ../silk/float/LPC_analysis_filter_FLP.c, line 242: assertion failed: 0

[Creiger]

What version of Janus is this happening on?
1201 (1.2.1)
Compiled with package: libopus-dev (1.3.1-0.1)
Audiobridge

Have you tested a more recent version of Janus too?
Yes

Was this working before?
Not sure

Is there a gdb or libasan trace of the issue?
Fatal (internal) error in ../silk/float/LPC_analysis_filter_FLP.c, line 242: assertion failed: 0

Additional context
This fatal error appears few times per day. It depends on number or type of clients connected, we cannot tell for sure. We think it has something to do with opus codec?

[lminiero]

Yes, it looks like a crash in libopus. Nothing we can do in Janus for sure. Try installing the library manually instead of from the repo to see if you get a different result.

[Creiger]

Just to let you know if someone else have the same problem. What we have found is that it happens randomly when switching between rooms with different sampling rate. When we switch only between rooms with same sampling rate, there is no problem at all.

[atoppi]

@Creiger I can't seem to reproduce the issue with my environment:

Chrome/Linux v121
janus master
libopus (installed from apt) 1.3.1-0.1build2

Basically I tried switching rooms several times, testing different sampling rates and channels too.
The request always succeeded and the participant was audible in the room.

Any chance you have multiple versions of libopus installed on your system and Janus is picking an older (bugged?) one?

[atoppi]

I left a script running in the background, and it crashed janus eventually

==10965==ERROR: AddressSanitizer: heap-use-after-free on address 0x6290000dd2ca at pc 0x7fa60e23a2c3 bp 0x7fa5f4cf4450 sp 0x7fa5f4cf3bf8
WRITE of size 32 at 0x6290000dd2ca thread T40
[Fri Feb  2 16:38:26 2024] Last user/forwarder/file just left room 161, going idle...
[Fri Feb  2 16:38:26 2024] First user/forwarder/file just joined room 481, waking it up...
    #0 0x7fa60e23a2c2 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x7fa5fe4c1c89  (/lib/x86_64-linux-gnu/libopus.so.0+0x23c89)
    #2 0x7fa5fe4ba078  (/lib/x86_64-linux-gnu/libopus.so.0+0x1c078)
    #3 0x7fa5fe4bb3bc  (/lib/x86_64-linux-gnu/libopus.so.0+0x1d3bc)
    #4 0x7fa5fe4d5aa1  (/lib/x86_64-linux-gnu/libopus.so.0+0x37aa1)
    #5 0x7fa5fe4d74ad  (/lib/x86_64-linux-gnu/libopus.so.0+0x394ad)
    #6 0x7fa5fe4d7843 in opus_decode (/lib/x86_64-linux-gnu/libopus.so.0+0x39843)
    #7 0x7fa5fbb50f6f in janus_audiobridge_participant_thread plugins/janus_audiobridge.c:8643
    #8 0x7fa60df71a50 in g_thread_proxy ../../../glib/gthread.c:827
    #9 0x7fa60cc94ac2 in start_thread nptl/pthread_create.c:442
    #10 0x7fa60cd2684f  (/lib/x86_64-linux-gnu/libc.so.6+0x12684f)

0x6290000dd2ca is located 4298 bytes inside of 18228-byte region [0x6290000dc200,0x6290000e0934)
freed by thread T31 here:
    #0 0x7fa60e2b4537 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127
    #1 0x7fa5fbb15d38 in janus_audiobridge_handler plugins/janus_audiobridge.c:7285
    #2 0x7fa60df71a50 in g_thread_proxy ../../../glib/gthread.c:827

previously allocated by thread T31 here:
    #0 0x7fa60e2b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7fa5fe4d6b7d in opus_decoder_create (/lib/x86_64-linux-gnu/libopus.so.0+0x38b7d)

Thread T40 created by T31 here:
    #0 0x7fa60e258685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x7fa60df97504 in g_system_thread_new ../../../glib/gthread-posix.c:1323

Thread T31 created by T0 here:
    #0 0x7fa60e258685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x7fa60df97504 in g_system_thread_new ../../../glib/gthread-posix.c:1323

[atoppi]

@Creiger I am unable to reproduce with the patch below, can you test it please?

diff --git a/src/plugins/janus_audiobridge.c b/src/plugins/janus_audiobridge.c
index 14a36eaf..9444fc78 100644
--- a/src/plugins/janus_audiobridge.c
+++ b/src/plugins/janus_audiobridge.c
@@ -7278,12 +7278,18 @@ static void *janus_audiobridge_handler(void *data) {
                                }
                                participant->reset = FALSE;
                                /* Destroy the previous encoder/decoder and update the references */
+                               while(!g_atomic_int_compare_and_exchange(&participant->encoding, 0, 1))
+                                       g_usleep(5000);
                                if(participant->encoder)
                                        opus_encoder_destroy(participant->encoder);
                                participant->encoder = new_encoder;
+                               g_atomic_int_set(&participant->encoding, 0);
+                               while(!g_atomic_int_compare_and_exchange(&participant->decoding, 0, 1))
+                                       g_usleep(5000);
                                if(participant->decoder)
                                        opus_decoder_destroy(participant->decoder);
                                participant->decoder = new_decoder;
+                               g_atomic_int_set(&participant->decoding, 0);
                        }
                        if(quality)
                                opus_encoder_ctl(participant->encoder, OPUS_SET_COMPLEXITY(participant->opus_complexity));

[atoppi]

@Creiger any update ?

[Creiger]

Hi sorry for no answer. We did not have time to test it yet. Will let you know ASAP.

[lminiero]

@Creiger ping 😁

[atoppi]

@Creiger still nothing to report?

[atoppi]

@Creiger we have merged the fix since it fixed the issue for us.