Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rtc2rtmp: Sometimes there may be a decoding failure in the audio, resulting in no audio frames in the rtmp. #2690

Closed
jamken opened this issue Oct 24, 2021 · 4 comments · Fixed by #2757
Closed

rtc2rtmp: Sometimes there may be a decoding failure in the audio, resulting in no audio frames in the rtmp. #2690

jamken opened this issue Oct 24, 2021 · 4 comments · Fixed by #2757
Assignees
@winlinvip
Labels
TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.
Milestone
5.0

Comments

@jamken
Copy link
Contributor

jamken commented Oct 24, 2021 •
edited by winlinvip
Loading

Description'

Please ensure that the markdown structure is maintained.

Please describe the issue you encountered here.
'
Make sure to maintain the markdown structure.

In the rtc-rtmp scenario, sometimes there is a decoding failure in the audio, resulting in no audio frames in the rtmp. From the logs, it can be seen that the decoder reported an "eof" error, which caused the decoding to fail. However, sometimes it works fine, and it is unclear what causes this situation to occur.

[2021-10-23 14:00:24.213][Warn][3266][43u26x95][11] handle udp pkt, count=11950/11950, err: code=5011 : 
size=112, data=[90 6f 71 c0 19 b8 58 dd] : cipher=112, plaintext=102, body=[be de 00 01 31 29 73 00] : 
on audio : source on rtp : bridger consume message : decode and resample : 
submit to dec(-541478725,End of file)

thread [3266][43u26x95]: cycle() [src/app/srs_app_listener.cpp:630][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_conn.cpp:1388][errno=11]
thread [3266][02247790]: do_on_rtp_plaintext() [src/app/srs_app_rtc_conn.cpp:1441][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_source.cpp:2415][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_source.cpp:630][errno=11]
thread [3266][02247790]: transcode() [src/app/srs_app_rtc_codec.cpp:145][errno=11]
thread [3266][02247790]: decode_and_resample() [src/app/srs_app_rtc_codec.cpp:314][errno=11]

  1. SRS version: 4.0

  2. SRS configuration is as follows:

listen 1935;
pid srs.pid;
srs_log_tank file;
srs_log_level trace;
srs_log_file ./logs/srs.log;
max_connections 1000;
daemon off;
asprocess on;

http_server {
    enabled         on;
    listen          0.0.0.0:1936;
    dir             ./html;
}

http_api {
    enabled         on;
    listen          0.0.0.0:1985;
    crossdomain     on;
}

rtc_server {
    enabled on;
    listen 8000;
    candidate *;
}

vhost __defaultVhost__ {
    rtc {
        enabled on;
        nack on;
        twcc on;
        rtc_to_rtmp on;
        pli_for_rtmp 2.0;
    }

    hls {
        enabled on;
        hls_fragment 1.5;
        hls_window 20;
        hls_td_ratio 1.5;
        hls_path /opensight/nginx/html/hls_srs/hls_srs;
        hls_m3u8_file   [app]/[stream].m3u8;
        hls_ts_file     [app]/[stream]-[seq].ts;
        hls_wait_keyframe on;
        hls_dispose     20;
    }

    http_hooks {
        enabled on;
        on_publish http://127.0.0.1:1938/api/srsproxy/v1/srscbk;
        on_unpublish http://127.0.0.1:1938/api/srsproxy/v1/srscbk;
        on_play http://127.0.0.1:1938/api/srsproxy/v1/srscbk;
    }

    http_remux {
        enabled     on;
        mount       [vhost]/flv_srs/[app]/[stream].flv;
    }

    play{
        gop_cache       on;
        queue_length    10;
        mw_latency      20;
        reduce_sequence_header  on;
        time_jitter     off;
        mix_correct     off;
    }
        
    min_latency     on;
    tcp_nodelay     on;
}
  1. SRS logs are as follows:
[2021-10-23 14:00:19.818][Trace][3266][k7176d6x] HTTP #1 192.168.0.23:1630 POST http://192.168.0.225:1985/rtc/v1/play/?vc=ivWxqY, content-length=5715
[2021-10-23 14:00:19.820][Trace][3266][k7176d6x] http: on_play ok, client_id=k7176d6x, url=http://127.0.0.1:1938/api/srsproxy/v1/srscbk, request={"server_id":"vid-oe5jd7c","action":"on_play","client_id":"k7176d6x","ip":"192.168.0.23","vhost":"__defaultVhost__","app":"quick","stream":"ClpheNxgRBy_XDhXXDq_CQ","param":"?vc=ivWxqY","pageUrl":""}, response=0
[2021-10-23 14:00:19.820][Trace][3266][k7176d6x] RTC play webrtc://192.168.0.225/quick/ClpheNxgRBy_XDhXXDq_CQ?vc=ivWxqY, api=http://192.168.0.225:1985/rtc/v1/play/?vc=ivWxqY, tid=12130e2, clientip=192.168.0.23, app=quick, stream=ClpheNxgRBy_XDhXXDq_CQ, offer=5210B, eip=, codec=, srtp=, dtls=
[2021-10-23 14:00:19.820][Trace][3266][k7176d6x] ignore attribute=, value=
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] RTC player nack=1, nnc=1
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] RTC connection player gcc=3
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] RTC: Init tracks {track: 6a03d261-f835-42c5-8451-fde672ae5d09, is_active: 0=>1},{track: d3073ca6-409d-4b1b-8081-a7c9d72df828, is_active: 0=>1}, ok
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] Best matched ip=192.168.0.225, ifname=enp2s0
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] Best matched ip=192.168.100.118, ifname=wlo1
[2021-10-23 14:00:19.823][Trace][3266][k7176d6x] RTC: Use candidates 192.168.0.225, 192.168.100.118
[2021-10-23 14:00:19.824][Trace][3266][k7176d6x] RTC init session, user=502t9716:mi1+, url=/quick/ClpheNxgRBy_XDhXXDq_CQ, encrypt=1/1, DTLS(role=passive, version=auto), timeout=30000ms, nack=1
[2021-10-23 14:00:19.824][Trace][3266][k7176d6x] RTC username=502t9716:mi1+, dtls=1, srtp=1, offer=5210B, answer=2005B
[2021-10-23 14:00:19.824][Trace][3266][k7176d6x] RTC remote offer: v=0\r\no=- 981799048898695567 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:mi1+\r\na=ice-pwd:qsEw7sPhB6KuUKYf91S8bLuz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 3D:B4:EA:58:12:55:CB:45:E6:C1:A5:6B:50:57:02:8A:75:E2:7D:D2:4D:C3:16:1E:0D:54:2B:57:EA:78:23:04\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 122 102 121 127 120 125 107 108 109 35 36 124 119 123 118 114 115 116 37\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:mi1+\r\na=ice-pwd:qsEw7sPhB6KuUKYf91S8bLuz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 3D:B4:EA:58:12:55:CB:45:E6:C1:A5:6B:50:57:02:8A:75:E2:7D:D2:4D:C3:16:1E:0D:54:2B:57:EA:78:23:04\r\na=setup:actpass\r\na=mid:1\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:122 VP9/90000\r\na=rtcp-fb:122 goog-remb\r\na=rtcp-fb:122 transport-cc\r\na=rtcp-fb:122 ccm fir\r\na=rtcp-fb:122 nack\r\na=rtcp-fb:122 nack pli\r\na=fmtp:122 profile-id=1\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:121 rtx/90000\r\na=fmtp:121 apt=102\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:120 rtx/90000\r\na=fmtp:120 apt=127\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 goog-remb\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 ccm fir\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=125\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:35 AV1X/90000\r\na=rtcp-fb:35 goog-remb\r\na=rtcp-fb:35 transport-cc\r\na=rtcp-fb:35 ccm fir\r\na=rtcp-fb:35 nack\r\na=rtcp-fb:35 nack pli\r\na=rtpmap:36 rtx/90000\r\na=fmtp:36 apt=35\r\na=rtpmap:124 H264/90000\r\na=rtcp-fb:124 goog-remb\r\na=rtcp-fb:124 transport-cc\r\na=rtcp-fb:124 ccm fir\r\na=rtcp-fb:124 nack\r\na=rtcp-fb:124 nack pli\r\na=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=124\r\na=rtpmap:123 H264/90000\r\na=rtcp-fb:123 goog-remb\r\na=rtcp-fb:123 transport-cc\r\na=rtcp-fb:123 ccm fir\r\na=rtcp-fb:123 nack\r\na=rtcp-fb:123 nack pli\r\na=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=123\r\na=rtpmap:114 red/90000\r\na=rtpmap:115 rtx/90000\r\na=fmtp:115 apt=114\r\na=rtpmap:116 ulpfec/90000\r\na=rtpmap:37 flexfec-03/90000\r\na=rtcp-fb:37 goog-remb\r\na=rtcp-fb:37 transport-cc\r\na=fmtp:37 repair-window=10000000\r\n
[2021-10-23 14:00:19.824][Trace][3266][k7176d6x] RTC local answer: v=0\r\no=SRS/4.0.161(Leo) 45839440 2 IN IP4 0.0.0.0\r\ns=SRSPlaySession\r\nt=0 0\r\na=ice-lite\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS quick/ClpheNxgRBy_XDhXXDq_CQ\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:502t9716\r\na=ice-pwd:7u3b4948w5698w64c60x40f5zw99k071\r\na=fingerprint:sha-256 52:2F:9A:27:AE:A6:67:35:AD:16:C6:A3:F8:B7:2E:AE:57:BE:8F:24:D7:EB:8F:9A:AB:B9:E7:B6:EC:7E:B0:4D\r\na=setup:passive\r\na=mid:0\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=ssrc:32989881 cname:504q030iv51e0o57\r\na=ssrc:32989881 msid:- d3073ca6-409d-4b1b-8081-a7c9d72df828\r\na=ssrc:32989881 mslabel:-\r\na=ssrc:32989881 label:d3073ca6-409d-4b1b-8081-a7c9d72df828\r\na=candidate:0 1 udp 2130706431 192.168.0.225 8000 typ host generation 0\r\na=candidate:1 1 udp 2130706431 192.168.100.118 8000 typ host generation 0\r\nm=video 9 UDP/TLS/RTP/SAVPF 125 114\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:502t9716\r\na=ice-pwd:7u3b4948w5698w64c60x40f5zw99k071\r\na=fingerprint:sha-256 52:2F:9A:27:AE:A6:67:35:AD:16:C6:A3:F8:B7:2E:AE:57:BE:8F:24:D7:EB:8F:9A:AB:B9:E7:B6:EC:7E:B0:4D\r\na=setup:passive\r\na=mid:1\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:114 red/90000\r\na=ssrc:32989882 cname:504q030iv51e0o57\r\na=ssrc:32989882 msid:- 6a03d261-f835-42c5-8451-fde672ae5d09\r\na=ssrc:32989882 mslabel:-\r\na=ssrc:32989882 label:6a03d261-f835-42c5-8451-fde672ae5d09\r\na=candidate:0 1 udp 2130706431 192.168.0.225 8000 typ host generation 0\r\na=candidate:1 1 udp 2130706431 192.168.100.118 8000 typ host generation 0\r\n
[2021-10-23 14:00:19.837][Trace][3266][k7176d6x] TCP: before dispose resource(HttpConn)(0x308ef60), conns=3, zombies=0, ign=0, inz=0, ind=0
[2021-10-23 14:00:19.837][Warn][3266][k7176d6x][104] client disconnect peer. ret=1007
[2021-10-23 14:00:19.837][Trace][3266][e035n59l] TCP: clear zombies=1 resources, conns=3, removing=0, unsubs=0
[2021-10-23 14:00:19.837][Trace][3266][k7176d6x] TCP: disposing #0 resource(HttpConn)(0x308ef60), conns=3, disposing=1, zombies=0
[2021-10-23 14:00:19.847][Trace][3266][k7176d6x] RTC: session address init 192.168.0.23:62858
[2021-10-23 14:00:19.847][Trace][3266][k7176d6x] RTC: session STUN done, waiting DTLS handshake.
[2021-10-23 14:00:19.854][Trace][3266][k7176d6x] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=159, cnt=22, size=146, hs=1
[2021-10-23 14:00:19.855][Trace][3266][k7176d6x] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=679, cnt=22, size=82, hs=2
[2021-10-23 14:00:19.861][Trace][3266][k7176d6x] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=579, cnt=22, size=300, hs=11
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=554, cnt=22, size=466, hs=4
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] RTC: DTLS handshake done.
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] RTC: session pub=0, sub=1, to=30000ms connection established
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] RTC: Subscriber url=/quick/ClpheNxgRBy_XDhXXDq_CQ established
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] create consumer, no gop cache
[2021-10-23 14:00:19.864][Trace][3266][k7176d6x] RTC: start play url=/quick/ClpheNxgRBy_XDhXXDq_CQ, source_id=02247790/dd8947g0, realtime=1, mw_msgs=0
[2021-10-23 14:00:19.932][Warn][3266][02247790][11] drop for reduce sh video, size=36
[2021-10-23 14:00:19.932][Trace][3266][02247790] 36B video sh,  codec(7, profile=Baseline, level=Other, 720x416, 0kbps, 0.0fps, 0.0s)
[2021-10-23 14:00:19.932][Trace][3266][02247790] set ts=3843957618047, header=30289, lost=30290
[2021-10-23 14:00:20.051][Trace][3266][79964353] TCP: before dispose resource(HttpConn)(0x31b0a40), conns=2, zombies=0, ign=0, inz=0, ind=0
[2021-10-23 14:00:20.051][Trace][3266][79964353] client finished.
[2021-10-23 14:00:20.051][Trace][3266][e035n59l] TCP: clear zombies=1 resources, conns=2, removing=0, unsubs=0
[2021-10-23 14:00:20.051][Trace][3266][79964353] TCP: disposing #0 resource(HttpConn)(0x31b0a40), conns=2, disposing=1, zombies=0
[2021-10-23 14:00:21.051][Trace][3266][424o1743] RTC: Need PLI ssrc=4131953666, play=[424o1743], publish=[02247790], count=53/53
[2021-10-23 14:00:21.051][Trace][3266][02247790] RTC: Request PLI ssrc=4131953666, play=[424o1743], count=53/53, bytes=12B
[2021-10-23 14:00:21.076][Warn][3266][02247790][11] drop for reduce sh video, size=36
[2021-10-23 14:00:21.077][Trace][3266][02247790] 36B video sh,  codec(7, profile=Baseline, level=Other, 720x416, 0kbps, 0.0fps, 0.0s)
[2021-10-23 14:00:21.077][Trace][3266][02247790] set ts=3843957619183, header=30326, lost=30327
[2021-10-23 14:00:21.336][Trace][3266][43u26x95] <- RTC RECV #11, udp 808, pps 5/80, schedule 808
[2021-10-23 14:00:21.573][Trace][3266][02247790] -> HLS time=536463786ms, sno=87, ts=ClpheNxgRBy_XDhXXDq_CQ-86.ts, dur=0.00, dva=512p
[2021-10-23 14:00:22.850][Trace][3266][424o1743] Hybrid cpu=4.00%,24MB, cid=320,54, timer=62,10,63, clock=0,38,9,1,0,0,0,0,0, free=1, objs=(pkt:128,raw:108,fua:18,msg:210,oth:1,buf:77)
[2021-10-23 14:00:22.850][Trace][3266][424o1743] RTC: Server conns=2, rpkts=(80,rtp:77,stun:1,rtcp:1), spkts=(36,rtp:20,stun:1,rtcp:24), rtcp=(pli:1,twcc:9,rr:1), snk=(96,a:48,v:48,h:0), fid=(id:1,fid:80,ffid:0,addr:1,faddr:80)
[2021-10-23 14:00:23.133][Warn][3266][02247790][11] drop for reduce sh video, size=36
[2021-10-23 14:00:23.133][Trace][3266][02247790] 36B video sh,  codec(7, profile=Baseline, level=Other, 720x416, 0kbps, 0.0fps, 0.0s)
[2021-10-23 14:00:23.133][Trace][3266][02247790] set ts=3843957621247, header=30381, lost=30382
[2021-10-23 14:00:24.213][Warn][3266][43u26x95][11] handle udp pkt, count=11950/11950, err: code=5011 : size=112, data=[90 6f 71 c0 19 b8 58 dd] : cipher=112, plaintext=102, body=[be de 00 01 31 29 73 00] : on audio : source on rtp : bridger consume message : decode and resample : submit to dec(-541478725,End of file)
thread [3266][43u26x95]: cycle() [src/app/srs_app_listener.cpp:630][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_conn.cpp:1388][errno=11]
thread [3266][02247790]: do_on_rtp_plaintext() [src/app/srs_app_rtc_conn.cpp:1441][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_source.cpp:2415][errno=11]
thread [3266][02247790]: on_rtp() [src/app/srs_app_rtc_source.cpp:630][errno=11]
thread [3266][02247790]: transcode() [src/app/srs_app_rtc_codec.cpp:145][errno=11]
thread [3266][02247790]: decode_and_resample() [src/app/srs_app_rtc_codec.cpp:314][errno=11]

Replay

How to replay bug?

Steps to reproduce the bug

Steps to reproduce the bug:

  1. Use a web page to push the stream using WebRTC.
  2. Play the corresponding RTMP connection using VLC.
  3. When pulling the stream through WebRTC, it is consistently normal with audio and smooth video. However, when playing through the RTMP URL using VLC, sometimes there is no audio and severe stuttering.

Expected behavior:

The expected behavior is that both the WebRTC stream and the RTMP stream should have consistent audio and smooth playback without any stuttering.

RTMP playback:
The AAC audio frames are transmitted normally, and the video playback is smooth.

TRANS_BY_GPT3
@winlinvip
Copy link
Member

winlinvip commented Oct 25, 2021 •
edited
Loading

Which browser is used for streaming? Can it be reliably reproduced? Specifically, which version is it, 4.0 or higher?

TRANS_BY_GPT3

@winlinvip winlinvip changed the title rtc-rtmp场景下，一些时候音频会出现解码失败导致rtmp没有任何音频帧 rtc2rtmp：有时音频会出现解码失败，导致rtmp没有任何音频帧 Oct 25, 2021
@winlinvip winlinvip self-assigned this Oct 25, 2021
@winlinvip winlinvip added the WebRTC WebRTC, RTC2RTMP or RTMP2RTC. label Oct 25, 2021
@winlinvip winlinvip added this to the 5.0 milestone Oct 25, 2021
@jamken
Copy link
Contributor Author

jamken commented Oct 26, 2021 •
edited by winlinvip
Loading

The specific version is 4.0.161. From the error message, we deduce that in certain situations, the audio frame data has a length of 0. If such a packet is sent to the decoder, the decoder will consider it as the end of decoding, and sending subsequent audio frames will result in an EOF error. Currently, we have implemented a simple patch to protect against this issue, and the aforementioned error with the audio frames has not occurred again. Attached are the details of the patch.

diff --git "a/trunk/src/app/srs_app_rtc_codec.cpp" "b/trunk/src/app/srs_app_rtc_codec.cpp"
index a8219a539..f7e4caaee 100644
--- "a/trunk/src/app/srs_app_rtc_codec.cpp"
+++ "b/trunk/src/app/srs_app_rtc_codec.cpp"
@@ -324,6 +324,12 @@ srs_error_t SrsAudioTranscoder::decode_and_resample(SrsAudioFrame *pkt)
 
     char err_buf[AV_ERROR_MAX_STRING_SIZE] = {0};
 
+    if (dec_packet_->data == NULL || dec_packet_->size == 0){
+        return srs_error_new(ERROR_RTC_INVALID_PARAMS, 
+                             "dec_pakcet is invalide(%p,%d)", 
+                             dec_packet_->data, dec_packet_->size);	
+    }
+
     int error = avcodec_send_packet(dec_, dec_packet_);
     if (error < 0) {
         return srs_error_new(ERROR_RTC_RTP_MUXER, "submit to dec(%d,%s)", error,

TRANS_BY_GPT3

@jamken
Copy link
Contributor Author

jamken commented Oct 26, 2021 •
edited by winlinvip
Loading

After solving this problem, the next phenomenon is that the audio and video of the rtmp stream are both noticeably lagging. I suspect it may be a timestamp issue and further analysis is needed.

TRANS_BY_GPT3

shinoi2 pushed a commit to shinoi2/srs that referenced this issue Nov 10, 2021
For ossrs#2690, empty package cause audio codec error.
213e45f
jamken pushed a commit to jamken/srs that referenced this issue Nov 29, 2021
fix ossrs#2690, 如果发现音频包为空，则不送解码，以免提前结束了解码上下文
f3823c8
@jamken jamken mentioned this issue Nov 29, 2021
Merged
jamken pushed a commit to jamken/srs that referenced this issue Dec 1, 2021
fix ossrs#2690，出错信息中，显示相关参数名称
0d0fe12
winlinvip pushed a commit that referenced this issue Dec 4, 2021
@jamken
fix #2690, 如果发现音频包为空，则不送解码，以免提前结束了解码上下文 (#2757)
2c0cd9b 
* fix #2690, 如果发现音频包为空，则不送解码，以免提前结束了解码上下文

* fix #2690，出错信息中，显示相关参数名称

Co-authored-by: jiankai <wangjiankai@cmhi.chinamobile.com>
@winlinvip
Copy link
Member

@jamken 👍 Thanks

peipeiguo pushed a commit to peipeiguo/srs that referenced this issue Jun 6, 2022
@jamken @peipeiguo
fix ossrs#2690, 如果发现音频包为空，则不送解码，以免提前结束了解码上下文 (ossrs#2757)
e6130de 
* fix ossrs#2690, 如果发现音频包为空，则不送解码，以免提前结束了解码上下文

* fix ossrs#2690，出错信息中，显示相关参数名称

Co-authored-by: jiankai <wangjiankai@cmhi.chinamobile.com>
@winlinvip winlinvip changed the title rtc2rtmp：有时音频会出现解码失败，导致rtmp没有任何音频帧 rtc2rtmp: Sometimes there may be a decoding failure in the audio, resulting in no audio frames in the rtmp. Jul 28, 2023
@winlinvip winlinvip added the TransByAI Translated by AI/GPT. label Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@winlinvip winlinvip

Labels
TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.
Projects
None yet
Milestone
5.0
2 participants
@winlinvip @jamken