Playing simultaneous tracks with media 3

[zoomGitS]

Hi, I want to play multiple audio tracks simultaneously. I know that we can use multiple ExoPlayer instances, but I prefer using a single instance and customizing it to play multiple audio files. I found some information on how to achieve this, but it seems to apply to older versions of ExoPlayer. The solution suggested modifying the RenderersFactory set on the player to include multiple audio renderers, and adjusting the TrackSelector to assign each audio track to a different renderer.

However, this approach doesn't seem to work with the new Media3 library. Can someone help me with this question? Thanks!

[tonihei]

If you are referring to comment like this one, I think the approach should still work as the underlying code hasn't changed in any significant ways since.

There is also ongoing work to allow proper audio mixing for playback of multiple tracks, but this may not be available soon.

Could you clarify in more detail what exactly isn't working and maybe we can point you in the right direction?

[zoomGitS]

//initilize

   private val exoPlayer by lazy {
        ExoPlayer.Builder(this, renderersFactory).setTrackSelector(trackSelector).build()
    }

//create media sources

val audioThreeFile = File(File(filesDir, "audio_test"), "audio_one_trimed.mp3")
        val audioFourFile = File(File(filesDir, "audio_test"), "auudio_two_trimed.mp3")

        val defaultDataSourceFactory = DefaultDataSource.Factory(applicationContext)

        val sourceOne = ProgressiveMediaSource.Factory(defaultDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.fromFile(audioThreeFile)))
        val sourceTwo = ProgressiveMediaSource.Factory(defaultDataSourceFactory)
            .createMediaSource(MediaItem.fromUri(Uri.fromFile(audioFourFile)))

        val mergingMediaSource = MergingMediaSource(true, sourceOne, sourceTwo)


        exoPlayer.setMediaSource(mergingMediaSource)
        exoPlayer.prepare()
        exoPlayer.playWhenReady = true

//create renderersFactory

private val renderersFactory = object : DefaultRenderersFactory(this) {
        override fun buildAudioRenderers(
            context: Context,
            extensionRendererMode: Int,
            mediaCodecSelector: MediaCodecSelector,
            enableDecoderFallback: Boolean,
            audioSink: AudioSink,
            eventHandler: Handler,
            eventListener: AudioRendererEventListener,
            out: ArrayList<Renderer>
        ) {
            super.buildAudioRenderers(
                context,
                extensionRendererMode,
                mediaCodecSelector,
                enableDecoderFallback,
                audioSink,
                eventHandler,
                eventListener,
                out
            )
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
            out.add(AudioRendererWithoutClock(context, mediaCodecSelector))
        }
    }

//create trackSelector

private val trackSelector = object : TrackSelector() {
        override fun selectTracks(
            rendererCapabilities: Array<out RendererCapabilities>,
            trackGroups: TrackGroupArray,
            periodId: MediaSource.MediaPeriodId,
            timeline: Timeline
        ): TrackSelectorResult {
            val audioRenderers: Queue<Int> = ArrayDeque()
            val configs = arrayOfNulls<RendererConfiguration>(rendererCapabilities.size)
            val selections = arrayOfNulls<ExoTrackSelection>(rendererCapabilities.size)
            for (i in rendererCapabilities.indices) {
                if (rendererCapabilities[i].trackType == C.TRACK_TYPE_AUDIO) {
                    audioRenderers.add(i)
                    configs[i] = RendererConfiguration.DEFAULT
                }
            }
            for (i in 0 until trackGroups.length) {
                if (MimeTypes.isAudio(trackGroups[i].getFormat(0).sampleMimeType)) {
                    val index = audioRenderers.poll()
                    if (index != null) {
                        selections[index] = FixedTrackSelection(trackGroups[i], 0)
                    }
                }
            }
            return TrackSelectorResult(configs, selections, Any())
        }

        override fun onSelectionActivated(info: Any?) {

        }
    }

//create AudioRenderer

final class AudioRendererWithoutClock extends MediaCodecAudioRenderer {
    public AudioRendererWithoutClock(Context context, MediaCodecSelector mediaCodecSelector) {
        super(context, mediaCodecSelector);
    }

    @Override
    public MediaClock getMediaClock() {
        return null;
    }
}

But getting this error-

[zoomGitS]

Maybe the issue lies in the RenderersFactory where I am adding three new AudioRendererWithoutClock instances. However, I actually only need one additional AudioRendererWithoutClock to render the second audio.

[tonihei]

Thanks for highlighting the problem! This is indeed related to the additional renderers. Having more renderers than needed is not usually an issue, but there is a bug in the TrackSelector code I provided in the GitHub issue. The configs[i] = RendererConfiguration.DEFAULT line needs to move further down to where we decide which renderer to use. I fixed my original post in google/ExoPlayer#6589 (comment) too to avoid further confusion. Note that this is still just an example though and it wouldn't handle other renderer types like video.

[tonihei]

Also added the fail-early checks in the commits above. Closing the issue.