Single App Package Screen Mirroring instead of whole screen

[Genxster1998]

• [* ] I have checked that a similar feature request does not already exist.

Is your feature request related to a problem? Please describe.
No, but feature enhancement. Instead of using single screen i would like to mirror different app windows in free form mode independently of main canvas.

Describe the solution you'd like
an option like passing package=com.android.contacts to mirror specific app only screen instead of whole ui.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[guanzhangrtk]

If I understand the feature request correctly, you are asking for app-specific screens to be mirrored. If this indeed can be implemented I wonder if that means we can have different scrcpy windows per app so that you can sort of run and control multiple apps at once?

[Genxster1998]

@guanzhangrtk yepp! Totally right , but I don't know it's possible to spawn single app window screen without system am policies implementation ?
Like clicking an app in special launcher app leads to open in new scrcpy window .

[guanzhangrtk]

Yeah I don't know if it is possible but I know that with MuMu App Player (android emulator) you can open multiple apps and then drag them off the main window and they would continue running in the background and you switch between them as if they're both running simultaneously. This is the only android emulator I'm aware of that has this feature (at least on the Mac).

[hltdev8642]

It isnt really a solution, but using a split terminal with windows terminal and termux over usb/adb is pretty effective.

Also, made a .run alias that generates a dialog list of apps and you can launch them. (can post that if anyone wants it).

I get it though, it would be great to be able to launch individual apps kind of like the way an x11 GUI app runs in wsl or something...

[guanzhangrtk]

@hltdev8642 What's the benefit of doing this instead of just running adb shell directly from Windows?

[Genxster1998]

You could have running shell + another app using adb shell /ssh but that's not what i am seeking for , instead modifying surfaceflinger behaviour to send different apps as seperate window like different Wayland clients and scrcpy attaching itself to windows not whole display ie. display 0 , display 1 .which also need changes in hwc to paint windows surface and launcher surface + status bar + nav bar differently by id .

[4nric]

I don't know if this is the solution that you are looking for. On Android 10+, the desktop mode can be forced on secondary displays and secondary displays can be simulated. Here's a simple PowerShell script (.ps1 file) that launches Chrome at the simulated display in fullscreen then opens the simulated display in scrcpy, also in fullscreen. The simulated display is automatically destroyed on scrcpy exit:

$sdk = adb shell getprop ro.build.version.sdk 

if ($sdk -ge 29){
    adb shell settings put global force_resizable_activities 1
    adb shell settings put global enable_freeform_support 1
    adb shell settings put global force_desktop_mode_on_external_displays 1
    
    if ($sdk -ge 30){
        adb shell settings put global enable_sizecompat_freeform 1
    }

    adb shell settings put global overlay_display_devices 1920x1080/160
    $disp = adb shell dumpsys display | sls "mDisplayId=" | select -last 1
    $disp = "$disp" | sls "\d+" | % {$_.Matches} | % {$_.Value}
    adb shell am start-activity -S --display $disp --windowingMode 1 com.android.chrome
    scrcpy --display $disp --forward-all-clicks -f
    
    adb shell settings put global overlay_display_devices none
}
else {
    "Android 10+ required"
}

[guanzhangrtk]

So you can have an app running on the secondary display and then still have access to the main display via scrcpy? How many secondary displays can one have running simultaneously?

[4nric]

If --display is not explicitly specified, scrcpy will mirror the main display which has an id of 0. So yes, you can still mirror the main display through scrcpy --display 0 or simply scrcpy.

You can have 4 secondary displays running simultaneously. Every time the adb overlay_display_devices is run, previous virtual displays, if any, will be closed (apps moved to main display) and new one/s with greater display id number/s will be generated. Simply running my sample above won't simply generate a new display. You need to explicitly specify how many displays you want to simulate. Also, you'll have an overlay for each on the main display which may block certain screen elements. There's no way to hide the overlays currently to my best knowledge.

Sample adb command for 4 simulated displays:
adb shell settings put global overlay_display_devices "1920x1080/160\;1920x1080/160\;1920x1080/160\;1920x1080/160\"
If you have a Samsung device, it's best that you install Taskbar and set it as a default launcher. This will prevent One UI Home from constantly crashing, then crash your phone. Though tasker may be installed with desktop mode enabled, apps may not always launch on the specific display. Tasker has to implement this yet. Tasker will work fine on a single simulated display. For now, we are stuck on manually opening an app to a specific display through adb.

To list the available displays:
adb shell dumpsys display | findstr "mDisplayId="

Sample to open an app on a specific display:
adb shell am start-activity --display 12 --windowingMode 1 com.android.chrome

[rom1v]

@4nric Thank you for the details to start a new display.

adb shell settings put global overlay_display_devices 1920x1080/160

On my OnePlus 7 Pro (Android 10), it starts a new display overlay visible on the main screen on the device (as you say). But on scrcpy, it shows a black screen (having the correct size). It's the same as when I enable a secondary display from developer options. I guess this is a ROM issue/limitation?

[guanzhangrtk]

Guys this is all pretty cool discussion, but too bad it requires Android 10+. I wonder if it is possible to have backward compatibility. MuMu App Player is able to have split app displays and it's running Android 6.0. It uses Qt as far as I know but there must be some support from Android for it to work... it's possible it's not using adb though. Here's a screenshot on the Mac version showing that I am running three apps simultaneously and they each have their own separate window. Sorry but I have no idea how it is implemented I just know it is possible, thanks :)

[4nric]

@rom1v Have you enabled Force Desktop mode first before simulating the secondary display? Force Desktop mode has to be enabled first. Force Desktop mode can be enabled from developer settings or by running the adb command adb shell settings put global force_desktop_mode_on_external_displays 1. If you already did, does the new display overlay on the main device show the secondary display UI or just black screen? Here's what the overlay looks like on my Samsung with Taskbar set as the default launcher and desktop mode support enabled within its settings:

I don't have other Android 10 devices to test with, unfortunately. I'm guessing System UI com.android.systemui needs to be restarted for desktop mode to work properly?

From an xda article, though specific for freeform:

Both of these methods require the System UI to be restarted before they can take effect. The easiest way to do this is to reboot your device (or, if your device is rooted, you can simply kill the com.android.systemui process)

[4nric]

MuMu App Player is able to have split app displays and it's running Android 6.0

@guanzhangrtk I don't use the emulator, but I am certain that the ROM it runs is modified to enable that behavior— just like other device manufacturers can modify AOSP to add their own skins (i.e One UI, MIUI, EMUI) and additional features (i.e Samsung DeX since Android 7.0, EMUI Desktop). Stock Android 6.0 only has experimental multiwindow. Later versions do have experimental freeform windows (multiple apps on same display). Only in Android 10+ that the experimental desktop mode option is available (open multiple apps not only on main display, but on other available ones as well).

[4nric]

I guess this is a ROM issue/limitation?

If nothing works, this probably is a ROM limitation, unfortunately. Maybe the implementation of Wireless DeX somehow influenced this behavior? I have yet to try other Samsung devices (non-flagship) with Android 10+

[rom1v]

Have you enabled Force Desktop mode first before simulating the secondary display?

Oh, I didn't. Now it works. Thank you 👍

So to summarize:

# enable a secondary display (2560x1440, dpi=160, use a higher value for bigger icons)
adb shell settings put global overlay_display_devices 2560x1440/160

# run on the secondary display (if the id does not exist, scrcpy will print available display ids)
# (set the bitrate to 16Mbps for better quality)
scrcpy -b16M --display=1

# disable secondary displays
adb shell settings delete global overlay_display_devices

To enable freeform window mode:

# enable freeform support
adb shell settings put global enable_freeform_support 1  # then reboot the device

(It could be added in the README I guess)

Some feedbacks: in practice (at least on my device), it's not very convenient for daily usage:

• the secondary display "overlay" is visible on the main device
• if an app is started on the main display, opening it from the secondary display launcher does nothing (it's already open, but not visible on the computer, which is confusing)
• dpi has not the same effects on all apps (for example, on Firefox, the text is big, while it's ok in some other apps)
• freeform window does not work for all apps
• when it works, the window has a given size, but cannot be resized (or I didn't find how)

[Genxster1998]

@guanzhangrtk I 99 % doubt that is another window but just another full android instance so too much overhead !
Also another simulated display rendering could also be burdening on GPU if it's like 5+(just random guess) but manageable I think , I haven't check GPU cpu stats using that yet .anybody ? There's display_config.xml like some config I think exit on /data/system?

[guanzhangrtk]

@guanzhangrtk I 99 % doubt that is another window but just another full android instance so too much overhead !

MuMu doesn't support multi-instance. Also if each window is a separate full instance then the apps wouldn't be able to share services and interop with each other (but they do).

[luke-jr]

am start-activity -S --display $disp --windowingMode 1 com.android.chrome

This doesn't seem to work for me... :/

# am start-activity -S --display 4 --windowingMode 1 com.android.chrome                                                                
Error: Intent does not match any activities: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.chrome }

[luke-jr]

Leaving off the -S works to launch it, but scrcpy can't interact with anything (mouse/keyboard actions are ignored), and it doesn't work for other apps (eg, Twitter)

[hltdev8642]

am start-activity -S --display $disp --windowingMode 1 com.android.chrome

This doesn't seem to work for me... :/

# am start-activity -S --display 4 --windowingMode 1 com.android.chrome                                                                
Error: Intent does not match any activities: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] pkg=com.android.chrome }

its a longshot, but try adding the flag --user 0 into that command

[luke-jr]

Didn't help

[4nric]

Leaving off the -S works to launch it, but scrcpy can't interact with anything (mouse/keyboard actions are ignored), and it doesn't work for other apps (eg, Twitter)

@luke-jr what is your device and what version of android are you on?

[luke-jr]

lineage-16.0-20190811-nightly-marlin-signed.zip I think (original Pixel XL)

[Genxster1998]

@hltdev8642 how about specifying main activity directly instead of specifying only package name like

org.chromium.chrome.browser.ChromeTabbedActivity

Like this

[4nric]

lineage-16.0-20190811-nightly-marlin-signed.zip I think (original Pixel XL)

LineageOS 16 is based on Android 9 Pie. I have yet to confirm if apps can be launched on simulated displays with Android 9 Pie and below. The "force desktop mode on external displays" option is also only available on Android 10+

To my best knowledge, scrcpy block controls on secondary displays if it detects the connected device is running Android 9 Pie and below.

If you mirrored only your main display (no --display #) and still can't get touch working, the problem could be related to something else.

With regards to launching Chrome, can you please try if this adb command works?
adb shell 'am start -n $(cmd package resolve-activity --brief com.android.chrome | tail -n 1)'

[luke-jr]

To my best knowledge, scrcpy block controls on secondary displays if it detects the connected device is running Android 9 Pie and below.

Why does it do that?

With regards to launching Chrome, can you please try if this adb command works?

Yes, it seems to.

[huynhtanloc2612]

As #1887 (comment)
This video is the idea which I mentioned in above comments to create an virtual display by Android app and use scrcpy tool to display and interact with it on PC (Hope this gives scrcpy developers some ideas and other users have a temporary solution).
Video: https://youtu.be/3AOxgzPi3dw?si=lBtHN3BFaoTewSzD
Android app: https://play.google.com/store/apps/details?id=com.htl.agdisplays

[astroskyoffical]

As #1887 (comment) This video is the idea which I mentioned in above comments to create an virtual display by Android app and use scrcpy tool to display and interact with it on PC (Hope this gives scrcpy developers some ideas and other users have a temporary solution). Video: https://youtu.be/3AOxgzPi3dw?si=lBtHN3BFaoTewSzD Android app: https://play.google.com/store/apps/details?id=com.htl.agdisplays

Exploring the use of desktop mode on Android devices reveals a mixture of potential and frustration. The journey led me to experiment with an HDMI dummy plug, a clever workaround that simulates a display connection, bypassing the need for direct physical output. This trick opens up possibilities for using SCRCPY more seamlessly, aiming to cast the desktop mode to laptops or other screens without the clutter of on-screen overlays or convoluted workarounds.

However, the solution is not without its drawbacks. The app intended to facilitate this process is currently buggy. At times, attempts to push the display result in nothing being shown; other times, the graphics appear distorted, far from the crisp, clean output desired. This inconsistency begs for a deeper dive into the code behind these operations. Is there a way to refine the app, to offer an option that directly supports second-screen desktop mode, bypassing per-app selection?

The current workaround, while functional, highlights a gap in the user experience. Ideally, an app would manage the complexity internally, executing ADB commands within a virtual environment to mirror the display without additional hardware. This approach would not only streamline the process but also encapsulate the functionality in a more user-friendly manner.

The requirement for an overlay on Android during these operations is puzzling. Why is this overlay mandatory, and why is it so challenging to hide or bypass it? It seems there should be a way to circumvent this, to make the transition to desktop mode more seamless and integrated. The exploration brought me to Miracast, which offers a partial solution by enabling casting to a new display, thus eliminating the overlay. But this, too, feels like a workaround rather than a direct solution.

In essence, the journey to optimize desktop mode on Android devices, to enable easy casting to other devices without the overlays

[astroskyoffical]

I'm having this one issue and idk how to make it stop. I'd like to go into desktop mode and when I open a gain it full screen but with out the black bar on the top that minizes or fullscreens it or go back button. I want like exclusive fullescreen. But for the life of me can't get this to work. Is there a command I need? I'm certain it's a window mode I need to enable. I disabled freeform windows. I disabled everything except desktop mode but all apps keep opening like this. Only the YouTube app seems to open and makes it become transparent which makes it much much better but clearly this is not system wide. Any advice?

[huynhtanloc2612]

I'm having this one issue and idk how to make it stop. I'd like to go into desktop mode and when I open a gain it full screen but with out the black bar on the top that minizes or fullscreens it or go back button. I want like exclusive fullescreen. But for the life of me can't get this to work. Is there a command I need? I'm certain it's a window mode I need to enable. I disabled freeform windows. I disabled everything except desktop mode but all apps keep opening like this. Only the YouTube app seems to open and makes it become transparent which makes it much much better but clearly this is not system wide. Any advice?

I am assuming that you want to open an app in window form in desktop mode, right? I suggest you starting an activity with windowingMode 5 or 6 as below command. You will see title bar with these buttons (it works for Samsung devices)

adb shell am start-activity -S --display <Dis
playId> --windowingMode 5 <Activity>

[astroskyoffical]

I'm having this one issue and idk how to make it stop. I'd like to go into desktop mode and when I open a gain it full screen but with out the black bar on the top that minizes or fullscreens it or go back button. I want like exclusive fullescreen. But for the life of me can't get this to work. Is there a command I need? I'm certain it's a window mode I need to enable. I disabled freeform windows. I disabled everything except desktop mode but all apps keep opening like this. Only the YouTube app seems to open and makes it become transparent which makes it much much better but clearly this is not system wide. Any advice?

I am assuming that you want to open an app in window form in desktop mode, right? I suggest you starting an activity with windowingMode 5 or 6 as below command. You will see title bar with these buttons (it works for Samsung devices)

adb shell am start-activity -S --display <Dis
playId> --windowingMode 5 <Activity>

Hi sorry I was half asleep when I wrote this. This is what I'm referring to.

I'm using a none Samsung device with custom rom to get HDMI out working again as OnePlus decided it was a fun idea to disable HDMI out on most of their roms. Running evolution x android 14 with root. I'm pretty close to the dex experience already. I have disabled freeform windows but it doesn't seem to be sticking. I had another dev show me his emulated android 14 running as intended like I want with freeform disabled. It seems my freeform is sticking no matter what the dev options say.

[oxwivi]

Did anyone figure out how to create a virtual screen without it showing up on the actual display?

Also, how do you unlock Android within the virtual screen itself? If I connect to locked phone, I get a gray background with a lock icon that changes position now and then. Can't click, gesture or otherwise interact with it. Only goes away after I unlock from main display, then use as normal.

[huynhtanloc2612]

As @yume-chan mentioned in #1887 (comment)

Is it possible to accomplete it with standalone java program running on app_process

Yes

I have made the standalone java program which based on @yume-chan 's code to create an virtual display and added/called it in my Android app which was mentioned in #1887 (comment) .
Btw, this only works on Android 13 and newer version.

And #1887 (comment)

Because Scrcpy client uses pure SDL without any UI library, I can't decide how the client side should work. Now I only implemented the support in my Web ADB project (https://app.tangoapp.dev/). It's much easier to add more complex UI into a Web app.

I also made an simple apps launcher in Desktop mode with other activity which can support scrcpy client tool of @rom1v

Demo video: https://youtu.be/ysv5OL3IL1w

For anyone who needs Desktop mode, please try it. Hope it help. Thanks

[sidevesh]

@huynhtanloc2612 thanks for all the work you have put into this, amazing stuff!
Is there a way to wire this app with scrcpy in a way that we can launch scrcpy with a particular package or app name specified and it creates a virtual display for that app and it shows up on desktop as a window ?

I am basically trying to recreate the experience that Windows Phone Companion has with Samsung phones where you can see a list of all apps on your phone and launch them from desktop in separate windows.

[huynhtanloc2612]

Thanks @sidevesh

Is there a way to wire this app with scrcpy in a way that we can launch scrcpy with a particular package or app name specified and it creates a virtual display for that app and it shows up on desktop as a window ?

It is quite difficult for me because I am not familiar with internal API which is used in scrcpy project (even I have been trying to learn them from scrcpy code). I think scrcpy developers can help do this.

[Gentleman36]

adb shell settings put global force_desktop_mode_on_external_displays

@4nric How to use adb command to create more than 4 monitors?

[oxwivi]

I have made the standalone java program which based on @yume-chan 's code to create an virtual display and added/called it in my Android app which was mentioned in #1887 (comment) .

I've used your AG Displays app for a day or so, and I couldn't get the Desktop display to take any touch input (using the mouse) except for the middle-click and right-click for home/back. I could use the keyboard to move around the app drawer and launch apps, but of course that's not very useful if I can't interact with said apps. I've suspect it's to do with the virtual "touchscreen" not aligned correctly with the virtual display.

Regardless, can you tech me how to crete the virtul screen using adb nd not show up on the physicl screen? That's the bre minimum I'm looking for.

[huynhtanloc2612]

I have made the standalone java program which based on @yume-chan 's code to create an virtual display and added/called it in my Android app which was mentioned in #1887 (comment) .

I've used your AG Displays app for a day or so, and I couldn't get the Desktop display to take any touch input (using the mouse) except for the middle-click and right-click for home/back. I could use the keyboard to move around the app drawer and launch apps, but of course that's not very useful if I can't interact with said apps. I've suspect it's to do with the virtual "touchscreen" not aligned correctly with the virtual display.

Regardless, can you tech me how to crete the virtul screen using adb nd not show up on the physicl screen? That's the bre minimum I'm looking for.

I believe that you are facing the issue
of #4598 which has been discussing with @rom1v for Android 14 devices.
In the mean time. You can try other phones with Android 13.

[oxwivi]

I believe that you are facing the issue of #4598 which has been discussing with @rom1v for Android 14 devices. In the mean time. You can try other phones with Android 13.

Sorry, it looks I forgot to mention. Yes, it's Android 14 (LineageOS 21). But the main thing is, if I manually create the display, using adb (adb shell settings put global overlay_display_devices 1920x1080/120), I don't face this problem at all. I can click, open, interact with the display and any app that I open in it, completely fine.

With manually creating the virtual screen, I've other problems:

• I can't unlock it from the virtual display, can only do it from the main.
• And also that it always shows on the main display always.

Since this is working for me, I'm asking you how to manually create this virtual display without it showing on the main screen all the time. This is the main issue I have with manually creating virtual display, otherwise I'm happy with the experience. I hope scrcpy can actually provide arguments to automatically create and destroy virtual displays in the future.

[huynhtanloc2612]

They are different display types.

But the main thing is, if I manually create the display, using adb (adb shell settings put global overlay_display_devices 1920x1080/120)

This display which is created by above command is "real" display but "virtual" display. The system simulates the real display by using your phone physical screen. It is similar to that you connect your phone (Android 14) with a monitor/TV, the display id is created. Then you can use scrcpy to mirror it via the display id. You will see all click events,... work as expectation. The monitor/TV is real display type. But they require physical screens.

I believe that you are facing the issue
of #4598 

The issue only happens with virtual display type (from Android 14) which the app and java program are using.

[oxwivi]

This display which is created by above command is "real" display but "virtual" display.

Very sorry for replying so much time later. So I understand your app doesn't use the adb shell settings put global overlay_display_devices command, but what does it use instead?

[mavink]

Does this mean that for apps which don't allow screenshots:
With shizuku adb permissions

1. Create a virtual display
2. Launch app on virtual display
3. Mirror the virtual display to an app running on the main display (i.e. run srccpy on the phone itself)
4. Take a screen shot of mirroring app in step 3 the main display, hence screenshot disabling is bypassed?

[mavink]

Does this mean that for apps which don't allow screenshots: With shizuku adb permissions

1. Create a virtual display

2. Launch app on virtual display

3. Mirror the virtual display to an app running on the main display (i.e. run srccpy on the phone itself)

4. Take a screen shot of mirroring app in step 3 the main display, hence screenshot disabling is bypassed?

Even Samsung dex is not able to mirror protected apps like incognito mode... so I guess this won't work

[StraggleCraft]

@yume-chan are the changes that permit --video-source=virtual going to be brought into the main scrcpy release or was this only a proof of concept?

[yume-chan]

@yume-chan are the changes that permit --video-source=virtual going to be brought into the main scrcpy release or was this only a proof of concept?

No. --video-source=virtual by itself is not useful. It requires some UI to launch, switch and manage apps on the virtual display.

[anirudhb]

I've created a proof of concept that creates a resizable virtual display (independent of @yume-chan's changeset): https://github.com/anirudhb/scrcpy/tree/virtual-display

It still needs a lot of cleanup before it can be integrated, and is quite hacky (segfaults on quit), but it works. I would appreciate any feedback that would help get this into scrcpy.

[rsanjuan87]

I've created a proof of concept that creates a resizable virtual display (independent of @yume-chan's changeset): https://github.com/anirudhb/scrcpy/tree/virtual-display

It still needs a lot of cleanup before it can be integrated, and is quite hacky (segfaults on quit), but it works. I would appreciate any feedback that would help get this into scrcpy.

@anirudhb just now, I end reading this, and your branch is great, even I have to upgrade the version to 2.5 of the server
with an emulator works fine, and opens a new virtual desktop there I open the app
and as @huynhtanloc2612 says on my MIUI 14 (Android 13) device, the system launcher is actually running on the virtual display, but it just displays nothing. its MiUI launcher (Redmi device my case)

but with
adb shell am start -n com.android.chrome/com.google.android.apps.chrome.Main --display X (you have to set the display number, opens chrome

but if desktop mode on developer options opens in a 💩 way (Xiaomi fault)

[rsanjuan87]

scrcpy like casting "single app" with virtual-display of @anirudhb and FUP work in progress

Grabacion.de.pantalla.2024-08-01.a.las.10.49.49.mp4