File system restrictions on Android 11

[mvglasow]

After upgrading to Android 11, Navit is no longer seeing any files (navit.xml or map files) in /sdcard/Android/data/org.navitproject.navit/files. This has still worked in Android 10.

The logcat shows this message:

W itproject.navit: type=1400 audit(0.0:479): avc: denied { read } for name="sdcard" dev="tmpfs" ino=6474 scontext=u:r:untrusted_app_29:s0:c244,c256,c512,c768 tcontext=u:object_r:mnt_sdcard_file:s0 tclass=lnk_file permissive=0 app=org.navitproject.navit

and a few debug messages from Navit telling us that a bunch of file access operations have failed.

Also, for file and media permission, only media access can be selected, full file access is not even available.

According to information on the net, Android 11 reversed the previous decision of locking apps out of the file system, see https://stackoverflow.com/questions/60360368/android-11-r-file-path-access and https://stackoverflow.com/questions/62782648/android-11-scoped-storage-permissions/66366102#66366102.

As far as I can see it, full filesystem access requires MANAGE_EXTERNAL_STORAGE permission (newly introduced in Android 11). The previous WRITE_EXTERNAL_STORAGE becomes ineffective (i.e. does nothing) on Android 11, though iirc this affects only apps targeting SDK 30+. MANAGE_EXTERNAL_STORAGE is subject to a PlayStore policy: Google will accept this permission only for certain use cases, and Navit likely does not fall into this category.

Another new restriction is that apps can no longer access each other’s private shared storage areas, or in fact anything underneath certain paths (among them /sdcard/Android/data) that doesn’t belong to them. Not sure if that is what is preventing us from accessing /sdcard/Android/data/org.navitproject.navit/files; after all, this path belongs to our app.

I tried yesterday to target SDK 30, request MANAGE_EXTERNAL_STORAGE and grant full filesystem permissions to Navit, but still get the exact same behavior and the same error message.

I have asked on Stack Overflow, with no answers so far.

@jkoan tells me he got similar reports by mail but also hasn’t figured out so far why this is not working. Any input is appreciated.

[mvglasow]

With jkoan’s help I got the emulator up and running and set up Navit:

• Installed Navit from a locally built APK via drag and drop
• Dragged and dropped a binfile map onto the device (which will place it in Downloads)
• adb shelled into the device and moved the map to /sdcard/Android/data.org.navitproject.navit/files
• ls -laed the folder and found the map file has a different user and permissions than the other files in the folder
• chowned and chmoded the file to match the others
• Started Navit and moved to a location on the map

Result: The map displays without any apparent issues.

However, permissions for the copied map file had to be fixed manually:

Original: -rw------- 1 u0_a151 u0_a151

Changed: -rw-rw---- 1 u0_a154 ext_data_rw (the user for Navit on the emulator is u0_a154).

On my phone (which had the issues described but which I have since migrated back to Android 10) all files in the Navit folder belong to Navit’s user and are at least readable and writable for owner and group – which contradicts my initial suspicion that the behavior might have to do with incorrect file ownership or permissions. However, the only way to tell for sure is to retry with Android 11.

[mvglasow]

Just to be sure, I copied navit.xml from my device to the emulator and adjusted permissions as needed. Navit still starts up and uses the custom config file with no issues.

In short, I am unable to reproduce the behavior of my hardware device on the emulator. I’ll have to upgrade my device to Android 11 and reproduce it there, then check permissions, possibly try recreating the app data dir completely and see what I find. However, that will take some time as it implies downtime on a production device.

[mvglasow]

Just out of curiosity, I reset the owner to the “incorrect” one on navit.xml and restarted Navit. However, the emulator doesn’t seem to care, Navit still uses the file. I then took away the group permissions and retried; this time Navit gives me just a black screen, and the logcat shows it keeps crashing and restarting over and over again.

[jandegr]

Hi,

I got everything working after my phone updated to android 11.
For sure, at first I had the same issues as described above but I think my mistake was I tried to upload a copy of a map I retreived from the device itself after the android 11 upgrade.

Retried with a fresh copy and it worked just fine by uploading it over USB with ADB into the navit private storage folder.

After the upgrade the contents of destinations.txt is unusable just as the map that was there before, but after a first destination is set it works again fine, just the destinations from before the update are lost.

At first speech did not work, but only after updating google tts by means of the playstore I noticed I had media volume set to zero, so It may have worked without updating tts too.

I am aware the above does not give a clear path to success but I wanted to make clear I got everything working without changing any permission related to storage or any code in navit. I found no relevant changes to storage permissions between android 10 and 11 for regular apps.

Shortdiscription of the issue : after upgrade to android 11 exsisting binfile(s) and destinations.txt are not usable anymore. Probably the same for any xml's and center.txt.

I experienced this as existing files corrupt after upgrade to android 11
I found no changes in filesystem restrictions.

[jkoan]

I think the biggest problem with this description is that this is using the default location, the problem I encountered and others people are also complaining about is if you set a custom location for you maps.

[jandegr]

not really, the very first line in the issue describes a problem when using the private storage area

[mvglasow]

@jkoan

if you set a custom location for you maps

you are referring to storage location, right? That is, filesystem path, not GPS coordinates, correct?

@jandegr so you are saying all you needed to do was “touch” the files? any idea what is happening behind the scenes, i.e. what causes this to fail initially?

[jkoan]

@jkoan

if you set a custom location for you maps

you are referring to storage location, right? That is, filesystem path, not GPS coordinates, correct?

Yep, I think this was the Problem all the time, because even the recent docs state that application specific directories shouldn't be a Problem. But for user defined folders I think there's a Problem because of the concept of skoped storage.

[jandegr]

@mvglasow

I think the symptoms I experienced and the context are exactly what you described in the first two lines of your opening post.

I would advise anybody on android 10 to keep a good copy of any custom bin's xml's (and destination.txt if of importance) on a desktop. If after an upgrade to android 11 problems arise, then delete the custom files in the private storage and transfer the good copy's with adb/usb.

So I did "replace", not just "touch" as you suggested.

I spent quite a bit of time investigating file ownership and permissions so I just thought posting the quickfix that eventually worked for me might save somebody else a few hours, who kows.

If this works for somebody else and you really want to know what happened in the upgrade, the you may want to compare the custom files before and after the upgrade.

Since the upgrade process is device-vendor provided, this quickfix comes with no guarantee, just informational.

FYI it was on a nokia 2.2 with a fast sd card formatted as internal storage

EDIT: downloading the map of "faroer" with the builtin map downloader still works and the map worked instantly after download, that confirmed the private storage folder itself was perfectly usable.

EDIT: futhermore I have a banking app that is broken after the android11 upgrade and will require work from the dev's of the bank to fix it, another app only lost my credentials after the upgrade

[mvglasow]

@jandegr I doubt the file contents change during the upgrade. After the failed upgrade, I restored my Android 10 installation and was able to use Navit again. I didn’t touch the contents of /sdcard, thus any changes made by the Android 11 upgrade would not have been rolled back by the downgrade.

I suspect a quick fix would be to move Navit’s private data to a different location in shared storage (using MTP), then start Navit once (to recreate the folder), then copy the files back (again using MTP). I’ll try that when I have some time on my hands and can afford some downtime.

[mvglasow]

I upgraded to Android 11 once more and tried to hunt this down a bit further today.

• Fired up Navit – same result as before.
• After closing Navit, renamed /sdcard/Android/data/org.navitproject.navit/, then launched Navit again; a new /sdcard/Android/data/org.navitproject.navit/ folder gets created. File owner is u0_a244, which (I suppose) is the Linux user for Navit on my device, group is sdcard_rw.
• After closing Navit again, I copied the contents of the old folder to the newly created one. File ownership changes automatically from root to the Navit user, but apparently files cannot be accessed. I keep getting the same error message in the log:

06-20 14:01:55.073 12075 12075 W itproject.navit: type=1400 audit(0.0:506): avc: denied { read } for name="sdcard" dev="tmpfs" ino=4396 scontext=u:r:untrusted_app_29:s0:c244,c256,c512,c768 tcontext=u:object_r:mnt_sdcard_file:s0 tclass=lnk_file permissive=0 app=org.navitproject.navit

The message mentions inode number is 4396, which on my device refers to /mnt/sdcard (which in turn is a link to /storage/self/primary).

I also see that Navit is not creating center.txt or destination.txt, even after setting a destination.

Digging a bit deeper, I found /data/data/org.navitproject.navit/shared_prefs/NavitPrefs.xml with the following content:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <boolean name="firstStart" value="false" />
    <string name="DefaultCountry">pl</string>
    <string name="filenamePath">/mnt/sdcard/Android/data/org.navitproject.navit/files/</string>
</map>

This is probably what directs Navit to look in /mnt/sdcard.

After deleting cache and local data (after creating a backup, of course), Navit could access its private external storage dir (aka /sdcard/Android/data/org.navitproject.navit/), and moving the files back there finally gave me my old map and config.

/mnt/sdcard is a result of setting a custom map path sometime prior to the upgrade (the file picker defaulted to /mnt on my device).

Conclusion:

Custom map paths may cause Navit to lose access to its private external storage after an upgrade to Android 11. The reason has to do with external storage access restrictions on Android 11:

• Apps need a special permission to read shared external storage (technically easy to do, but if you want to get your app on the Play Store, Google may reject it unless you have a use case which they deem valid).
• Apps can no longer access each other’s private external storage (/sdcard/Android/data/ followed by a package ID that is not your own).
• Apps can still access their own private storage without needing to obtain additional permissions.
• There are multiple ways to address /sdcard. On my device, /mnt/sdcard and /sdcard are both symlinks to /storage/self/primary, which is itself a symlink to /storage/emulated/0. (Is using symlinks in that way an anti-pattern, by the way? Looks like the filesystem equivalent of goto: a great way to cause a lot of confusion.)
• Apparently, not all ways of addressing /sdcard are created equal. Put another way, Google appears to determine whether to grant access by doing a simple string comparison, rather than normalize the path reference first. As a result, access to /sdcard/Android/data/org.navitproject.navit is allowed whereas /mnt/sdcard/Android/data/org.navitproject.navit is not, although both refer to the same filesystem object.

Since I had previously set a custom map path using the built-in file browser (which starts at /mnt), I had a path in there which Android 11 no longer accepts.

Solution:

By the time we run into this condition, it is probably too late to automatically recover from this – the user has upgraded to Android 11, and the configured path is now out of our reach, so we likely cannot even resolve symlinks to determine whether that path can be reached by another name. Basically we have two options:

• If filenamePath is set in shared preferences and points to an inaccessible path, clear this preference and present the user with a warning dialog. The user then needs to choose another path and/or move their files to a location which Navit can access (which the warning message should instruct them to do).
• Do some guessing: if filenamePath somewhere contains the portion Android/data/org.navitproject.navit/files, strip it and everything before it, and replace it with the canonical private external storage path (and again warn the user so they can verify everything works correctly).

Also, we should change the default path for the custom map location file picker; I suggest we start in the app’s private external storage folder. The other button (which lacks a clear label, by the way) currently sets the path to /mnt/sdcard/navit (which is no longer accessible on Android 11); this also should be changed to the private external storage folder (an alternative would be to clear the preference altogether).

[mvglasow]

I changed the default dir for the file picker. Currently, Navit.java#736 is still breaking things for us: if the path does not contain /navit, it is appended to the path, preventing us from setting /sdcard/Android/data/org.navitproject.navit/files. It probably makes sense to accept the path unmodified also if it contains the package ID (indicating we’re in some place intended for Navit).

If I edit NavitPrefs.xml manually, changing the path to /sdcard/Android/data/org.navitproject.navit/files, everything works as intended. /storage/emulated/0 instead of /sdcard also works.