Shared folders are read-only on sd card (<29) and inaccessible on all ext. storage (>=29, ~November 2020)

[jonnojohnson]

SyncThing works great for repositories created on the internal SDCard. However if I try to create a rep on my external SDCard it doesn't complain. The other node attempts to sync & just shows sync progress set at 0%.
I'm running Android 4.4.2 on a GS4 & I know external SD card access for apps got restricted more since 4.4 but I think SyncThing is one of those apps where it makes complete sense to have ext SD access.
In the short term perhaps SyncThing for Android could complain if ext SD card access is not an option?

[Nutomic]

I'll need a logcat for that. Unfortunately, I don't have an external sd card to test with myself.

[jonnojohnson]

I'll try to do that when I get chance. I don't suppose you can point me to some instructions how to get the log? Not much of an Android expert.

[jonnojohnson]

Incidentally I think this article pretty much explains the issue: http://www.androidcentral.com/kitkat-sdcard-changes
Essentially in 4.4 Android requires an app to have write permissions for the directory it wants to write to and external SD cards are FAT formatted which don't have permissions.
I think the app has to create a single folder on the ext SD card which it then has permission to write to. All repositories would need to be in there.

[Nutomic]

I searched around, and it really seems there is no way to write to external sdcard without root. But I will add a directory chooser for repo creation in #5, and might add the external app dir to that.

[Nutomic]

This should work with the new android.content.Context.getExternalMediaDirs() method in Android L.

[zombielinux]

So is that new method already in the commits or is that coming down the pipes still?

[Nutomic]

Nope, not implemented yet.

The above API only works on Android L (though it might come via support library), so I'll have to check which folders are writeable on external sdcard on older APIs.

[stilvoid]

It's worth noting that PowerTools is able to sync to folders in the emulated SD card. No idea how it does it but I've got it working just fine on my Nexus 4 (android 4.4)

[Nutomic]

Okay I researched this a bit further:

• 4.4 only: using Context.getExternalFilesDirs(), we can write to [device]/Android/data/com.nutomic.syncthingandroid
• in 4.3 and lower, sdcard can be accessed directly, either by scanning /mnt/ or using eg Environment2 (which is commented in German, haven't seen that before for Android)
• Using Storage Access Framework, we can access all locations (but folder support is only added in Android L afaik)

Of all these, option 3 seems to be the best, assuming it is accessible through support libraries. Otherwise, we might have to implement all three.

Edit: Looks like Context.ACTION_GET_CONTENT and Context.ACTION_OPEN_DOCUMENT would be even better (those would replace the folder chooser, but on 4.3 and lower, a seperate file browser app would be needed).

[ferongr]

At least you can switch to the WebUI and enter the paths manually. Since the majority of devices are not running 4.4 they are unaffected by Google's boneheadedness it's be nice if the path picker could navigate anywhere around the filesystem.

[Nutomic]

I plan to remove the internal file chooser and use external file browser using the intents in the edit above. The only disadvantage is that it won't work if the user has no file browser installed (on 4.3 and lower). So maybe the internal file chooser should be kept as a fallback for that. I'll look into that.

[ferongr]

I don't know of any ROM that can't satisfy the file chooser intent, even
official manufacturer ones so you should be ok.

On 4 July 2014 18:42, Felix Ableitner notifications@github.com wrote:

I plan to remove the internal file chooser and use external file browser
using the intents in the edit above. The only disadvantage is that it won't
work if the user has no file browser installed (on 4.3 and lower). So maybe
the internal file chooser should be kept as a fallback for that. I'll look
into that.

—
Reply to this email directly or view it on GitHub
#29 (comment)
.

[Nutomic]

I'm pretty sure Android 4.3 and below doesn't have any file browser by default.

[ferongr]

I don't know about AOSP but many manufacturer ROMs and CM do. But it needs
testing

On 4 July 2014 22:43, Felix Ableitner notifications@github.com wrote:

I'm pretty sure Android 4.3 and below doesn't have any file browser by
default.

—
Reply to this email directly or view it on GitHub
#29 (comment)
.

[Nutomic]

Yeah I'll do a test release first once I have it implemented.

[Nutomic]

OK this won't work as well as I thought, because the DocumentsProvider returns an URI, and there's no general way to get file path from that.

@calmh Would it be possible to specify the repository folder as an URI?

[micahh2]

Can't you also use specific folders on on sd card? I'm not sure at this moment but something like this: com.exampleappname.subapp

[Nutomic]

Right, but it would be better if we could also write to sdcard root (and I'm not quite sure how to get that path in Java).

[calmh]

@Nutomic What would we then do with the URI? We need a path...

[Nutomic]

At least Java can perform normal file operations on an URI using File. Maybe this is also somehow possible in Go.

Edit: The URI points to a local directory where we have full read/write permissions.

[calmh]

Sounds like we could just strip any leading file:// and get a path then?

[Nutomic]

That's what I tried basically, but Android uses its own URIs, they look like content://com.android.providers.media.documents/document/image:62 (for an image in this case, but similar for folders).

So we'd really have to access it through the URI.

[calmh]

What we have available is a file system. If the URI can be mapped to a local directory as you say, we can write to that local directory. Accessing the URI as is would require going through some API, I guess, which sound like something that would need to happen Java-side or wherever.

[Nutomic]

Ok so I guess I'll have to see myself how this could be done.

Or I might open an issue on golang for Android URI support (as part of general Android support).

[sp00kie]

Would really appreciate it if you could find a workaround for this STUPID google decision. All of the stuff I want to sync is on my SD card! Who stores media in their main memory?

[dionorgua]

Samsung removed it only from UI. But you can convert it using adb. Don't want to advertise here, just google 'Samsung S7 adoptable storage'. Probably same for others.

[wweich]

I've used adoptable storage on a near stock device (Moto X Style/Pure), so I don't know how it is on Galaxy S7.
But on that device, you cannot use both internal and SD card at the same time. In the UI, you have to migrate the data to the sd card. Then /storage/emulated/0/ will point to the sd card and no file explorer app (or Syncthing) can access the real internal storage anymore.

That way, Syncthing can write to sd card without problems.

As you seem to have to have an S7 and used that method, why don't you try it and confirm for us if it works.

[dionorgua]

I've not used it (yet). Just discovered that it's possible to enable 'adoptable storage'. Since it'll erase everything on SD card I need to be ready for this.

In any case thanks for confirmation that syncthing works with adoptable SD card.

[dper]

On the Samsung Galaxy Note 4, using adb I just enabled adoptable storage. As described above, now /storage/emulated/0 points to the SD card, and the real internal storage isn't used. It wastes storage space, but it successfully works around this bug.

[wweich]

I used a 64GB card in my 32GB phone. But as the system and Apps are still on the phone memory, my usable space more than tripled.
Also, in some Apps (like Google Play Music or Spotify) you can choose where the offline data should be stored. That way, you have options to use both storage spaces.

[jult]

What I find to be really strange is that I used to run syncthing silently in the background on my rooted Samsung Galaxy S3, it synced all my preferred external sdcard directories without a glitch for months, never an issue. Now android has 'upgraded' and syncthing doesn't do that anymore? What a sad sad way to go for android. Really, have they killed such basic functionality in favor of security theatre, or what's the reason for this change?

[dper]

@jult That kind of thing is outside the scope of this issue. :-/ Here we're just trying to figure out how syncthing should handle the current situation, regardless of why it came about.

[licaon-kter]

@jult That's an old device, what Android upgrade did you make?

[frispete]

@jult Yes, security theatre obscure.

Isn't it quite interesting, that google phones doesn't come with an sd card slot, hence their users don't have this problem... This is further obscured, that the choice of implementing syncthing in go is the major blocker in this respect. Needless to say, that go is a - google innovation - too.

This is a perfect recipe for damaging the last remains of a good reputation.

[Nutomic]

There was a post on the forum with an idea how this could be solved, but afaik no one is working on this.

Some related discussion is in #1008.

[Catfriend1]

Syncthing 0.10.6+ supports sdcard write access when root privs are granted.
Without root privs, theres a way to patch with https://github.com/Catfriend1/tingle - be careful about this and backup first! (Can be unstable but already worked for me)

[lkwg82]

@Catfriend1 Why dont you port your patch upstream (this) or make a little tutorial?

[Catfriend1]

@lkwg82 It should not basically be needed to patch the os deeply to just run this app. Its much easier and common to use root. It is a PR for for the original tingle already. There is an explanation how it works.

[capi]

@Catfriend1 I'd vote to keep this issue open. The current solutions are for rooted and for very advanced users only. Having a synchronization tool run for the "ordinary" user should not require any such things. Therefore I still think that this issue is still open.

My benchmark here is: it still doesn't work on my dad's phone, who I neither will root nor patch his OS :-)

[AudriusButkevicius]

Sorry, I don't see a reason why this was closed in the first place. Sure, it works with root, but that's a workaround, not a solution.

[Catfriend1]

PR #1170 has the new SAF UI that will guide users on newer Android os to a supported, "no-root requiring" path on their external sdcard to create a readwrite share folder.

[Catfriend1]

Hmm the issue should stay open but honestly will never be totally fixed...

[AudriusButkevicius]

Sure, that doesn't mean the issue does not exist.

[imsodin]

Status update (please correct if I got anything wrong):

It looks like this problem will become even more urgent, as currently it looks like google will pull trough on introducing scoped storage in SDK 29, which is planned to become mandatory in the play store by November 2020. This means Syncthing can only access files it created itself or that resides in it's "own dir" (some directories in deeply-nested structures like .../Android/data/com.nutomic.syncthing-android and the like). At that point not even the current read-only workaround will work on any storage. For Syncthing to stay viable in a non-root context we need either a change of mind by google (I'd say unlikely), someone taking up the java implementation of Syncthings protocol (https://github.com/syncthing/syncthing-lite) or giving go-bindings a try to implement filesystem access through android APIs. A first step towards the last would be to run Syncthing with bindings instead of as a binary with the recently added "library-interface". In case anyone tries, you currently need to vendor dependencies (go mod vendor) until golang/go#27234 is fixed.

[Catfriend1]

For everyone brought here in 2020: The current discussion about Android 11 being expected to be the first version since 4.4 supporting writing to sdcard again is here: https://forum.syncthing.net/t/android-11-all-files-access-for-the-syncthing-app

[imsodin]

As mentioned above the restrictions have changed, such that on Android >= 11 Syncthing can now write to SD cards. This issue was thus fixed by #1721 which made use of this new policy, and that was released in https://github.com/syncthing/syncthing-android/releases/tag/1.19.0.