[Patreon] 403s and 401s

[SpaceAceMonkey]

I am having an issue similar to the issues described in #5731, #6140, #6239, and #6241. Requests to https://www.patreon.com/CreatorName, and https://www.patreon.com/CreatorName/posts always result in 403 errors after CloudFlare challenges.

None of the solutions suggested in those issues work for me. Changing browser options in order to modify the fingerprint doesn't work. Using different browsers doesn't work. Using the Patreon application user agent doesn't work.

I have tried using both the latest stable version and 1.27.7-dev.

The only thing that sort of works is pointing gallery-dl to https://www.patreon.com/home as mentioned in #6140 , which seems to bypass the CloudFlare challenge, but which then results in a 401 response for each attempted download. I have tried using cookies from both Brave and Firefox, both of which are logged in to my Patreon account. Even if this worked, I'm not sure if it would get all posts from the creator I am interested in.

[cookies][info] Extracted 2938 cookies from Firefox

[cookies][info] Extracted 2179 cookies from Brave

There are no reports of failure to decrypt any of the cookies with either browser. Initially, that was an issue with Brave, but once I specified the keyring, that problem disappeared.

I have also tried leaving 12+ hours between attempts as suggested by someone in one of the linked issues.

I don't know if it was mentioned in those other issues, but the Patreon URLs you see in your browser address bar include a /c/ portion that gallery-dl doesn't seem to expect. If you include the /c/ as in https://www.patreon.com/c/CreatorName/posts, gallery-dl reports that it cannot find that creator. Presumably, it thinks the creator is c. I imagine this is handled behind the scenes by 30x redirects, but it seemed worth mentioning.

I am using ubuntu 24.04.

[mikf]

Since https://www.patreon.com/home works, maybe you are "allowed" to do API requests, but can't access the website itself. Could you try using https://www.patreon.com/id:CAMPAIGN_ID as URL (e.g. https://www.patreon.com/id:2367430)? You can find the CAMPAIGN_ID of a creator in the HTML source by searching for campaign.

but which then results in a 401 response for each attempted download

I really don't know what to do about that or why it happens in the first place. Maybe --print-traffic could help with debugging.

[SpaceAceMonkey]

Hi. After trying the CAMPAIGN_ID solution, I went back to trying /creator/posts. The 403s seem to have disappeared when using either the campaign or creator routes, but the 401s remain.

I switched from having gallery-dl extract the cookies to specifying a cookie file in the Mozilla format, but I still get nothing but 401s.

Below is a partial example of the cookie file I am using.

.patreon.com TRUE / FALSE 12345678 patreon_device_id GUID_GOES_HERE
.patreon.com    TRUE    /       TRUE    12345678      session_id SESSION_ID_GOES_HERE
...

I have no problems using patreon-dl with the same cookies. With patreon-dl, the cookies are supplied as a string on the command-line.

I am confident that gallery-dl is reading the cookies, because I don't get the no 'session_id' cookie set warning I get when the cookies are not properly specified.

The --print--traffic flag isn't showing anything odd except possibly the Report-To: CloudFlare header. I am guessing that isn't the source of the problem. The rest is normal 30x responses leading to the 401s.

[mikf]

The 401 errors might be from .m3u8 video downloads by yt-dlp/youtube-dl without cookies. Enable downloader.ytdl.forward-cookies and see if that helps (#6401).

[conkerts]

Requests to https://www.patreon.com/CreatorName, and https://www.patreon.com/CreatorName/posts always result in 403 errors after CloudFlare challenges.

None of the solutions suggested in those issues work for me. Changing browser options in order to modify the fingerprint doesn't work. Using different browsers doesn't work. Using the Patreon application user agent doesn't work.

I'll join this issue, too.
I tested today the latest release v1.27.7 (from 5 days ago) , and the latest nightly,
and still exactly the same issues as mentioned above. None of the workarounds work either.

[FriedGenera]

Requests to https://www.patreon.com/CreatorName, and https://www.patreon.com/CreatorName/posts always result in 403 errors after CloudFlare challenges.
None of the solutions suggested in those issues work for me. Changing browser options in order to modify the fingerprint doesn't work. Using different browsers doesn't work. Using the Patreon application user agent doesn't work.

I'll join this issue, too. I tested today the latest release v1.27.7 (from 5 days ago) , and the latest nightly, and still exactly the same issues as mentioned above. None of the workarounds work either.

I had to manually set the user-agent in the config file to match my browser so-

        "user-agent": "Firefox",
        "browser": false, 

then I grabbed the cookies from said browser using - --cookies-from-browser firefox, and that was the only way I could get it to work

[Hrxn]

That is the way it is supposed to work, yes.

Although your config example here is wrong:
A real "user-agent" would look like this, for example:

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

Just use the same as the one with your valid cookies.

And "browser" is not a boolean, it is a string value.
If you don't need to set it for some specific sites, its default is simply null.

[FriedGenera]

That is the way it is supposed to work, yes.

Although your config example here is wrong: A real "user-agent" would look like this, for example:

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

Just use the same as the one with your valid cookies.

And "browser" is not a boolean, it is a string value. If you don't need to set it for some specific sites, its default is simply null.

Yeah I put my own user-agent, didn't want to clutter to reply since they might be using a different browser. For the "browser", I just copy pasted it since it was mentioned in this post - #6241 (comment) when trying out the mobile user-agent

[WyohKnott]

Requests to https://www.patreon.com/CreatorName, and https://www.patreon.com/CreatorName/posts always result in 403 errors after CloudFlare challenges.
None of the solutions suggested in those issues work for me. Changing browser options in order to modify the fingerprint doesn't work. Using different browsers doesn't work. Using the Patreon application user agent doesn't work.

I'll join this issue, too. I tested today the latest release v1.27.7 (from 5 days ago) , and the latest nightly, and still exactly the same issues as mentioned above. None of the workarounds work either.

I had to manually set the user-agent in the config file to match my browser so-

        "user-agent": "Firefox",
        "browser": false, 

then I grabbed the cookies from said browser using - --cookies-from-browser firefox, and that was the only way I could get it to work

Thanks! The browser false did the trick:

        "patreon": {
            "filename": "{date:%Y%m%d}_{creator['full_name']!g}_{title[:60]!g}_{hash}.{extension}",
            "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
            "browser": null
        },

I am using a cookie file with https://addons.mozilla.org/en-US/firefox/addon/export-cookies-txt/ because I am using multiples Firefox profile and I am not sure how to tell into which one to fetch cookies :

gallery-dl --cookies ~/cookies-patreon-com.txt https://www.patreon.com/*******/posts --verbose

I was stuck for months with this.

[Hrxn]

If you mean null, then yes...

[mikf]

Thanks! The browser false did the trick:

The browser option for patreon has not been enabled by default since v1.27.6 (321161c)

I am not sure how to tell into which one to fetch cookies

--cookies-from-browser firefox:PROFILE
or
"cookies": ["firefox", "PROFILE"]

where PROFILE is the profile name or path to a profile folder.

gallery-dl --cookies-from-browser firefox:~/.mozilla/firefox/a1b2c3d4.default/ URL

And "browser" is not a boolean, it is a string value.

Technically yes, but for quite a few options, including browser, it is enough to set it to any falsy value like false, 0, "", [], etc to disable it.