GoToSocial

[chri2]

In README.md is said that this works with Pleroma or Mastodon. I'm wondering whether I could use it with GoToSocial also or if there are certain requirements to the fediverse server that are not covered by standards.

If there is anybody having experience with this already I'd be happy if you'd share information.

I like the idea of this software very much and would like to integrate it into my yunohost running GoToSocial (maybe creating a yunohost app package for this reason).

[chri2]

I followed https://docs.gotosocial.org/en/latest/api/authentication/ to get a token and now I have values for client_id, client_secret, out-of-band token and access_token. I looked at the configuration files, but didn't find any way to tell which lines in the files <group>_clientcred.secret and <group>_usercred.secret do have which meaning.

Also: The URLs in tootgroup.conf and the two files per group mentioned above do they need to reflect the official URL of the instance like known to the fediverse (e.g. https://social.doma.in/) or do they need to describe the way to connect to the instance (like e.g. http://127.0.0.1:1234 if the fediserver is running on the same host) or both in different places.

To try the values I got for my GoToSocial instance it would be helpful if there'd be a description of the content of the configuration files.

Can anybody help with that?

[chri2]

Got this far: Version check failed (Need version 1.0.0) - probably because of GoToSocial version 0.16.0 …

Looking at it found halcy/Mastodon.py#248

[oe4dns]

The URL in the config file is where you would get the access tokens from. Full URL should always work, 127.0.0.1 could work if its on the same machine (naturally) and the webbrowser knows where to direct the request to. So if GoToSocial is the only thing running - there is a good chance. If there are more than one domains on the server it would not.

It looks to me like you would need client_id and access_token in the corresponding lines in tootgroup.conf. But it seems that you have already figured this out.

[chri2]

The URL in the config file is where you would get the access tokens from. Full URL should always work, 127.0.0.1 could work if its on the same machine (naturally) and the webbrowser knows where to direct the request to. So if GoToSocial is the only thing running - there is a good chance. If there are more than one domains on the server it would not.

It looks to me like you would need client_id and access_token in the corresponding lines in tootgroup.conf. But it seems that you have already figured this out.

Yes, I made a config for some other mastodon server and looked at that. I tried to adopt that and filled in the values I got from an app registration and authorization using curl.

Maybe I even guessed right, but I now I need to find out how to disable the version check - I guess. version_check_mode='none'. See the above link to the PR for GoToSocial at Mastodon.py .

Edit Forgot to explain the network setup: GoToSocial runs as a http service internally on http://127.0.0.1:<gotosocial-port>. To let the fediverse connect to it an nginx ssl reverse-proxy is used. So putting anywhere http://127.0.0.1:<gotosocial-port> makes really sure that the GoToSocial service is reached unencrypted.

[oe4dns]

Edit Forgot to explain the network setup: GoToSocial runs as a http service internally on http://127.0.0.1:<gotosocial-port>. To let the fediverse connect to it an nginx ssl reverse-proxy is used. So putting anywhere http://127.0.0.1:<gotosocial-port> makes really sure that the GoToSocial service is reached unencrypted.

Yes this should work. But as long as you stay on the same server, even using a world-reachable URL should not leave the machine itself. But this is not an issue anyway as it seems.

I have looked at the provided link but I am not sure if I understand correctly. The merged PR states it fixed the issue but you are still experiencing it? As far as I can understand, you'd have to disable version checking on the Mastodon.py side.

[chri2]

Extending this line to api_base_url=my_config[group_name]["mastodon_instance"], version_check_mode='none' disables the version check and give me Successful tootgroup.py run for @account at http://127.0.0.1:<port>.

I guess I guessed filling the files with my data right. The file format I generated manually in an editor for the configuration files looks like this:

main configuration file tootgroup.conf

[<group name>]
mastodon_instance = http://127.0.0.1:<GoToSocial-port>
client_id = <group name>_clientcred.secret
access_token = <group name>_usercred.secret
accept_retoots = <yes|no>
accept_dms = <yes|no>
dm_visibility = <private|unlisted|public>

file <group name>_clientcred.secret for each <group name> section in tootgroup.conf:

<client_id>
<client_secret>
http://127.0.0.1:<GoToSocial-port>
<client_name>

file <group name>_usercred.secret for each <group name> section in tootgroup.conf:

<access-token>
http://127.0.0.1:<GoToSocial-port>
<client_id>
<client_secret>

By default the files reside in .local/share/tootgroup.py/ of the current user.

If default is used as <group name> the configuration is used when no -g GROUP, --group GROUP parameter is given to tootgroup.py.

[oe4dns]

Great - I will think about how to integrate this into the README - and mention that GoToSocial works as well!

Thank you for figuring this out!

[chri2]

Sorry to say: GoToSocial doesn't work, yet. Authentication using the token works.

But for the rest there's still some work ahead:
curl -X GET 'http://127.0.0.1:25447/api/v1/notifications' --header 'Authorization: Bearer <token> | jq . shows me IDs like "id": "01J1DKBEHB1XYHJ8717GX80FMG" which doesn't seem to be expected by tootgroup.py (which seems to expect int IDs counting up?).

I'll look into that later.

I think it is a great idea to put the information about the configuration format in the README.md anyway! Thanks for your help so far!

[oe4dns]

Yes, Mastodon uses an ID that is ever increasing. This is how latest seen posts etc. are determined. If GoToSocial uses a Hash value instead, this could be a problem. Although it might be possible using the ID and a timestamp.

[chri2]

GoToSocial uses https://github.com/oklog/ulid which are by there definition Lexicographically sortable .

The example ID "01J1DKBEHB1XYHJ8717GX80FMG" is derived from a millisecond timestamp and some random data.

A solution to the problem would be to interpret the ID as a sortable string if it is not an integer.

[chri2]

I patched tootgroup.py on my instance like this:

$ diff tootgroup.py.orig tootgroup.py
62c62
<         api_base_url=my_config[group_name]["mastodon_instance"],
---
>         api_base_url=my_config[group_name]["mastodon_instance"], version_check_mode='none'
107c107
<     latest_notification_id = 0
---
>     latest_notification_id = str(0)
122c122,123
<                 latest_notification_id = 0
---
>                 # latest_notification_id is already initialized above with the value "0"
>                 # latest_notification_id = 0
134c135
<             my_config[group_name]["last_seen_id"] = str(latest_notification_id)
---
>             my_config[group_name]["last_seen_id"] = latest_notification_id
142c143,145
<                 notification.id > int(my_config[group_name]["last_seen_id"])
---
>                 # TODO replace comparision by a function - this breaks mastodon
>                 # notification.id > int(my_config[group_name]["last_seen_id"])
>                 notification.id > my_config[group_name]["last_seen_id"]
154,155c157,160
<     if latest_notification_id > int(my_config[group_name]["last_seen_id"]):
<         my_config[group_name]["last_seen_id"] = str(latest_notification_id)
---
>     # TODO replace comparision by a function - this breaks mastodon
>     # if latest_notification_id > int(my_config[group_name]["last_seen_id"]):
>     if latest_notification_id > my_config[group_name]["last_seen_id"]:
>         my_config[group_name]["last_seen_id"] = latest_notification_id
183c188
<                                 "Retooted from notification ID: " + str(notification.id)
---
>                                 "Retooted from notification ID: " + notification.id
188c193
<                                 + str(notification.id)
---
>                                 + notification.id
229c234
<                                 + str(notification.id)
---
>                                 + notification.id
234c239
<                                 + str(notification.id)
---
>                                 + notification.id
258c263
<                                 + str(notification.id),
---
>                                 + notification.id,
269c274
<                                 + str(notification.id)
---
>                                 + notification.id

This compares the notification.id lexically and seems to work for GoToSocial. This breaks compatibility to mastodon.

To get this right with all the information from GoToSocial and Mastodon people I suppose the easy way would be

• put the comparision using > in a function that compares on pure digits as int() and otherwise lexically
• give up on sorting by notification.id at all (Mastodon might change the notification.id and it is not defined to be sortable nor to be an integer) and get the timestamps of notifications.created_at to decide by those if a given notification is new

[chri2]

I put the changes into a Draft PR: #13
My branch with these changes can be found here: https://github.com/chri2/tootgroup.py/tree/GoToSocial

I'll install from my branch to test this with my GoToSocial.

[oe4dns]

Could I register an account on your gotosocial instance? I'd like to try to figure out a solution that works on Mastodon/Pleroma as well.

Alternatively, a pointer to an open instance that would allow some testing would be great too.

[oe4dns]

Finally some time to tackle this!

I followed https://docs.gotosocial.org/en/latest/api/authentication/ to get a token and now I have values for client_id, client_secret, out-of-band token and access_token. I looked at the configuration files, but didn't find any way to tell which lines in the files <group>_clientcred.secret and <group>_usercred.secret do have which meaning.

I tried to register tootgroup.py as a client for my account on your GoToSocial instance and now I understand what you mean. It looks like it is not possible to just provide username and password via the Mastodon API and get the required access tokens in return. ('Mastodon API returned error', 400, 'Bad Request', 'unsupported_grant_type')

But this is how it is done with Mastodon and Pleroma. The required tokens/credentials are automatically stored and it just works. That's the reason why it is not explained more thoroughly in the README - There's just nothing to explain.

If password auth does not work with GoToSocial one has tp aquire the credentials manaully like you did. This is possible but much more involved as you have experienced.

This means I will first solve the message ID problem and then think about the setup process. In the worst case one has to manually create the GoToSocial config. I hope I can think of a better solution though.

[oe4dns]

Mastodon.py also supports an OAuth Workflow which is good. Just have to figure out how to implement and select it if reqiuired on first run

[oe4dns]

The easy part - figuring out how to manage the different forms of notification IDs - is done. See commit e70155a

Application registration using either username/password or OAuth will be harder but I'm on it.

[oe4dns]

First implementation of OAuth authorization - tested to work on GoToSocial and Mastodon

See commit 821c04d

Some cleanup required before this can be merged into master. But it works already

[chri2]

Let me know when I should test!

[oe4dns]

It can be tested NOW directly from Github. I will immediately upload it to pypi.org when I'm done here.

The version that enables GoToSocial compatibility is Release v1.5