Implemented sorting of ReallyUserFriendlyTypes using unidecode

[rohnsha0]

as discussed in plone/Products.CMFPlone#3985 (comment)

[mister-roboto]

@rohnsha0 thanks for creating this Pull Request and helping to improve Plone!

TL;DR: Finish pushing changes, pass all other checks, then paste a comment:

@jenkins-plone-org please run jobs

To ensure that these changes do not break other parts of Plone, the Plone test suite matrix needs to pass, but it takes 30-60 min. Other CI checks are usually much faster and the Plone Jenkins resources are limited, so when done pushing changes and all other checks pass either start all Jenkins PR jobs yourself, or simply add the comment above in this PR to start all the jobs automatically.

Happy hacking!

[jensens]

+1 for sane sorting.

But: I vote against using the pyICU package, because of its lack of providing wheels together with it's need for the C-Library. Usage on several platforms may get a PITA.

There is pyUCA which works in a similar way but without having this problem. What do you think?

cc @zopyx @petschki

[zopyx]

But: I vote against using the pyICU package, because of its lack of providing wheels together with it's need for the C-Library. Usage on several platforms may get a PITA.

+1

However, pyuca appears unmaintained, last release from 2017, future unclear.

Proposal:

• provide a universal utility method for sorting a list of strings for a given language
• default sorting using sorted()
• if pyuca or pyICU is installed: facilitate the sort based on the collation provide by one of those packages.

[rohnsha0]

Proposal:

• provide a universal utility method for sorting a list of strings for a given language
• default sorting using sorted()
• if pyuca or pyICU is installed: facilitate the sort based on the collation provide by one of those packages.

@zopyx then according to you, we first need to check if pyicu or pyuca is installed then sorting using their collarator else using default sorting method?

[wesleybl]

And if the unidecode package is used:

>>> from unidecode import unidecode
>>> unidecode("Águia").lower()
'aguia'

[rohnsha0]

And if the unidecode package is used:

>>> from unidecode import unidecode
>>> unidecode("Águia").lower()
'aguia'

does unicode provide the same func? im actually unaware of it... @wesleybl

[jensens]

And if the unidecode package is used:

It is a dependency of the Plone universe anyway.

[rohnsha0]

And if the unidecode package is used:

It is a dependency of the Plone universe anyway.

then performing the sorting using unicode is a better option... right?

[wesleybl]

does unicode provide the same func? im actually unaware of it... @wesleybl

The unicode function no longer exists in Python 3. In any case, it was used for something different in Python 2.

You must use the unidecode function within the function passed in the key parameter of the sort method. For example:

>>> from unidecode import unidecode
>>> list = ['Café', 'Águia', 'Água']
>>> list.sort(key=lambda string: unidecode(string).lower())
>>> list
['Água', 'Águia', 'Café']

[wesleybl]

It is a dependency of the Plone universe anyway.

Anyway, I think it's worth putting it as a dependency in setup.py

[rohnsha0]

@wesleybl @jensens verify the changes... it worked for me!

[wesleybl]

Please check for test errors.

[rohnsha0]

autofix pushing was disabled in the configuration. to autofix this pr, comment or apply the label "pre-commit.ci autofix" to the pull request

it was the error found in one of failing test

[rohnsha0]

pre-commit.ci autofix

[rohnsha0]

@jenkins-plone-org please run jobs

[rohnsha0]

@jenkins-plone-org please run jobs

[rohnsha0]

@jenkins-plone-org please run jobs

[jensens]

+1

[wesleybl]

LGTM