Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

standardize : allow users to specify output encoding #118

Merged
merged 11 commits into from
Dec 3, 2023
Merged

standardize : allow users to specify output encoding #118

merged 11 commits into from
Dec 3, 2023

Conversation

jbdesbas
Copy link
Contributor

@jbdesbas jbdesbas commented Oct 13, 2023

Add -E --target-encoding argument, so the user can specify an output encoding.
If omitted, keep the original encoding.

Copy link
Collaborator

@GjjvdBurg GjjvdBurg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jbdesbas, thanks for opening this PR and apologies for the delay in reviewing! I've left some nitpicks, but generally I think this is a useful feature. One thing I would like to ask though, is whether you can add a unit test or two that demonstrate the desired behavior. This will help to verify that your change is doing what it's supposed to do. Thanks again!

clevercsv/console/commands/standardize.py Outdated Show resolved Hide resolved
clevercsv/console/commands/standardize.py Outdated Show resolved Hide resolved
@jbdesbas
Copy link
Contributor Author

hi @GjjvdBurg , thank you for you review . I agree with your comments 👍 and I'll work on it as soon as possible

@GjjvdBurg
Copy link
Collaborator

Change looks good to me! There's some formatting errors that hopefully should be easy to fix by running black on your code.

@jbdesbas jbdesbas requested a review from GjjvdBurg November 29, 2023 20:09
# Excel format (i.e. RFC4180) *requires* CRLF
crlf = "\r\n"
exp = crlf.join(["Å,B,C", "é,ü,中", "4,5,6", ""])
with open(tmpoutname, "r", newline="") as fp:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a 100% sure but I wonder if the unit test is failing on windows because this open call doesn't specify an encoding

@GjjvdBurg
Copy link
Collaborator

The code looks good to me but unfortunately the tests are failing on windows. Let me know if you'd like any help investigating

@jbdesbas
Copy link
Contributor Author

jbdesbas commented Dec 3, 2023

You're right, it makes sense since Windows use non-utf8 as default encoding.
a6f548e should solve.
Don't have Windows environment at hand, but I hope it will be OK.
Fortunaly, the tests alone seem to be affected.

@jbdesbas jbdesbas requested a review from GjjvdBurg December 3, 2023 15:26
@GjjvdBurg GjjvdBurg merged commit f48ab1a into alan-turing-institute:master Dec 3, 2023
7 checks passed
@GjjvdBurg
Copy link
Collaborator

All tests passed! 🎉 Thanks again for contributing @jbdesbas!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants