Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update wikipedia_scratcher.py #93

Merged
merged 2 commits into from
Mar 26, 2024
Merged

Conversation

siddheshhr
Copy link
Contributor

@siddheshhr siddheshhr commented Mar 17, 2024

Avoiding Duplicate Requests: In the get_response_elems function, its making two separate requests to the same URL. Instead, you can reuse the response object obtained in the with block. This reduces redundant HTTP requests and improves efficiency.

Fixes

Optimizes get_response_elems by reusing a single request for the URL, avoiding duplicates.

Description

In the get_response_elems function, its making two separate requests to the same URL. Instead, you can reuse the response object obtained in the with block. This reduces redundant HTTP requests and improves efficiency.

Checklist

  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no
    visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Avoiding Duplicate Requests: In the get_response_elems function, its making two separate requests to the same URL. Instead, you can reuse the response object obtained in the with block. This reduces redundant HTTP requests and improves efficiency.
@siddheshhr siddheshhr requested review from a team as code owners March 17, 2024 14:48
@siddheshhr siddheshhr requested review from TimidRobot and annatuma and removed request for a team March 17, 2024 14:48
@IamMQaisar
Copy link
Contributor

Checking it!

@TimidRobot
Copy link
Member

@siddheshhr please update the description: fix the issue link (#93 is this PR) and complete the appropriate check boxes

@siddheshhr
Copy link
Contributor Author

@siddheshhr please update the description: fix the issue link (#93 is this PR) and complete the appropriate check boxes

Have made the changes,if any more corrections let me know

Copy link
Member

@TimidRobot TimidRobot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for this work! please see requested changes

Comment on lines 101 to 109
except requests.HTTPError as e:
print(f"HTTP Error: {e}", file=sys.stderr)
return {}
except requests.RequestException as e:
print(f"Request Exception: {e}", file=sys.stderr)
return {}
except KeyError as e:
print(f"KeyError: {e}. Search data is: {search_data}", file=sys.stderr)
return {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think these errors should result in continued operation. the original code resulted in either a script exit or a raised exception

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made the below change
except requests.HTTPError as e:
print(f"HTTP Error: {e}", file=sys.stderr)
raise
except requests.RequestException as e:
print(f"Request Exception: {e}", file=sys.stderr)
raise
except KeyError as e:
print(f"KeyError: {e}. Search data is: {search_data}", file=sys.stderr)
raise
Instead of printing messages and continuing execution, errors are raised using raise. @TimidRobot please check it once

@siddheshhr siddheshhr requested a review from TimidRobot March 19, 2024 16:01
Copy link
Member

@TimidRobot TimidRobot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

@TimidRobot TimidRobot merged commit b9ee703 into creativecommons:main Mar 26, 2024
1 check passed
@TimidRobot TimidRobot self-assigned this Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants