Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DateField used with auto_now can set wrong date when django timezone is not UTC #70

Closed
M1ha-Shvn opened this issue Mar 12, 2021 · 1 comment
Assignees
Labels
bug Something isn't working wontfix This will not be worked on
Milestone

Comments

@M1ha-Shvn
Copy link
Owner

Django native QuerySet.bulk_create(...) method uses datetime.date.today() in DateField.pre_save() which is timzeone aware and does not know about django's TIME_ZONE setting. Here is my ticket in django bug tracker. This causes error in django_pg_bulk_update methods when Postgres version is lower than 9.5 or key_is_unique=False parameter is used:

  1. This library bulk_update inserts database NOW() funcion. Django sets correct timezone and date after update is correct.
  2. Native bulk_create inserts date.today() which may differ from NOW().
@M1ha-Shvn M1ha-Shvn added the bug Something isn't working label Mar 12, 2021
@M1ha-Shvn M1ha-Shvn added this to the 3.4.3 milestone Mar 12, 2021
@M1ha-Shvn M1ha-Shvn self-assigned this Mar 12, 2021
@M1ha-Shvn M1ha-Shvn modified the milestones: 3.4.3, 3.4.4 Jul 1, 2021
@M1ha-Shvn M1ha-Shvn modified the milestones: 3.4.4, 3.4.5 Oct 13, 2021
M1ha-Shvn added a commit that referenced this issue Oct 29, 2021
2. Fixed tests failing in case of #70
3. QA refactoring
4. Optimized batching function
@M1ha-Shvn
Copy link
Owner Author

For future readers, if someone catches this issue.

Problem

Bug occures when:

  1. You have DateField(auto_now=True) or DateField(auto_now_add=True) field
  2. You call pg_bulk_update_or_create with key_is_unique=False modifier or on PostgreSQL <= 9.4.
  3. Method is called on day change when date respsecting django TIME_ZONE setting differs from date returned by python datetime.date() result.

What happens

When these conditions are met, created items use native QuerySet.bulk_create(...) method while updates use pg_bulk_update method of this library.

  • QuerySet.bulk_create(...) has a known bug in django ORM which is documented and won't be fixed. As a result it sets timezone aware datetime.date(), ignoring TIME_ZONE setting.
  • pg_bulk_update method uses database NOW() function with cursor and respects TIME_ZONE setting. This is a correct behavior.

As a result, created instances can have dates other than updated instances if method

Why wontfix

  1. I don't see any reason to fix it in this library. This is a django bug.
  2. I don't see not hacky solution to fix this problem
  3. PostgreSQL 9.4 is not actual any more and key_is_unique flag is thread unsafe. It should not be used.

What can be done

  1. Django recommends to redeclare DateField.save() method behavior if you have problems with this method.
  2. I can replace native QuerySet.bulk_create(...) with pg_bulk_create method of this library which gives correct results. This may be done in future releases, though is not backwards compatible and can break some code.

What has be done

  1. I've written a test on this situation with @expectedFailure so it can be easily catched in the future
  2. I've corrected other tests, so they would not fail in provided circumstances, even when pg_bulk_update_or_create generates incorrect results.

@M1ha-Shvn M1ha-Shvn added the wontfix This will not be worked on label Oct 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

1 participant