DeepDiff 5.7.0

.github/ISSUE_TEMPLATE/bug_report.md

@@ -7,6 +7,8 @@ assignees: ''
 
 ---
 
+Please checkout the [F.A.Q](https://zepworks.com/deepdiff/current/faq.html) page before creating a bug ticket to make sure it is not already addressed.
+
 **Describe the bug**
 A clear and concise description of what the bug is.
 

AUTHORS.md

@@ -41,3 +41,5 @@ Authors in order of the timeline of their contributions:
 - Sun Ao [eggachecat](https://github.com/eggachecat) for adding custom operators.
 - Sun Ao [eggachecat](https://github.com/eggachecat) for adding ignore_order_func.
 - [SlavaSkvortsov](https://github.com/SlavaSkvortsov) for fixing unprocessed key error.
+- Håvard Thom [havardthom](https://github.com/havardthom) for adding UUID support.
+- Dhanvantari Tilak [Dhanvantari](https://github.com/Dhanvantari) for Bug-Fix: `TypeError in _get_numbers_distance() when ignore_order = True`.

CHANGELOG.md

@@ -1,5 +1,9 @@
 # DeepDiff Change log
 
+- v5-7-0:
+    - https://github.com/seperman/deepdiff/pull/284 Bug-Fix: TypeError in _get_numbers_distance() when ignore_order = True by @Dhanvantari
+    - https://github.com/seperman/deepdiff/pull/280 Add support for UUIDs by @havardthom
+    - Major bug in delta when it comes to iterable items added or removed is investigated by @uwefladrich and resolved by @seperman 
 - v5-6-0: Adding custom operators, and ignore_order_func. Bugfix: verbose_level==0 should disable values_changes. Bugfix: unprocessed key error.
 - v5-5-0: adding iterable_compare_func for DeepDiff, adding output_format of list for path() in tree view.
 - v5-4-0: adding strict_checking for numbers in DeepSearch.

README.md

@@ -1,4 +1,4 @@
-# DeepDiff v 5.6.0
+# DeepDiff v 5.7.0
 
 ![Downloads](https://img.shields.io/pypi/dm/deepdiff.svg?style=flat)
 ![Python Versions](https://img.shields.io/pypi/pyversions/deepdiff.svg?style=flat)
@@ -14,13 +14,16 @@
 
 Tested on Python 3.6+ and PyPy3.
 
-**NOTE: Python 2 is not supported any more. DeepDiff v3.3.0 was the last version to support Python 2**
+- **[Documentation](https://zepworks.com/deepdiff/5.7.0/)**
 
-**NOTE: The last version of DeepDiff to work on Python 3.5 was DeepDiff 5-0-2**
+## What is new?
 
-- [Documentation](https://zepworks.com/deepdiff/5.6.0/)
+DeepDiff 5-7-0 includes bug fixes and improvements:
+
+- https://github.com/seperman/deepdiff/pull/284 Bug-Fix: TypeError in _get_numbers_distance() when ignore_order = True by @Dhanvantari
+- https://github.com/seperman/deepdiff/pull/280 Add support for UUIDs by @havardthom
+- Major bug in delta when it comes to iterable items added or removed is investigated by @uwefladrich and resolved by @seperman 
 
-## What is new?
 
 DeepDiff 5-6-0 allows you to pass custom operators.
 
@@ -110,13 +113,13 @@ Note: if you want to use DeepDiff via commandline, make sure to run `pip install
 
 DeepDiff gets the difference of 2 objects.
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.6.0/diff.html)
-> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/5.6.0/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.7.0/diff.html)
+> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/5.7.0/>
 > - Tutorials and posts about DeepDiff can be found on <https://zepworks.com/tags/deepdiff/>
 
 ## A few Examples
 
-> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/5.6.0/> for full documentation.
+> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/5.7.0/> for full documentation.
 
 ### List difference ignoring order or duplicates
 
@@ -320,8 +323,8 @@ Example:
 ```
 
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.6.0/diff.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.6.0/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.7.0/diff.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.7.0/>
 
 
 # Deep Search
@@ -353,17 +356,17 @@ And you can pass all the same kwargs as DeepSearch to grep too:
 {'matched_paths': {"root['somewhere']": 'around'}, 'matched_values': {"root['long']": 'somewhere'}}
 ```
 
-> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/5.6.0/dsearch.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.6.0/>
+> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/5.7.0/dsearch.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.7.0/>
 
 # Deep Hash
 (New in v4-0-0)
 
 DeepHash is designed to give you hash of ANY python object based on its contents even if the object is not considered hashable!
 DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash.
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.6.0/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.6.0/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.7.0/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.7.0/>
 
 Let's say you have a dictionary object.
 
@@ -411,8 +414,8 @@ Which you can write as:
 At first it might seem weird why DeepHash(obj)[obj] but remember that DeepHash(obj) is a dictionary of hashes of all other objects that obj contains too.
 
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.6.0/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.6.0/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.7.0/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.7.0/>
 
 
 # Using DeepDiff in unit tests

deepdiff/__init__.py

@@ -1,6 +1,6 @@
 """This module offers the DeepDiff, DeepSearch, grep, Delta and DeepHash classes."""
 # flake8: noqa
-__version__ = '5.6.0'
+__version__ = '5.7.0'
 import logging
 
 if __name__ == '__main__':

deepdiff/delta.py

@@ -279,12 +279,19 @@ def _do_attribute_added(self):
         if attribute_added:
             self._do_item_added(attribute_added)
 
+    @staticmethod
+    def _sort_key_for_item_added(path_and_value):
+        elements = _path_to_elements(path_and_value[0])
+        # Example elements: [(4.3, 'GET'), ('b', 'GETATTR'), ('a3', 'GET')]
+        # We only care about the values in the elements not how to get the values.
+        return [i[0] for i in elements] 
+
     def _do_item_added(self, items, sort=True, insert=False):
         if sort:
             # sorting items by their path so that the items with smaller index
             # are applied first (unless `sort` is `False` so that order of
             # added items is retained, e.g. for dicts).
-            items = sorted(items.items(), key=lambda x: x[0])
+            items = sorted(items.items(), key=self._sort_key_for_item_added)
         else:
             items = items.items()
 
@@ -392,7 +399,7 @@ def _do_item_removed(self, items):
         """
         # Sorting the iterable_item_removed in reverse order based on the paths.
         # So that we delete a bigger index before a smaller index
-        for path, expected_old_value in sorted(items.items(), key=lambda x: x[0], reverse=True):
+        for path, expected_old_value in sorted(items.items(), key=self._sort_key_for_item_added, reverse=True):
             elem_and_details = self._get_elements_and_details(path)
             if elem_and_details:
                 elements, parent, parent_to_obj_elem, parent_to_obj_action, obj, elem, action = elem_and_details

deepdiff/diff.py

@@ -13,7 +13,7 @@
 from collections import defaultdict
 from itertools import zip_longest
 from ordered_set import OrderedSet
-from deepdiff.helper import (strings, bytes_type, numbers, times, ListItemRemovedOrAdded, notpresent,
+from deepdiff.helper import (strings, bytes_type, numbers, uuids, times, ListItemRemovedOrAdded, notpresent,
                              IndexedHash, unprocessed, add_to_frozen_set,
                              convert_item_or_items_into_set_else_none, get_type,
                              convert_item_or_items_into_compiled_regexes_else_none,
@@ -1148,6 +1148,11 @@ def _diff_datetimes(self, level):
         if level.t1 != level.t2:
             self._report_result('values_changed', level)
 
+    def _diff_uuids(self, level):
+        """Diff UUIDs"""
+        if level.t1.int != level.t2.int:
+            self._report_result('values_changed', level)
+
     def _diff_numpy_array(self, level, parents_ids=frozenset()):
         """Diff numpy arrays"""
         if level.path() not in self._numpy_paths:
@@ -1309,6 +1314,9 @@ def _diff(self, level, parents_ids=frozenset(), _original_type=None):
         elif isinstance(level.t1, times):
             self._diff_datetimes(level)
 
+        elif isinstance(level.t1, uuids):
+            self._diff_uuids(level)
+
         elif isinstance(level.t1, numbers):
             self._diff_numbers(level)
 

deepdiff/distance.py

@@ -194,9 +194,9 @@ def _get_numbers_distance(num1, num2, max_=1):
     """
     if num1 == num2:
         return 0
-    if isinstance(num1, float):
+    if not isinstance(num1, float):
         num1 = float(num1)
-    if isinstance(num2, float):
+    if not isinstance(num2, float):
         num2 = float(num2)
     # Since we have a default cutoff of 0.3 distance when
     # getting the pairs of items during the ingore_order=True

deepdiff/helper.py

@@ -2,6 +2,7 @@
 import re
 import os
 import datetime
+import uuid
 import logging
 import warnings
 import time
@@ -126,6 +127,7 @@ def copy(self):  # pragma: no cover. Only used in pypy3 and py3.5
 bytes_type = bytes
 only_numbers = (int, float, complex, Decimal) + numpy_numbers
 datetimes = (datetime.datetime, datetime.date, datetime.timedelta, datetime.time)
+uuids = (uuid.UUID)
 times = (datetime.datetime, datetime.time)
 numbers = only_numbers + datetimes
 booleans = (bool, np_bool_)

deepdiff/path.py

@@ -1,7 +1,6 @@
 import logging
 from ast import literal_eval
-
-# TODO: it needs python3.6+ since dictionaries are ordered.
+from functools import lru_cache
 
 logger = logging.getLogger(__name__)
 
@@ -33,6 +32,7 @@ def _add_to_elements(elements, elem, inside):
 DEFAULT_FIRST_ELEMENT = ('root', GETATTR)
 
 
+@lru_cache(maxsize=1024 * 128)
 def _path_to_elements(path, root_element=DEFAULT_FIRST_ELEMENT):
     """
     Given a path, it extracts the elements that form the path and their relevant most likely retrieval action.

docs/authors.rst

@@ -52,6 +52,9 @@ Authors in order of the timeline of their contributions:
 -  Sun Ao `eggachecat`_ for adding custom operators.
 -  Sun Ao `eggachecat`_ for adding ignore_order_func.
 -  `SlavaSkvortsov`_ for fixing unprocessed key error.
+-  Håvard Thom `havardthom`_ for adding UUID support.
+-  Dhanvantari Tilak `Dhanvantari`_ for Bug-Fix:
+   ``TypeError in _get_numbers_distance() when ignore_order = True``.
 
 .. _Sep Dehpour (Seperman): http://www.zepworks.com
 .. _Victor Hahn Castell: http://hahncastell.de
@@ -89,6 +92,8 @@ Authors in order of the timeline of their contributions:
 .. _Tony-Wang: https://github.com/Tony-Wang
 .. _eggachecat: https://github.com/eggachecat
 .. _SlavaSkvortsov: https://github.com/SlavaSkvortsov
+.. _havardthom: https://github.com/havardthom
+.. _Dhanvantari: https://github.com/Dhanvantari
 
 Thank you for contributing to DeepDiff!
 

docs/changelog.rst

@@ -5,6 +5,15 @@ Changelog
 
 DeepDiff Changelog
 
+-  v5-7-0:
+
+   -  https://github.com/seperman/deepdiff/pull/284 Bug-Fix: TypeError
+      in \_get_numbers_distance() when ignore_order = True by
+      @Dhanvantari
+   -  https://github.com/seperman/deepdiff/pull/280 Add support for
+      UUIDs by @havardthom
+   -  Major bug in delta when it comes to iterable items added or
+      removed is investigated by @uwefladrich and resolved by @seperman
 - v5-6-0: Adding custom operators, and ignore_order_func. Bugfix: verbose_level==0 should disable values_changes. Bugfix: unprocessed key error.
 - v5-5-0: adding iterable_compare_func for DeepDiff, adding output_format of list for path() in tree view.
 - v5-4-0: adding strict_checking for numbers in DeepSearch.

docs/conf.py

@@ -60,9 +60,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '5.6.0'
+version = '5.7.0'
 # The full version, including alpha/beta/rc tags.
-release = '5.6.0'
+release = '5.7.0'
 
 load_dotenv(override=True)
 DOC_VERSION = os.environ.get('DOC_VERSION', version)

docs/faq.rst

@@ -5,7 +5,7 @@ F.A.Q
 
 
 Q: DeepDiff report is not precise when ignore_order=True
------------------------------------------------------
+--------------------------------------------------------
 
     >>> from deepdiff import DeepDiff
     >>> from pprint import pprint
@@ -58,7 +58,7 @@ Bump up these 2 parameters to 1 and you get what you want:
 
 
 Q: TypeError: Object of type type is not JSON serializable
--------------------------------------------------------
+----------------------------------------------------------
 
 I'm trying to serialize the DeepDiff results into json and I'm getting the TypeError.
 

docs/index.rst

@@ -4,7 +4,7 @@
    contain the root `toctree` directive.
 
 
-DeepDiff 5.6.0 documentation!
+DeepDiff 5.7.0 documentation!
 =============================
 
 *****************
@@ -27,18 +27,17 @@ The DeepDiff library includes the following modules:
 
 - **Commandline** Most of the above functionality is also available via the commandline module  :doc:`/commandline`
 
-*************************
-Supported Python Versions
-*************************
-
-DeepDiff is rigorously tested against Python 3.6 up to 3.10 and Pypy3
-
-NOTE: Python 2 is not supported any more. DeepDiff v3.3.0 was the last version to supprt Python 2.
-
 ***********
 What is New
 ***********
 
+New In DeepDiff 5-7-0
+---------------------
+
+- https://github.com/seperman/deepdiff/pull/284 Bug-Fix: TypeError in _get_numbers_distance() when ignore_order = True by @Dhanvantari
+- https://github.com/seperman/deepdiff/pull/280 Add support for UUIDs by @havardthom
+- Major bug in delta when it comes to iterable items added or removed is investigated by @uwefladrich and resolved by @seperman
+
 New In DeepDiff 5-6-0
 ---------------------
 

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 5.6.0
+current_version = 5.7.0
 commit = True
 tag = True
 tag_name = {new_version}

setup.py

@@ -10,7 +10,7 @@
 if os.environ.get('USER', '') == 'vagrant':
     del os.link
 
-version = '5.6.0'
+version = '5.7.0'
 
 
 def get_reqs(filename):

tests/test_delta.py

@@ -197,7 +197,8 @@ def test_list_difference_add_delta_when_index_not_valid(self, mock_logger):
         delta = Delta(diff, log_errors=False)
         assert delta + t1 == t1
 
-        expected_msg = ELEM_NOT_FOUND_TO_ADD_MSG.format(20, 'root[20]')
+        # since we sort the keys by the path elements, root[3] is gonna be processed before root[20]
+        expected_msg = ELEM_NOT_FOUND_TO_ADD_MSG.format(3, 'root[3]')
 
         delta2 = Delta(diff, verify_symmetry=True, raise_errors=True, log_errors=False)
         with pytest.raises(ValueError) as excinfo:
@@ -588,6 +589,20 @@ def test_delta_dict_items_added_retain_order(self):
         'to_delta_kwargs': {},
         'expected_delta_dict': {'values_changed': {"root[4]['b']": {'new_value': 'world!\nGoodbye!\n1\n2\nEnd'}}}
     },
+    'delta_case17_numbers_and_letters': {
+        't1': [0, 1, 2, 3, 4, 5, 6, 7, 8],
+        't2': [0, 1, 2, 3, 4, 5, 6, 7, 8, 'a', 'b', 'c'],
+        'deepdiff_kwargs': {},
+        'to_delta_kwargs': {},
+        'expected_delta_dict': {'iterable_item_added': {'root[9]': 'a', 'root[10]': 'b', 'root[11]': 'c'}}
+    },
+    'delta_case18_numbers_and_letters': {
+        't1': [0, 1, 2, 3, 4, 5, 6, 7, 8, 'a', 'b', 'c'],
+        't2': [0, 1, 2, 3, 4, 5, 6, 7, 8],
+        'deepdiff_kwargs': {},
+        'to_delta_kwargs': {},
+        'expected_delta_dict': {'iterable_item_removed': {'root[9]': 'a', 'root[10]': 'b', 'root[11]': 'c'}}
+    },
 }