Skip to content

Commit

Permalink
Deterministic sorting in modular converter when adding new functions (h…
Browse files Browse the repository at this point in the history
…uggingface#35795)

deterministic sort
  • Loading branch information
Cyrilvallez authored Jan 21, 2025
1 parent 920f34a commit e867b97
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions utils/modular_model_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -773,6 +773,9 @@ def compute_relative_order(self, missing_dependencies: set[str]) -> dict[str, in
original_dependencies.append(class_dep)
else:
merged_dependencies.append(class_dep)
# We need to sort deterministically before actual sorting, so that entries missing (i.e. with value 1e10)
# will always get the same order independently of the system (they come from a set, which has no deterministic order)
original_dependencies = sorted(original_dependencies, reverse=True)
# Sort both list according to the order in their respective file
original_dependencies = sorted(original_dependencies, key=lambda x: self.start_lines.get(x, 1e10))
merged_dependencies = sorted(merged_dependencies, key=lambda x: self.modular_file_start_lines[x])
Expand All @@ -798,6 +801,9 @@ def compute_relative_order(self, missing_dependencies: set[str]) -> dict[str, in
merged_dependencies.append(dep)
else:
original_dependencies.append(dep)
# We need to sort deterministically before actual sorting, so that entries missing (i.e. with value 1e10)
# will always get the same order independently of the system (they come from a set, which has no deterministic order)
original_dependencies = sorted(original_dependencies, reverse=True)
# Sort both list according to the order in their respective file
original_dependencies = sorted(original_dependencies, key=lambda x: self.start_lines.get(x, 1e10))
merged_dependencies = sorted(merged_dependencies, key=lambda x: self.modular_file_start_lines[x])
Expand Down

0 comments on commit e867b97

Please sign in to comment.