Skip to content

Commit

Permalink
populate quantization_config for kv-cache-scheme only configs (huggin…
Browse files Browse the repository at this point in the history
  • Loading branch information
horheynm authored and BernardZach committed Dec 5, 2024
1 parent 51e1e12 commit 4179a92
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/transformers/utils/quantization_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -1105,7 +1105,7 @@ def __init__(
self.sparsity_config = None

# parse from dict to load nested QuantizationScheme objects
if config_groups:
if config_groups or kv_cache_scheme:
self.quantization_config = QuantizationConfig.parse_obj(
{
"config_groups": config_groups,
Expand Down

0 comments on commit 4179a92

Please sign in to comment.