Update Benchmarks and Documentation for GraniteCausalLM

[fabianlim]

In this PR we update the benchmarks for GraniteCausalLM

• in addition, the README.md is also updated to describe how a new model can be added in the future
• NOTE: we did not update the GPTQ results in the bench. will do this possibly at a later time.

Note this PR requires the following dependency updates

• transformers>=4.45: for GraniteCausalLM
• accelerate>=0.34.1: required for transformers>=4.45 if GraniteCausalLM is needed.
• trl > 0.11.1: when using baseline bnb, requires this fix for a bug that was introduced in transformers==4.45 Fix Inconsistency with IsShardedQLoRA Setting huggingface/trl#2089
• bitsandtbyes==0.43.3: it seems that the later versions give segmentation fault errors

Known issues with quant peft

• single GPU w/o FOAK
• single GPU w FOAK -> fused lora dequant problem (this is an issue with the compiled binaries in bitsandbytes 0.43.3, that is not compatible with maybe the CUDA toolkit or torch version)
• multi GPU w/o FOAK -> rank 1 stuck at prepare_model (this is resolved by disabling low_cpu_mem_mode)
• multi GPU w FOAK -> meta device problem (see 2 in Distributed Training Problems for QLoRA models with Transformers pre-release 4.45  #83) (this is resolved by disabling low_cpu_mem_mode)
• bad loss with BNB+FOAK -> (resolved by updating lora fused ops to support bias)

Performance

Overall impressive improvements with kernels.

FULL FT

PEFT

Quantized Peft (BNB)

[wynterl]

awesome, great results @fabianlim

[raghukiran1224]

Indeed, awesome results @fabianlim !

[fabianlim]

@wynterl @raghukiran1224 the loss for BNB + fused ops looks problematic. Needs more debugging, Ok i found that its because Granite has a bias in the Linear, but the FOAK kernels do not support bias. This just requires some minor (but tedious) modifications