How do I combine VLLM with flash Attention-based llama

[alex1996-ljl]

How to combine VLLM and llama based on flash attention? My current application is llama model based on flash attention, but I want to improve efficiency through vllm. Do you have any scheme that can effectively combine the two?

[robertgshaw2-redhat]

vllm uses flash attention (+ many other optimizations for inference). There is nothing you have to do to enable these, should work out of the box

[alex1996-ljl]

But isn't the core technology of VLLM paged attention? If I intend to use flash attention as my primary attention mechanism, will it conflict with the structure of VLLM? Or can I just implement a new layer of flash attention that is completely usable? (I'm a newbie and don't know much about these things.)