AMD RX 6900 XT is selected but not used

[tilkinsc]

Feature request

Currently, I am unable to get GPT4All app to use my Rx 6900 xt. Instead, it uses my ryzen 9 3900 x despite me forcefully setting it to my GPU. Model: Wizard Uncensored. Windows.

Motivation

I am unable to achieve sane generation times with the parameters I want.

Your contribution

I've never developed for AI. Only used it. Sorry.

[cosmic-snow]

It looks like there are still a few bugs (with AMD especially) that need to be sorted out.

Also, do you have enough VRAM to fit both the model + some more for all generated text in it? If not, it'll fall back to CPU automatically.

I think partial offloading is planned, but if you don't have the VRAM and need it now, you'd simply have to get a GPU with more of it.

[tilkinsc]

Rx 6900 xt has 16gb of ram

[cebtenzzre]

The latest pre-release displays a specific message explaining why the selected device was not used.

[tilkinsc]

Ill give it a spin

[tilkinsc]

So I installed the latest pre release and I said hi, then it kept spitting out #'s until I cancelled the response, actively using my gpu. Mistral OpenOrca, defaults. Restarting didn't fix anything. Falcon does ; instead of #.

[manyoso]

What operating system and driver. If you are running on Linux you should try RADV driver. The AMDGPU driver is known to be problematic on Linux.

[tilkinsc]

Updated main post with windows 10

[manyoso]

Appears as duplicate of #1507