llama.cpp AMD ROCM cuBLAS GPU Acceleration

by paralin
GNU/Linux ◆ xterm-256color ◆ bash 171 views

AMD ATI Radeon RX 6750 XT running a 13B model (StableBeluga)

llama.cpp with cuBLAS and rocm running in Docker on SkiffOS