Running a LLM on unsupported GPU : a tale for the bravest
Preface In a burst of determination to hop on the modern train, I decided to run my own LLM model locally. I quickly realized that my GPU is unsupported by the ROCm technology. However, thanks to amazing technological improvements, it is still possible to circumvent this to run LLM models on unsupported hardware. As it happens quite often during researching and/or debugging stuff on the fly, I quickly got myself into an unnecessary rabbit hole and lost several hours for almost nothing, but I still managed to run various medium-sized models (up to four billions parameters) pretty easily.