Running a LLM on unsupported GPU : a tale for the bravest

Preface In a burst of determination to hop on the modern train, I decided to run my own LLM model locally. I quickly realized that my GPU is unsupported by the ROCm technology. However, thanks to amazing technological improvements, it is still possible to circumvent this to run LLM models on unsupported hardware. As it happens quite often during researching and/or debugging stuff on the fly, I quickly got myself into an unnecessary rabbit hole and lost several hours for almost nothing, but I still managed to run various medium-sized models (up to four billions parameters) pretty easily.

Windows Remote Thread Injection

Fundamental of Process Injection To understand this project, let’s first define the core concept: Process Injection. According to MITRE : Process injection is a method of executing arbitrary code in the address space of a separate live process. This technique can grant access to the target process’s memory, system/network resources, and potentially elevated privileges. This definition perfectly encapsulates the technique used in this project. Brief description of the project This project began as a suggestion from a friend when I was seeking a simple yet practical way to start learning malware development.