I’ve been doing some more experimenting.
Answering my own question here:
llama.cpp compiles and works fine on Rocky 8.
I have Meta-Llama-3-8B-Instruct.Q2_K.gguf running on this computer right now. 
It isn’t breaking any speed records but it works fine.