I got the idea of trying to run a local LLM on one of my Rocky Linux 8 computers but every front end (framework?) I try errors out looking for a newer glibc or libc++. Jan actually allowed me to download a llm but errored out looking for a newer libstdc++.so.6 when I tried to interact with the llm I downloaded. The others that I tried (gpt4all and lm-studio) didn’t finish loading themselves before they quit.
Has anyone successfully run a local LLM on Rocky 8? Or am I outta-luck unless I upgrade to version 9? I’m really not in a hurry to do that and this is the first thing I’ve come across that I want to do with el8 and can’t.