2dcad9…6ab387

i've experimented a bit, lm studio, vllm, on a ryzen 5 ai + 96GB system ram, the problem isn't really the inference, it's the prefill that's super slow