Just for the the contribs name

A Python intereface to Lllama.cpp which I have been playing with on a 8gb rk3588 and its extremely usable and can be rather addictive with what prompts you can use.
The model and user space just tips over 4gb and on my 4gb the swapping really slowed perf so maybe a 8gbPi but prob say rk3588 is likely a better minimum.

2 Likes