depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.
you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.
depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.
Thanks, I’ll dig into this. BTW, I have a 9070 XT, with 16 Gb VRAM, so it should do the job I guess.
you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.