Please help me pick a model for my 3060.

Edit: As of late Jan 2025, Gemma 2 9b, Qwen 2.5 Coder 14b are the best I've used. Llama 3.1 7b is also decent. Deepseek R1 14b is interesting, it doesn't ever stop thinking. All models are from Ollama, which be default uses Q4. I've set the KV_Q to 8 and context length to 8k. Using the PageAssist front end. Have fun!

I want to use it for general assistant tasks (explaining things, coding, summarising) and to build toy LLM-apps. Basically I'm looking to dip my toes into the local LLM space.

There are so many models though, and so many variants (different parameters, FPs, mixtures). How do I pick? If you have any suggestions, please let me know.

Thanks!