OpenCode + Openrouter is a good start if you don’t want to pay for Claude Code. There are usually some free models you can try out.There are so many tools, configurations out there that's it's a bit overwhelming. Where do I start? What setup should I have, which tools?
I've been using Ollama & Aider and run aider from a batch file so I can switch between models (one batch file per model I use).
-
Notice regarding image hosts and attachments. See here.
AI Models thread
- Thread starter mid_gen
- Start date
I've got a free week of Claude Pro I can give out if its allowed?There are so many tools, configurations out there that's it's a bit overwhelming. Where do I start? What setup should I have, which tools?
I've been using Ollama & Aider and run aider from a batch file so I can switch between models (one batch file per model I use).
For locally hosting LLM have a look at Strata (https://github.com/niko1221/strata)
It utilises GPU, RAM, and storage to run Qwen3.8-Flash-Next at useable speeds.
Soldato
- Joined
- 19 Oct 2008
- Posts
- 6,131
Cheers but I've paid for Claude for months, only cancelling recently as go through periods where I don't need it and I managed to get done most of what I needed to do for now. Trialling local LLM's to see if can use any as a good alternative, moving forward.
Last edited:
Done some rejiggling on my CI server and made some existing jobs run on the intel IGPU, and having a go at running Qwen 3.8 coder with Strata. I don't expect it to be terribly good on a 9060XT but I'm interested in seeing how these simple local models can start doing some donkey work (i.e. driving my playtest sessions).
I wouldn't call it CI in my case but I'm looking to do similar with testing a few service builds etc. to ease upgrading bits and bats. For stuff I don't mind taking hours, even days when I'm busy elsewhere don't even care if it's running on CPU!
I test quite a few AI tools as part of my role, between off-the-shelf products and industry-specific ones.
What I've noticed is that:
1. Most industry specific ones talk like ChatGPT / Gemini – like an excited work intern who wants to please rather than disappoint.
2. There's a difference between a powerful and useful tool.
Claude is very powerful, but not very useful since it lacks any real context. When you give it good data, it's incredible.
The industry specific tools we're using are very useful because they have access to lots of data that it understands, but as a product they're not very powerful yet.
What I've noticed is that:
1. Most industry specific ones talk like ChatGPT / Gemini – like an excited work intern who wants to please rather than disappoint.
2. There's a difference between a powerful and useful tool.
Claude is very powerful, but not very useful since it lacks any real context. When you give it good data, it's incredible.
The industry specific tools we're using are very useful because they have access to lots of data that it understands, but as a product they're not very powerful yet.
Interesting @MrRongCar , it's where I've found Claude most useful in my work, being an assistant to retrieve and process datasets to my liking, but only after the data sources have been as good intel.
I'm interested to hear about the challenges people have had with the hype and what's seemingly, at least in my workplace, turned into competition of who can do someone else's job themselves which, frankly, is driving me mental.
I'm still determined to get some local stuff running soon but for now I appear to be tied to Claude
I'm interested to hear about the challenges people have had with the hype and what's seemingly, at least in my workplace, turned into competition of who can do someone else's job themselves which, frankly, is driving me mental.
I'm still determined to get some local stuff running soon but for now I appear to be tied to Claude
Last edited:
Local models are still quite limiting and not useful for doing anything without close supervision. Plus the economics of it just don't add up until you're needing to serve a decent amount of concurrency and can justify a six-figure hardware spend.Interesting @MrRongCar , it's where I've found Claude most useful in my work, being an assistant to retrieve and process datasets to my liking, but only after the data sources have been as good intel.
I'm interested to hear about the challenges people have had with the hype and what's seemingly, at least in my workplace, turned into competition of who can do someone else's job themselves which, frankly, is driving me mental.
I'm still determined to get some local stuff running soon but for now I appear to be tied to Claude
I use Deepseek 4.1 flash for some simple tasks and automations as it's super fast and cheap, and good enough when it's not got to problem solve....but until Anthropic up their prices massively, there's no reason to not use Opus 5.5 for everything.
Anthropic are clearly putting a lot of effort into distilling these models down and getting their inference costs lower, learning from the techniques DS 4.1 flash has led the way with no doubt.
Simply because they are tiny and outdated models relatively speaking and simply won't be able to draw niche information like that from their limited training data.
LLMs will always come up with some kind of plausible sounding answer, it's on the user to understand the capabilities and context available to be able to make a call on how trustworthy it's output is.