This is a very good point. Many people just assume that bigger budget automatically means better quality output.
While Lord of the Rings and The Hobbit has nothing to do with The Expanse; we saw what Peter Jackson achieved with limited resources he had with LotR. Because of the success of that, he had too much when it came to making The Hobbit. Look how that turned out; more was certainly not better.
The Expanse creators are not Peter Jackson though, so it depends whether they understand what made the first 3 seasons so good. If they do, then maybe they can avoid pitfalls and continue to make the show as good as it has been up until now. If not, then maybe like Jackson, they will get distracted by new shiny more expensive toys to play with...
Costs of shows go up due to wages and things, tying down increasingly popular and in demand writers, cast or directors and other people to contracts to keep it going.
Second off, Jackson didn't do a good job with The Hobbit, but it also wasn't his project. The entire thing was put together by Guillermo del Toro, then he either left the project because the studio was pushing for things he didn't want or the studio pushed him out because they didn't like what he was doing. Jackson came in stupid late with half the film, sets and I think all the casting done meaning there was only so much he could change. Also because of the concurrency of filming them he couldn't really get through one film and fix the others. The project was mishandled from the start.
I don't think a big budget really means much of anything, it can depend on many things. It could mean they tied up the actors for series 4 and 5 and to do so meant offering more money, it could be that Amazon asked them for ideas for S4 and they had various options and one meant visiting more worlds or more sets and would cost more and Amazon approved it. That's often how shows work in that they'll have an idea or path that can save money, more stuck on a single ship stuff, less sets and less costs but other ideas for episodes that would cost more. So a bigger budget could mean no core changes, no more CGI but just Amazon chose what would usually be their preferred, less cut down ideas for Season 4 than the budget options. It could also mean throwing more battles and CGI in for the sake of it, you won't know till later on but if it's the same show runners and they know what has made the show work no reason to assume they'll change it up massively for S4.
I kind of want to read the books but don't want to ruin the show so decided not to, so no idea what's supposed to come next. If it's a action heavy season coming up that actually requires more cgi and cost to do right, it could be that it makes perfect sense for the cost to go up, just as S5 could be less space stuff and cheaper to do and the costs go down.