Spent most of today reworking a rack and rig of gpus for all of our internal ai work… our big server is 8 rtx 6000 pro and 3 psu, I definitely feel I made a mistake not upgrading our wall power to 240v but so far we have multiple 30amp 120v and with 3 PSU uninterrupted power we have been very stable. My big upgrade will be moving to epyc motherboard from threadripper so we get gen5x8 with bifurcation instead of what we are stuck with today gen5x4 due to bios limitations . What has been so encouraging though is first deepseek v4 flash at 200+ t/s for single user and much more in aggregate- now on qwen 3.8 flash next for image support and eyes on glm5.3 flash for some testing … next is realistically considering co location and quiet a sizable loan to scale this to real hardware instead of miner rip vibes
What kind of cost are you looking at and how many people could use it? I’d be interested to know what the payback period is like, because Claude code is getting ridiculously expensive.
Bought the rtx 6000 pros one per month starting in January- since they doubled in price I wish I just used a line of credit back in Jan to buy all of them. For me it’s about keeping internal company content internal. Slack channels etc with tools for teams to use. Triage tools for Zendesk etc.
love this and it's timely as I think with AI more of my code will just be native assembly for example: https://github.com/taf2/tic-tac-toe my m series native assembly based tic tac toe windowed game... very portable too you just simply execute codex --yolo port my app to x y z host...
I haven't played with yolo yet. I'll have to use my airgapped, old Lenovo ThinkPad for this! Whenever I see yolo, I can't stop thinking of Joseph Redmon's YOLO and his astounding resume format!
Children of Ruin is about advanced Octopi, not as good as Children of Time, but interesting. That series or Fire Upon the Deep are really good for conceptualizing truly alien intelligence.
the mountain in the sea is also a good read. focuses specifically on ingredient octopus, I didn't enjoy it as much as children of time but still an interesting read.
The one thing that keeps me in codex is that Claude and grok have done all of this work to make the cli tools feel like windows application with mouse etc… I want to scroll back with my terminal history not inside a window within my terminal….
Got this up and running at 250 tok/s about 2500 agg tok/s for 16 users and now this morning got https://huggingface.co/webbrain-one/DeepSeek-V4-Flash-0731-V... working - patched it to support multiple images... should post this soon to huggingface - having about gpt-5.6 terra high locally feels pretty good
reply