AI Bits for Techies | Issue #3 | 26 Jan 2026

AI Bits for Techies | Issue #3 | 26 Jan 2026

Your weekly Aussie-flavoured deep dive into what changed in AI/ML, what matters, and what to do next (without living on release-note social media).

This week in one breath: A paper benchmarking the energy, water, and carbon footprint of LLM inference showing 70x+ differences between models, tools for local image generation and multilingual translation, and a shift in thinking: "model quality" might actually be "infrastructure efficiency"—the same model can use 70% less energy just by switching infrastructure.

The one paper you should pretend you read at lunch

How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference

What is the setup?

Everyone argues about training being expensive, but the real day-to-day bill is inference: the prompts you send all day, every day. The paper's point is simple: we still lack clean, standardised, prompt-level numbers that factor in infrastructure, not just "the model."

What they did (yes, really)

They benchmarked 30 models using public API performance data, then layered on infrastructure multipliers like PUE and regional carbon intensity, with hardware configurations inferred statistically. They also use a probabilistic (Monte Carlo) approach and rank "eco-efficiency" with cross-efficiency DEA.

What happened

The spread is massive. The most energy-hungry models (they call out o3 and DeepSeek-R1) can exceed ~33 Wh per long prompt, which is 70x+ more than much smaller/efficient deployments. At the other end, they note a single short GPT-4o query at about 0.43 Wh, which looks tiny until you scale it.

Why it is interesting (beyond the number)

Because "same model" does not mean "same footprint." Datacentre overhead and where the workload runs can swing the outcome dramatically. In other words: model choice matters, but infrastructure choice is right behind it.

The real question

If per-query costs keep getting cheaper and faster, usage will explode anyway (Jevons vibes). So the real builder question becomes: are you optimising for capability only, or for capability per watt, per litre, per tonne of CO₂, at the scale your product is heading?

Full paper: https://arxiv.org/abs/2505.09598

Tools worth poking this week (in a sandbox first)

FLUX.2 [klein] (Black Forest Labs)

Best for: Developers and creators with consumer-grade hardware. These compressed models allow for professional-grade image generation and multi-reference editing directly on local GPUs (like the RTX 3090/4090) with sub-second response times.
https://bfl.ai/

TranslateGemma (Google)

Best for: Building low-latency, multilingual applications. These open-weights models are optimized for speed and accuracy across 55 languages, fitting on everything from H100 GPUs down to mobile devices while maintaining high-fidelity translation.
https://deepmind.google/models/gemma/

GLM-Image (Zhipu AI)

Best for: Designers requiring precise text rendering and multi-subject consistency. This hybrid model combines autoregressive and diffusion techniques to excel at complex tasks like style transfer and generating high-resolution images with legible text.
https://github.com/THUDM/GLM-Image

Book recommendation (because your brain deserves more than changelogs)

The Atlas of AI – Kate Crawford

Crawford's work is the grounding counterweight to this week's paper. It zooms out and maps the physical stuff AI is actually made of.

Her core argument is blunt: AI is neither artificial nor intelligent. It is an extractive industry. One that runs on lithium mines, exploited labour, and relentless data harvesting.

Where this week's paper measures the operational hunger of LLMs, Crawford exposes the deeper, structural costs. The supply chains, the labour, the land.

The "cloud" stops looking fluffy very quickly. In her framing, it is a planetary-scale industrial system that centralises power and steadily drains natural resources.

Same story. Different layers.

Geeky thought of the day

Is "Model Quality" actually just "Infrastructure Efficiency"?

We often treat an AI's intelligence as a fixed property, but this research suggests that a model's "goodness" is actually an emergent property of the data center it lives in. If the same model uses 70% less energy and water simply by switching from a proprietary server to a highly optimized cloud provider, then a model's "quality" isn't just about weights and biases—it's about the grid, the cooling, and the hardware.

In an era of adaptive routing, we have to stop asking "How smart is this AI?" and start asking "How effectively can this infrastructure support its reasoning?"

Housekeeping (so we stay honest)

This is general information, not legal advice. If you ship user-facing AI, be transparent about where AI is used, what it cannot do, and where humans stay in the loop.

About the Authors

Dr Sam Donegan

Founder & Lead Editor

Sam leads the MLAI editorial team, combining deep research in machine learning with practical guidance for Australian teams adopting AI responsibly.

Jun Kai (Luc) Chang

AI Software Developer

Luc is an AI Software Developer at Monash AIM, building neural networks on FPGA boards. He is pursuing a Master of AI at Monash and co-founding a startup in the event space.

Julia Ponder

Technical Writer

Julia specialises in translating developer jargon into plain English. She creates clear, expertly formatted documentation and tests products before they go to market.

Shivang Shekhar

Technical Writer

Shivang is a mechanical engineer and AI masters student at Monash University with a diverse science background. He is the main author for AI Bits for Techies each week.