A local LLM on our own hardware, who benefits?
A handful of use cases with strict data isolation. For most SMBs it costs more than a cloud DPA.
Try this first
- 1Who benefits: organisations contractually unable to send data outside their own server room (defence supply, some health, some legal).
- 2Hardware: a 70B-class model (Llama 3.x 70B or similar) wants at minimum 1x H100 80GB for FP16; with quantisation (FP8/INT4) it runs on weaker GPUs at a quality trade-off. That is investment, not subscription.
- 3Operations: model updates, monitoring, security patches. Budget half an FTE for ongoing operations if you mean it.
- 4Quality: open-source models can compete with earlier-generation closed-source models on specific tasks after fine-tuning, not automatically general purpose.
- 5Run the business case first. Many parties asking this never even checked Azure OpenAI with EU residency under a DPA.
When to bring us in
Pre-investment check before buying hardware: we run the TCO compare between local and cloud, saves a costly misstep.
See also
- Can I paste a customer file or email into ChatGPT?Depends on the account and settings. Free ChatGPT and a Team tenant behave very differently from what most people assume.
- I want a one-page AI policy for my teamA real one-pager beats a thick document nobody reads. Four headers and concrete examples.
- How do I tell if an AI answer is made up?Models sound confident even when they are wrong. A few habits catch most mistakes.
None of the above fits?
Describe your situation below. We pass your input plus the steps you already saw to our AI and return tailored next-step advice. If it's too risky to DIY, we'll say so.
Or skip the DIY entirely
Our Managed IT clients do not look these things up. One point of contact, a fixed monthly price, resolved within working hours.