Status: Active researchSovereign and Local AI
Local and Sovereign Inference
On-device and edge inference, model optimisation for constrained hardware, and multi-provider portability to reduce single-vendor dependence.
Abstract
We evaluate how much useful inference can run locally or at the edge, how models are optimised for constrained hardware, and how workloads remain portable across inference providers.
Problem & motivation
Reliance on a single external inference provider creates cost, privacy and sovereignty risks; the practical limits of local inference are unclear.
Research questions
- For which tasks is local inference already sufficient on commodity hardware?
- What is the accuracy/cost trade-off of optimisation techniques on edge devices?
Methods
- Benchmark local vs hosted inference across representative tasks.
- Measure optimisation effects on constrained hardware.
Limitations
- Benchmarks are internal and hardware-specific; they are not a universal ranking.
- Portability findings depend on the specific providers tested.
Disclosures
- Funding
- Infrastructure and hardware provided by Octopus Core Pty Ltd.
- Conflicts of interest
- Octopus Core offers commercial local-deployment products; findings are reported independently.