Local & sovereign AI

Monster
Scanner

A Qwen 2B vL model, ON AN iPhone

A working example of using local AI on an iPhone. Qwen3-VL-2B on an iPhone 14 Pro, with no network connectivity.

AI Engineering meetup · Bill Mcintyre · October 22 2026

Seven Questions

What we set out to explore

On-device LLM

Can a local OSS LLM reasonably run on an iPhone?

What it takes

Which runtime, models, hosts, and quantizations fit in memory?

Capabilities

Can it analyze photos and answer in strict, tool-ready JSON?

Performance

Fast enough for photo analysis in realtime, under 1 GB?

Deterministic

Can the model narrate within a testable code harness?

Guardrails

Can every AI response stay safe, reasonable and in-bounds?

Fallback

Can we maintain flow when/if the model fails?

Monster Scanner · Halloween edition

The question behind the game: can a real LLM live inside a deterministic iPhone app? Before writing any code, these were the seven questions. One: can a real language model run on an iPhone with no cloud? Two: what does it take: which runtime, which model size and quantisation fit in memory? Three: capabilities: can it see a photo and answer in strict JSON that code can act on, the same building block as tool calling? Four: performance: fast enough for a ten-second scan, under a gigabyte, even when the phone is warm? Five: can it live inside a deterministic app, where the model narrates and testable code decides? Six: guardrails and safety: can every AI sentence stay kind and never comment on looks or identity? Seven: fallback: does the game still work when the model is missing, slow or too hot? The rest of the talk answers them: the model and runtime slides for one and two, the jobs and grammar slides for three, performance and heat for four, the rule and architecture for five, guardrails for six, and Classic Mode throughout for seven.

A Monster Scanner

Scan a friend. Find their inner monster.

parameters

A16

iPhone 14 Pro

network calls

Vampire · Alien · Kaiju · Werewolf · Zombie · Robot · Ghost · Certified Human

Monster Scanner · Halloween edition

AI (monster) Brain, Deterministic Wrapper

The model only narrates

The model

Describes the photo and adds playful lines.

The code

Measures, decides and checks every AI sentence.

Monster Scanner · Halloween edition

How It Works

Photo

In memory

Classic · always

Vision: face, tilt, eyes
Labels: hats, glasses
Pixels, clock, moon

AI observer

Qwen3-VL-2B, llama.cpp
Grammar-forced JSON
Strict kindness check

Verdict engine

Deterministic Swift
Seeded by the photo
47 clues: clothes, pose

Reveal

Code-written jokes
AI’s playful lines
Voice, sound, haptics

late or hot: skip

Monster Scanner · Halloween edition

1.55 GB

on disk

< 1 GB

peak memory

Meet Qwen3-VL-2B

Part	Bits	Size
Language model	4-bit	1,107 MB
Vision projector	8-bit	445 MB

Apache 2.0 open weights from the official Qwen org. Files: Qwen3VL-2B-Instruct-Q4_K_M and mmproj-Qwen3VL-2B-Instruct-Q8_0, both GGUF.

Monster Scanner · Halloween edition

Three Jobs

Observer

Every scan

Reads the photo, returns JSON

Chaos Mode

Opt-in

Scores monsters; code clamps

AI Notes

Experimental

Streams checked notes

Monster Scanner · Halloween edition

Why This Model

Official GGUFs from Qwen itself

llama.cpp runs its architecture

Fits beside the camera on 6 GB

Apache 2.0: free to ship

Tested in PocketPal first

Not (yet)

Qwen3.5-2B

No official GGUF yet

MLX

The fallback runtime

F16 projector

819 MB, kept in reserve

Monster Scanner · Halloween edition

llama.cpp on Metal

Official prebuilt xcframework · build b11799 · MIT · mtmd for images

No Simulator slice

Device builds only

A Swift actor

Main thread stays free

Lifecycle

Loads in 0.73 s

A Mac twin

126 tokens/s on a Mac

Monster Scanner · Halloween edition

JSON Every Time

Setup (Mac, LlamaProbe)	Tokens/s	Output
No grammar	130	JSON in a code fence
Grammar, full ~152k vocabulary	55	Valid JSON
Grammar, whitespace squeezed	55	Invented “glasses, headphones”
Grammar after top-k 40	126	Valid; crashes above temp 0
Check-then-resample (shipped)	~105	Always valid

5 of 5 valid on the phone

Monster Scanner · Halloween edition

Four More Lessons

Keep loops in C

Swift loop: −25% speed

Decode greedily

Same photo, same reading

Examples get copied

Show style, not samples

Only what you see

No more invented hats

On a props photo: “a top hat” · “two red balloons” · “like a happy, slightly sleepy explorer”

Monster Scanner · Halloween edition

unkind sentences kept, out of 187 in 50 field notes

Three Safety Layers

1 · Prompt rules

Clothes, props, pose, setting

2 · Grammar limits

Letters only, 100 chars max

3 · Kindness check in model

Safety reinforced in code

Word lists: unkind, age, looks, stereotypes

Monster Scanner · Halloween edition

Fast When Cool

Seconds per photo · dot = middle of the measured range · final prompt unless noted

9.5 s model deadline

Mac M3 Max, final prompt

iPhone, short prompt

iPhone, ordinary photo

iPhone, detail-rich photo

Before the sampler fix

Hot iPhone (“serious”)

1.1–1.6 s

4.4–4.5 s

6.4–6.6 s

8.5 s

11 s

15–20 s

0 s

5 s

10 s

15 s

20 s

Source: LlamaProbe (Mac) · in-app benchmark (iPhone 14 Pro, Debug)

9.1 s

worst reveal

budget: 12 s

Heat Is the Monster

“Serious” heat = half speed

Then the app skips the model

Soak: “fair” at 6, “serious” at 8

Scans 9–10 used Classic. No crash.

Load 0.73 s

~20 tokens/s

978 MB peak

Monster Scanner · Halloween edition

Works Offline

One download

HTTPS, pinned, Wi-Fi only

Pinned by hash

SHA-256 every launch

Checked first

GitHub, HF and CVE lists

One binary

llama.cpp + Qwen, that’s it

Nothing stored

Photos stay in memory

Approval gate

New stuff needs a human yes

Monster Scanner · Halloween edition

After a one-time 1.55 GB download, photos, model and results never leave the phone; it works in airplane mode. The only network code is the model download: HTTPS to the official Qwen org on Hugging Face, pinned to one repo commit, size shown first, Wi-Fi only by default. The runtime and both model files are pinned by SHA-256 and hashed again before the first load every launch, 1.41 seconds on the phone; a mismatch deletes the files and the app stays in Classic Mode. The build and repo were checked against GitHub, Hugging Face and the NIST CVE database first. One binary (llama.cpp, MIT) and one set of weights (Apache 2.0): no analytics, ads or accounts. Photos stay in memory, there is no face recognition, and shared images carry no metadata. New dependencies, network calls and permissions need the owner’s yes.

Apple Does the Rest

Framework	Used for
Vision, Core Image	Face geometry, smiles, blinks, hats and setting
AVFoundation, Core Haptics	Camera, narration voice, synthesised sounds, haptics
SwiftUI	Every screen and the certificate image
PhotosUI	Photo import with no library permission
CryptoKit	SHA-256 checks and the per-photo seed
URLSession	The model download: the only network code

Monster Scanner · Halloween edition

Built by an Agent

Claude Code (Opus 5.5)

Wrote it all, built, deployed

Helper agents

Sounds, effects, certificate

LlamaProbe

Same engine, on the Mac

devicectl + tests

Benchmarks, soak, 76 tests

Working agreement

Logs kept together

Small commits

Ask before adding

Flag spec problems

Monster Scanner · Halloween edition

An AI coding agent, Claude Code running Claude Opus 5.5, built the app from a written spec: the code, tests, scripts and docs. It built and deployed to the Simulator and the iPhone from the command line and ran the benchmarks. Helper agents in separate git worktrees built the sound effects, reveal effects and certificate in parallel. LlamaProbe is a Mac CLI on the same engine source, for prompt experiments. devicectl and scripts ran the benchmarks, a 10-scan soak and a 50-note kindness check; Swift Testing runs 76 tests covering verdict balance, kindness, the parser and determinism. The working agreement: plan, progress and decision logs updated together; small commits with human review at every milestone; ask before any dependency, network call or permission; flag spec problems rather than silently improvising.

Nine Lessons

Model only sees

Code decides

Use a grammar

No broken JSON

Loops in C

Swift cost 25%

Design for heat

Plan a fallback

Describe style

Don’t show it

Build a Mac twin

1-second tests

AI is optional

Failures vanish

Pin and verify

Hash every file

Seed everything

Test the mix

Monster Scanner · Halloween edition

The biggest lesson: give a small local model the job it is good at, looking and describing, and keep judgement, jokes and safety in code. A grammar turned usually-JSON into always-valid JSON, but the sampler design decides whether that costs a few percent or more than half your speed. One Swift loop over the vocabulary cost about 25% of phone speed, so keep per-token work in C. Heat, not peak speed, is the phone’s real limit: plan the fallback. Small models copy examples word for word, so describe the style instead. A Mac twin of the engine turned every phone deploy into a one-second experiment. With a full non-AI mode, every failure is invisible to the player. Pin the commit, size and SHA-256 of every file and check before load. And a seeded, deterministic engine lets tests measure the verdict mix: a player found “everyone is a robot” twice, and now a balance test catches it first.

The Whole Stack

Component	Version	Licence	Purpose
llama.cpp xcframework	b11799	MIT	Inference on Metal + mtmd
Qwen3-VL-2B Q4_K_M	commit 52d6c8f	Apache 2.0	Language model, 1,107 MB
Qwen3-VL-2B mmproj Q8_0	commit 52d6c8f	Apache 2.0	Vision projector, 445 MB
Apple frameworks	iOS 17+ SDK	Apple SDK	Camera, Vision, speech, UI
MonsterCore package	in the repo	Ours	Verdicts, prompts, jokes

Target: iPhone 14 Pro (A16, 6 GB) · iOS 17+ · built on a MacBook Pro, M3 Max

Monster Scanner · Halloween edition

QR code linking to github.com/BillDX/monster-scanner

Scan me

Thank
You

Questions?

The model narrates. The code decides.

Copyleft, GPL-3.0: use it, change it, share it, keep it open.

Bill Mcintyre · github.com/BillDX/monster-scanner

MonsterScanner

Seven Questions

A Monster Scanner

AI (monster) Brain, Deterministic Wrapper

How It Works

Photo

Classic · always

AI observer

Verdict engine

Reveal

Meet Qwen3-VL-2B

Three Jobs

Observer

Chaos Mode

AI Notes

Why This Model

Not (yet)

llama.cpp on Metal

No Simulator slice

A Swift actor

Lifecycle

A Mac twin

JSON Every Time

Four More Lessons

Keep loops in C

Decode greedily

Examples get copied

Only what you see

Three Safety Layers

1 · Prompt rules

2 · Grammar limits

3 · Kindness check in model

Fast When Cool

Heat Is the Monster

Works Offline

One download

Pinned by hash

Checked first

One binary

Nothing stored

Approval gate

Apple Does the Rest

Built by an Agent

Working agreement

Nine Lessons

Model only sees

Use a grammar

Loops in C

Design for heat

Describe style

Build a Mac twin

AI is optional

Pin and verify

Seed everything

The Whole Stack

ThankYou

Monster
Scanner

Thank
You