PocketTune — Your phone is 5× faster than your AI thinks.

The problem

One APK.
A dozen different chips.

Android phones run wildly different Arm silicon under the same OS: different matrix instructions, different core layouts, different memory bandwidth. Almost every local-LLM app ships one generic build and hopes for the best, which leaves most of the phone's speed unused.

PocketTune answers the one thing a spec sheet can't: what's actually fastest on the phone in your hand.

1 build

ships to every phone

On-device LLM apps compile once, generically — using none of the SIMD instructions the chip actually has.

?

the spec sheet lies

The chip with more matrix instructions should win. Measured on four phones, it doesn't: the biggest winner is missing the headline feature entirely.

6 ≠ 2

"all cores" is wrong

Extra threads land on slow little cores and everyone waits for them. The right count differs per phone.

0 data

nobody publishes this

No public per-phone dataset says which config wins where — so every phone has to find out for itself.

Measured

faster at reading your prompt

Measured

faster at writing the answer

Measured

0%

gain from the library everyone assumes helps

Always

0

cloud calls. Nothing leaves the phone.

These are the speed ranges across , each one measured against its own unoptimized baseline. The spread is the whole point: the same optimization is worth a different amount on different silicon, so no single number describes your phone. See the per-device evidence ›

The evidence

Same method.
Every phone its own answer.

One panel per device we've benchmarked. Same model, same workload — only the build changes.

Read each ladder top to bottom. The speedup shows up with the compiler flags, which are really just permission to use the matrix instructions the silicon already has. Which instructions exist, and what they're worth, is a property of the chip, not the app. That's the reason we measure instead of assume.

What we found

Three results the
folklore gets wrong.

1.0–5.6×

The compiler flag is the whole ballgame — when the silicon plays.

One build flag lets the compiler emit the SIMD instructions the chip already has, and most Android builds never pass it. How much it buys moves around unpredictably: 1.00×, 3.88×, 4.94×, 5.27× across four phones. It doesn't track the feature list either — the biggest winner has no i8mm at all, and on a chip below dotprod the flag buys nothing, because there's nothing for it to unlock.

+0%

KleidiAI adds nothing — and can even cost.

Arm's own acceleration library, tested head-to-head: no measurable gain on the i8mm chip, and a measured −3.6% on the Pixel 7a. Plain compiled code already saturates this hardware for Q4_0, and llama.cpp's own repack path beats the kernels on Tensor G2. On other quants and other silicon it may well flip, which is why it stays in the sweep.

2 · 2 · 4 · 6

The right thread count is per-phone.

Four phones, four different optima, and no, it isn't “all of them” by default. The two big.LITTLE chips want 2; the tri-cluster Pixel wants 4, because it has exactly four fast cores, and asking it for 6 drops prefill from 143 to 102 tok/s. But the Snapdragon 710 really does want all 6: its big cores are weak enough that the little ones still add throughput. Even the direction of the rule flips from chip to chip.

Every one of these contradicts what a reasonable developer would assume, and none of them holds across chips by default. Which is the whole reason the app works the way it does: your phone's fastest configuration is a question you can only answer by measuring, so PocketTune measures it on your phone.

How it works

Detect. Sweep.
Apply. Chat.

01

Read the silicon

Which fast instructions does this chip have — dotprod? i8mm? How many fast versus slow cores? Read straight off the device.

02

Benchmark on-device

Compression formats, instruction paths, thread counts — measured for speed, memory and energy per token, right on the phone.

03

Install the winner

The best configuration isn't a report. It becomes the app's live settings, chosen on evidence.

04

Chat offline

A private assistant running the tuned setup. Airplane mode is the demo — nothing leaves the phone.

Conceptual — each cell is one benchmarked configuration

warming up…

Under the hood

Five layers.
TypeScript to silicon.

Every line we wrote is TypeScript. Every fast path below it is measured, never assumed.

Arm silicon llama.cpp · C++ llama.rn bridge tuning core · TS four screens · TS

01

Four screens — Device · Models · Tune · Chat

React Native, all TypeScript. Device reads the silicon, Models manages GGUF downloads, Tune runs the sweep and applies the winner, Chat runs it offline.

02

Tuning core

Builds the sweep plan from this phone's core layout, drives the benchmarks, scores every configuration (weighted 65% decode, 35% prefill), samples battery power for tokens per joule, and persists the applied config and history.

03

llama.rn bridge

A prebuilt binding to the engine — no native code written in this project. It ships six arm64 kernel builds and picks one at startup by reading the chip's features: i8mm silicon gets the i8mm build, a dotprod-only chip gets the dotprod build.

04

llama.cpp engine

Loads the GGUF model and runs the matmuls. One context lives at a time — a phone doesn't have RAM for two — and every engine call is serialized, so a mid-benchmark tab switch can never corrupt a measurement.

05

Arm silicon — whichever chip is in the phone

The ground truth: which matrix instructions exist (dotprod? i8mm? SVE2?) and where the big cores sit. Different on every phone — which is exactly why every layer above asks instead of assumes.

The whole stack exists to take one decision — which configuration to run — and settle it by measurement on the phone in your hand, rather than by guesswork.

The optimizations

Seven levers.
Two families.

Build-time levers are baked into the binaries and proved by the harness. Runtime levers are what the app tunes on your phone. Every one is measured, and two of the "obvious wins" turned out to be worth nothing.

SIMD · instruction setBuild-time

Arch-targeted compilation

-march=armv8.2-a+dotprod+i8mm gives the compiler permission to emit the chip's matrix-multiply instructions. Generic arm64 builds — what most apps ship — use none of them. This single flag is the largest gain in the entire project.

1.0–5.3×

prefill vs generic build, across four phones (Realme 5 Pro 1.00× · A34 3.88× · 2a 4.94× · Pixel 7a 5.27×). Same flag, same model — the spread is the silicon, and it is not predictable from the feature list. Below dotprod the flag is worth nothing at all: the floor is measured, not assumed

Data layoutLoad-time

Weight repacking

At model load, llama.cpp rearranges Q4_0 weight blocks into the tile order the i8mm instructions consume — so the SIMD units stream contiguous data instead of gathering it. Free at inference time; it costs a moment at load.

included

folded into the 4.94×; isolated by the norepack builds in the attribution ladder

Kernel libraryBuild-time

KleidiAI microkernels

Arm's hand-written matmul routines — the thing everyone assumes brings the speed. Tested head-to-head on the i8mm chip: ≈0% once the compiler flags are right. On the Pixel 7a, isolating it against a KleidiAI-free control went further — KleidiAI costs ~3.6% there, because llama.cpp's own repack path beats the kernels for Q4_0. It stays in the sweep because other silicon and other quantizations may disagree.

≈0% / −3.6%

Q4_0 · Nothing 2a / Pixel 7a, the two chips where it was isolated · the honest negative

QuantizationModel format

4-bit weights

Decode speed is bounded by memory traffic, not arithmetic — the CPU mostly waits for weights to arrive from RAM. Shrinking every weight from 16 bits to ~4 cuts that traffic roughly 4×, and is what makes a 1B-parameter model fit on a phone at all.

~700 MB

Llama 3.2 1B at Q4_0, down from ~2.3 GB at f16

ThreadingRuntime · app

big.LITTLE-aware thread count

More threads is actually worse when the extra workers are little cores, because the whole team waits for the slowest. And the winning count is different on every phone: the two big.LITTLE chips want 2 threads, while the tri-cluster Pixel wants 4, because it has exactly four fast cores (2× X1 + 2× A78) and the 5th thread spills onto an A55. The app builds its candidates from this phone's actual core map instead of defaulting to "all of them".

up to +41%

choosing the right thread count instead of “all 6” · Pixel 7a prefill 101.8 → 143.5 t/s at 4 threads; Galaxy A34 decode 12.8 → 17.3 t/s at 2. The winning count is 2 on the MediaTek phones and 4 on the Pixel — no single default is right twice

Attention pathRuntime · app

Flash attention + quantized KV cache

Flash attention computes attention in cache-sized tiles instead of materializing the whole score matrix; a q8_0 KV cache halves the memory the growing conversation occupies. Both help some chips and hurt others — which is why they're swept, not defaulted.

swept

measured per device by the in-app tuner

EnergyMeasurement

Tokens per joule

Where the kernel exposes the battery's power rails, every configuration is also scored for energy per token. On a phone, a config that's 5% faster but 30% hungrier is a bad trade — efficiency is a ranking axis, not an afterthought.

per config

sampled live during each sweep point

Measured numbers come from the four phones that have actually been through the harness — MediaTek, Google and Qualcomm silicon: one with i8mm, two with dotprod only, one with neither — and trace to raw JSON in results/. They disagree, and the disagreement is the finding: the same compiler flag buys 1.00×, 3.88×, 4.94× and 5.27×, and the biggest winner is a chip with no i8mm while the pre-dotprod chip gains nothing at all. You can't read the answer off a spec sheet. The kind of each optimization carries over to any Arm phone; its value doesn't, and closing that gap is what the app is for.

Reproducible by design

One command reruns
every number.

Plug in any Android phone. The harness reads its silicon, sweeps the builds, and writes its own JSON — no phone is special-cased.

$ python harness/bench.py --model models/Llama-3.2-1B-Instruct-Q4_0.gguf

The app

Four tabs.
One tuned phone.

Everything on this page ships as one Android app. It reads the chip it was installed on, benchmarks the configurations that chip can actually run, applies the winner, and leaves you with a private assistant running exactly that setup.

The harness proves the gains on a bench; the app is where they land. The whole loop of detect, sweep, recommend and apply happens on the phone, with no cloud and no spec-sheet guessing.

Device tab — “Knows your exact silicon”. Reads the Arm CPU features and the big.LITTLE core map straight from the phone: dotprod, i8mm, SVE2, and which cores are the fast ones.

Device tab — “The right kernel, automatically”. Runtime dispatch loads the inference binary built for this exact CPU, and every config gets a tokens-per-joule score.

Device tab — “What actually fits”. Model sizes weighed against real RAM, leaving headroom for the KV cache, the app and Android itself.

Models tab — “A model matched to your phone”. Device-aware recommendations; Q4_0 repacks into this chip's fast kernels at load time, or bring your own GGUF.

Tune tab — “One tap to tune”. Sweeps thread counts, flash attention and KV-cache layout on the device itself, with no cloud and no guessing from a spec sheet.

Tune tab — “Every configuration, measured”. Decode speed for each config, benchmarked live on the phone and charted as each result lands.

Tune tab — “Faster replies, proven”. The winning config with decode, prefill, tokens-per-joule and power draw: 1.16× faster than the llama.cpp default on this phone.

Tune tab — “Baseline vs tuned”. The speedup where you feel it: decode rate, time to first token, and the wait for a full reply.

Tune tab — “Fast isn't always efficient”. Tokens-per-joule from the battery rail; the config that wins on speed doesn't always win on battery, so the app measures both.

Tune tab — “Apply in one tap”. The best config is saved and carried straight into chat; the whole loop of detect, sweep, recommend and apply runs on the phone.

Chat tab — “Private, and fully offline”. The model runs on your phone and works in airplane mode; nothing you type ever leaves the device.

Chat tab — “Real answers, on-device”. Full markdown replies from a local LLM running the configuration tuned for this exact silicon.

Chat tab — “Measured speed on every reply”. Real tokens-per-second under each message — the same metric the tuner optimizes, verified in everyday use.

Four tabs, one loop. Device reads the Arm features, the core map and what will actually fit in memory. Models recommends the ones matched to this chip — or takes your own GGUF. Tune sweeps thread counts, flash attention and KV-cache layout on the phone itself, scores each result for speed and tokens per joule, and applies the winner. Chat runs the tuned model offline, printing measured tokens per second under every reply. React Native and TypeScript over llama.cpp via llama.rn; the screens above are a Nothing Phone (2a). See the loop step by step ›

Stop guessing.
Measure.

Every component — quantization, i8mm, compiler flags, the harness — explained from zero.

Download the app Inside PocketTune

One APK.A dozen different chips.

ships to every phone

the spec sheet lies

"all cores" is wrong

nobody publishes this

Same method.Every phone its own answer.

Three results thefolklore gets wrong.

The compiler flag is the whole ballgame — when the silicon plays.

KleidiAI adds nothing — and can even cost.

The right thread count is per-phone.

Detect. Sweep.Apply. Chat.

Read the silicon

Benchmark on-device

Install the winner

Chat offline

Five layers.TypeScript to silicon.

Four screens — Device · Models · Tune · Chat

Tuning core

llama.rn bridge

llama.cpp engine

Arm silicon — whichever chip is in the phone

Seven levers.Two families.

Arch-targeted compilation

Weight repacking

KleidiAI microkernels

4-bit weights

big.LITTLE-aware thread count

Flash attention + quantized KV cache

Tokens per joule

One command rerunsevery number.

Four tabs.One tuned phone.

Stop guessing.Measure.

One APK.
A dozen different chips.

Same method.
Every phone its own answer.

Three results the
folklore gets wrong.

Detect. Sweep.
Apply. Chat.

Five layers.
TypeScript to silicon.

Seven levers.
Two families.

One command reruns
every number.

Four tabs.
One tuned phone.

Stop guessing.
Measure.