Local OCR with Ollama and Qwen2.5-VL: Perfect on Screenshots, Confidently Wrong on Blurry Photos
I often need to get a table out of a screenshot: a parts list from a supplier page, a spec table from a PDF, a statement from an app that won't export. Cloud vision models do this well, but some screenshots I'd rather not upload anywhere. So I run OCR locally with Ollama and qwen2.5vl:7b, a 7B vision-language model, on my ASUS Ascent GX10.
For this post I tested it on a parts table I generated, so the correct answer is known exactly. The short version: on a clean screenshot it was perfect, in about 5 seconds. On a small, blurry image it invented a part that isn't there, and it can't be trusted to add up the numbers it reads.
Setup
ollama pull qwen2.5vl:7b # about 6 GBThen a small script that sends the image to Ollama's chat API. It uses only the Python standard library:
#!/usr/bin/env python3
"""Local OCR with Ollama + qwen2.5vl: turn a screenshot (e.g. a table) into Markdown.
The image never leaves your machine.
Usage:
python3 ocr_table.py screenshot.png
python3 ocr_table.py screenshot.png "List only the part numbers"
"""
import base64
import json
import sys
import time
import urllib.request
MODEL = "qwen2.5vl:7b"
DEFAULT = ("Transcribe all text in this image exactly as written. "
"Format tables as Markdown tables. Do not add any explanation.")
if len(sys.argv) < 2:
sys.exit(__doc__.strip())
image, prompt = sys.argv[1], " ".join(sys.argv[2:]) or DEFAULT
body = {
"model": MODEL,
"stream": False,
"options": {"temperature": 0}, # same image -> same answer
"messages": [
{"role": "system", "content": "You are a precise OCR assistant. "
"If something is unreadable, write [unreadable] instead of guessing."},
{"role": "user", "content": prompt,
"images": [base64.b64encode(open(image, "rb").read()).decode()]},
],
}
req = urllib.request.Request("http://127.0.0.1:11434/api/chat",
json.dumps(body).encode(),
{"Content-Type": "application/json"})
t = time.time()
reply = json.load(urllib.request.urlopen(req, timeout=600))
print(reply["message"]["content"].strip())
print(f"\n[{time.time() - t:.1f} s]", file=sys.stderr)Two settings matter. temperature: 0 makes the output repeatable, so the same image gives the same text every time (I checked: two runs were identical). And the system prompt tells the model to write [unreadable] instead of guessing. As you'll see, it doesn't always obey.
Test 1: a clean screenshot
This is the test image: a 9-row parts list with units (kΩ, µF), part numbers and prices, at normal screenshot resolution (660 × 326 pixels).
The output, unedited:
| Part | Qty | Value | Package | Unit price |
| --- | --- | --- | --- | --- |
| Resistor | 12 | 4.7 kΩ | 0603 | $0.012 |
| Resistor | 4 | 220 Ω | 0805 | $0.010 |
| Capacitor | 8 | 100 nF | 0603 | $0.018 |
| Capacitor | 2 | 10 µF | 1206 | $0.095 |
| LED (red) | 6 | Vf 2.0 V | 0805 | $0.041 |
| NPN transistor | 3 | 2N2222A | SOT-23 | $0.064 |
| Voltage regulator | 1 | AMS1117-3.3 | SOT-223 | $0.210 |
| ESP32-C3 module | 1 | 4 MB flash | SMD | $2.350 |Every cell is correct, including the Ω and µ symbols, AMS1117-3.3 and SOT-223. The first run took 9.8 seconds including loading the model into memory; after that each image took about 5 seconds. I also tried a half-size copy (340 × 173 pixels): still 100% correct, in 5.9 seconds.
Test 2: a small, blurry, tilted image
Next I made the image look like a bad phone photo: shrunk to 40%, rotated 3 degrees, slightly blurred, and saved as a low-quality JPEG (272 × 144 pixels).
The output:
| Part | Qty | Value | Package | Unit price |
| --- | --- | --- | --- | --- |
| Resistor | 12 | 4.7 kΩ | 0603 | $0.012 |
| Resistor | 4 | 220 Ω | 0805 | $0.009 |
| Capacitor | 8 | 100 nF | 0603 | $0.018 |
| Capacitor | 2 | 10 μF | 1206 | $0.095 |
| LED (red) | 8 | VF 2.0 V | 0805 | $0.043 |
| MOSFET | 1 | IRFZ44N | SOT-23 | $0.064 |
| Voltage regulator | 2 | AMS1117-3.3 | SOT-223 | $0.210 |
| ESP32-C3 module | 1 | 4 MB flash | SMD | $2.350 |The table structure survived, and most cells are right. But compare it with the original:
- The NPN transistor, qty 3, 2N2222A row became MOSFET, qty 1, IRFZ44N. That's a real part number, and it isn't anywhere in the image.
- LED quantity 6 became 8, and its price $0.041 became $0.043.
- The regulator quantity 1 became 2; one resistor price $0.010 became $0.009.
- It never wrote
[unreadable]. Every wrong value looks as confident as the right ones.
This is the important failure mode. The model doesn't produce garbled text when it can't read something. It produces plausible text: a common MOSFET in place of a common transistor, nearby digits in place of the real ones. A quick glance at the output won't catch it.
Test 3: asking it to do the math
Back to the clean image, I asked: "What is the total cost of all parts (Qty times Unit price, summed)? Show the calculation." It listed every line item correctly (12 × $0.012 = $0.144, and so on), then added them up:
$0.144 + $0.040 + $0.144 + $0.190 + $0.246 + $0.192 + $0.210 + $2.350 = $4.286The correct sum of those eight numbers is $3.516. Every input was right and the final addition was wrong by 22%. Use the model to read the numbers, and do the arithmetic in code.
How I use it now
- Screenshots, not photos. Native-resolution screenshots were perfect. Photos of screens and paper are where it starts inventing.
- Ask for transcription only. Get the Markdown table, then parse it and compute totals in Python.
- Check the numbers that matter. Quantities, prices and part numbers: compare a few against the source, especially if the image is small.
- Crop and enlarge. If the source is small, crop to just the table and upscale before sending it; more pixels per character is what helped.
For sensitive screenshots, this is now my default: nothing leaves the machine, and on a GX10 it's fast enough that I don't think about it. For the speed of the text models on the same machine, see Ollama on the GX10: real tokens per second.
Comments
Post a Comment