A September vision fix is a reminder to keep your image test set

Documentation snapshot: October 14, 2026. This guide discusses the dated release described below; check the linked official documentation for current API behavior and account availability before implementation.

If an image workflow suddenly gets worse, it is easy to blame the prompt. Sometimes the problem is further down the pipeline. On September 25, 2026, OpenAI reported a fix for image encoding affecting GPT-6 Sol and GPT-6 Luna and recommended rerunning affected evaluations. That is the specific event in the official changelog; it does not tell us how your own images performed. Documentation checked October 14.

Preserve the input you actually sent

An original photo is only the beginning. Your application may resize it, rotate it, crop it, compress it, or convert its color space. Keep a non-sensitive fixture and record the exact transformation settings and hash of the bytes sent. Otherwise a later comparison can quietly test a different picture.

For Japanese learning tools, build fixtures covering small kana, mixed kanji and Latin text, punctuation, vertical text if supported, and a low-contrast case. Use original or licensed material with a known transcription. Include an image with no useful text so the model has to admit that rather than inventing a sentence.

Score the useful outcome

Separate transcription accuracy from interpretation. A model might read a word correctly but explain its grammar incorrectly. For OCR, compare normalized text while preserving distinctions your application cares about. Decide whether spacing and punctuation count before scoring. For a reading aid, also check that highlighted regions correspond to the intended text.

A simple fixture record can contain image_sha256, transform_version, expected_text, language, and review_notes. Keep sensitive images out of shared test repositories. When the expected answer is uncertain, mark the fixture for human review rather than giving it a false gold label.

Isolate the changed component

Run the current and candidate model path on the same transformed bytes. Separately test a preprocessing change using the same model configuration. Include service failures and malformed outputs; a beautiful successful example does not prove the surrounding application recovers correctly.

Compare by category, not only one average. A gain on large English print can hide a regression in small Japanese annotations. Recheck the cases that previously failed and retain the results with dates and configuration. Provider changes are one reason to rerun a compact regression set even when you have not changed your app.

I would rather maintain twenty meaningful images than a folder of screenshots nobody can score. The point is to recognize a real change, locate it, and decide whether to ship—not to make every model look good on the same attractive demo.