Qwen3.8-Flash-Next applies the fundus laterality rule backwards

An AI server inside the medical center

We are installing a local artificial intelligence server in a medical center. Patient images and records stay in the building. The model we chose, Qwen3.8-Flash-Next, is an open model that reads text and images, and it runs on two of the server’s graphics cards.

Before connecting the model to any clinical use, we test it on tasks where we know the answer. One of them is saying which eye a fundus photograph shows. For an ophthalmologist, this is immediate: the optic disc, the bright spot the vessels radiate from, lies on the side of the nose. On a fundus photograph taken from the front, a disc in the left half of the image therefore indicates a left eye.

We gave the model a normal fundus photograph of a left eye and asked it to describe it and state the laterality. It answered “right eye”. Its description of the image was accurate: optic disc cupping, temporal scleral crescent, vascular arcades. Its justification was not: “The optic disc is located on the left of the image and the macula in the center-right. Since the nasal pole is therefore on the left, this is a right eye (classic rule: disc on the left = OD; disc on the right = OS).”

The model found the disc in the right place and quoted a reversed rule. Increasing the image resolution changed nothing. A system message stating the correct convention made it answer “left eye”.

The protocol

A single image proves nothing. We took 30 fundus photographs from the public IDRiD dataset, 15 right eyes and 15 left eyes spread across the whole test set, 20 of which show diabetic retinopathy. For each one, the dataset’s authors provide the position of the optic disc and the fovea as marked by experts, which gives the laterality without ambiguity. We added the mirror image of each photograph, which turns a right eye into a left eye. That makes 60 images, with as many right eyes as left eyes. Figure 1 shows one of these pairs.

Each image received three questions, asked in English. Each one offers two answers, labeled A and B, and the model must reply with a single letter.

The convention added to question 3 is the anatomical rule given above. Here is the text sent to the model:

Convention: the photograph is oriented as seen by an examiner
facing the patient, and the optic disc lies on the nasal side
of the macula. Hence a disc in the left half of the image means
a LEFT eye, a disc in the right half means a RIGHT eye.

In other words, the photograph is seen by an examiner facing the patient, the optic disc is on the side of the nose, and a disc in the left half therefore indicates a left eye.

The model does not write an answer. We use SemIf, an open replica of Jev, TypeSafe AI’s decision model. We introduced it in our article on decision models. SemIf reads the probability the model assigns to the letter A and to the letter B.

A language model may prefer answer A, whatever it says. To measure this preference and then remove it, each question was asked twice, with the answers swapped: once with Right eye (OD) as A and Left eye (OS) as B, once with Left eye (OS) as A and Right eye (OD) as B. For each image, we then average the two probabilities obtained for each answer, and we keep the answer whose average is above 0.5. In total, 360 decisions (60 images, 3 questions, 2 versions of the answers), without a single technical error.

The results

Figure 2 gives the share of correct answers. Question 2 appears twice: with the raw answers, then with the answer options swapped, which removes the preference for answer A.

Perception is good. The model knows where the optic disc is, with a median probability of 0.95.

The model is biased: it prefers answer A, whatever it says. When the text of answers A and B is swapped, it still answers A almost every time. Reminding it of the laterality rule makes it give the right eye in 100% of cases.

On question 2, the model picks answer A in 118 decisions out of 120, whether it names the right eye or the left eye. This position bias is known in multiple-choice questions put to language models, and it hides everything else: the score, 58 correct answers out of 120, looks like chance. Once this preference is removed by averaging over the swapped answers, laterality is wrong for 51 images out of 60.

The reversal is systematic, but the preference is weak: the median probability of the correct answer is 0.45. It shows up on the original photographs (26 out of 30) as well as on the mirror images (25 out of 30), on healthy eyes (17 out of 20) as well as on diseased eyes (34 out of 40). When an image is flipped, the model’s answer switches in 21 pairs out of 30. So it does follow what it sees, and it then applies the rule the wrong way round.

On question 3, the convention is written into the question. Out of 120 decisions, only two are wrong, and narrowly: 0.52 against 0.48. After averaging over the swapped answers, no image is wrong.

This fix comes down to one sentence added to the question: the model is configured in the question, in plain language. A classic classifier learns its rule from examples. The one by Liu et al., designed for laterality alone, required 18,394 annotated fundus photographs, and changing what it does requires new images and new training. Here, an ophthalmologist reads the rule in the question, checks it and changes it in a few seconds, and the same method works for other questions without retraining anything.

What is already known

The difficulty multimodal models have with fundus laterality has already been published. FunBench, a benchmark presented at MICCAI 2025, asks GPT-4o, Qwen2.5-VL, InternVL2.5, LLaVA and other models “left eye or right eye?” on fundus photographs from several public datasets, including IDRiD. They all score close to chance. The authors also show that the part of the model that encodes the image contains the information: a simple classifier trained on its outputs recognizes the eye in more than 8 cases out of 10. They attribute the failure to how rarely the subject appears in training text, laterality being too basic for anyone to write it down.

The preference for the first option is also documented: language models favor certain answer letters regardless of content, and this effect is corrected by permuting the options.

We found no work describing the mechanism we observe: an optic disc located correctly on every image, then a wrong eye in 85% of cases once the preference for answer A is removed, well below chance, and a reversed rule that the model states itself. The raw answers gave 58 correct out of 120, a score that looks like chance. A score close to chance can therefore hide a reversed rule, masked by position bias.

What we changed

Every laterality question put to the model now contains the anatomical rule, as written in question 3. The model already places the optic disc in the right spot, and this reminder is enough for it to deduce the right eye.

We also ask each question twice, with the answers swapped, and average the results. Without that, position bias alone would have hidden the reversed rule, and we would have concluded that the model was answering at random.

The limits of this test

The 30 eyes come from a single public dataset, and all the photographs are centered on the macula. We tested a single version of the model, compressed to fit on our graphics cards, and we do not know whether the full version makes the same mistake. The comparison with other models covers only one image. Finally, laterality is only a first check: we are preparing a glaucoma test based on OCT images.

If you are considering a local AI server for your center or practice, a thirty-minute conversation is usually enough to find out what it can take on. Write to us at the bottom of the page, or book a slot directly in our calendar.

Sources

• Porwal et al., Indian Diabetic Retinopathy Image Dataset (IDRiD)
  Public dataset of fundus photographs with the position of the optic disc and the fovea marked by experts, under a CC BY 4.0 license. We used 30 images from its localization subset.

• Wei et al., FunBench: Benchmarking Fundus Reading Skills of MLLMs (MICCAI 2025)
  Benchmark of fundus reading by multimodal models; the laterality task stays close to chance for every model tested.

• Liu et al., A self-adaptive deep learning method for automated eye laterality detection based on color fundus photography (PLoS One, 2019)
  Neural network trained on 18,394 fundus photographs to recognize the right or left eye; 99.13% correct answers on 2,000 external validation images.

• Zheng et al., Large Language Models Are Not Robust Multiple Choice Selectors (ICLR 2024)
  Measurement of language models’ bias toward certain answer letters in multiple-choice questions.

• Theodore Lee, SemIf, GitHub repository
  Method that reads the probability of each option from an open model, without having it write an answer.