Gemini Pro vs Gemini Flash: Which is more accurate?

In my Udemy course, I tried to extract the outcome for a VAERS report using both Gemini Flash and Gemini Pro (latest).

class RecoveryStatus(Enum):
    Yes = "Yes"
    Recovered = "Yes"
    Recovered_With_Treatment = "Yes"
    No = "No"
    Recovering = "No"
    Resolving = "No"
    Recovering_With_Treatment = "No"
    Unknown = "Unknown"

class Explanation(typing.TypedDict):
    matching_sentence_if_any: str
    explanation: str

class Symptom(typing.TypedDict):
    symptom_value: str
    symptom_recovery_status: RecoveryStatus

class PatientInfo_v5(typing.TypedDict):
    recovered_from_all_symptoms: RecoveryStatus
    recovered_from_all_symptoms_explanation: Explanation
    symptom_list: list[Symptom]

Here is how it works:

• the LLM first needs to extract the full list of symptoms that the patient experienced
• then it has to extract the outcome for each symptom
• the worst outcome among all the symptoms should be selected
• Y = Yes, N = No, and U = Unknown

I first tried using Gemini Pro and got pretty poor accuracy on this task.

Surprisingly, even though Gemini Flash is supposed to be a “less powerful” model (which means it is both cheaper and faster!) it comfortably outperformed Gemini Pro for this particular task.

So I ended up creating the course chapter using Gemini Flash.

This is why one of my 10 tips for improving LLM accuracy is to also test the less powerful models from the same vendor.

While you may not get the same results, it is still a good idea to set up a quick test to figure out which LLM is most accurate for your specific task.