Cognitive Baseline: Calibrating your Mental Model
Activity Overview
Before using AI for complex tasks, it is crucial to establish a "cognitive baseline"—an understanding of what the raw model can and cannot do without tools (like search or code execution) or specific prompting strategies.
In this activity, you will test the semantic understanding of the model, explore its hallucinations, and see how size affects reasoning.
Semantic Ability & Failure Modes
The Critique Loop
Go to your preferred AI playground or chat interface.
- Ask the model to write a short text (2-3 paragraphs) about a niche topic you know well (e.g., "The history of the local coffee shop in your town" or "The rules of a made-up game").
- Start a new chat.
- Paste the generated text back in and ask: "Critique this text for factual accuracy and style."
Observe: Does the model hallucinate errors? Does it offer generic praise? This reveals the "stochastic parrot" nature when it lacks ground truth.
Compare Factual Recall
Fact Check
Ask Gemini 2.5 Pro, Flash, and Flash Lite (or comparable models) the exact same factual questions. Ensure you turn off "Google Search" grounding if possible to test intrinsic knowledge.
- "Who won the 1982 World Cup?"
- "What is the population of Ostrava?"
Compare consistency. Do smaller models hallucinate more often?
Model Size Effects
Size Matters
Pick a model family (like Gemma or Llama) at multiple sizes (2b, 7b, 70b) via a playground like HuggingFace or AI Studio.
Ask identical logic questions (e.g., "Sally has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?").
Note the difference in the output quality and the time it takes to generate.
Reasoning Toggles
Thinking vs. Predicting
Using Gemini 2.5 Flash (or a model with a "Thinking" toggle), run a complex logic puzzle.
- Run 1: Thinking OFF.
- Run 2: Thinking ON.
Note the difference in the output steps. Does the model "show its work"? Does it catch its own errors?
Code Execution for Math
Tools vs. Tokens
LLMs are bad at math because they predict tokens, they don't calculate numbers.
- Ask: "Calculate the 150th Fibonacci number." (With code execution OFF).
- Ask the same question with Code Execution ON.
The first will likely give a plausible-looking but incorrect number. The second will write a Python script to calculate it exactly.