What AI is actually bad at

It doesn't know when it doesn't know#

The single most consequential weakness, and the root of most of the others. A model produces the most plausible continuation of your text. Plausible and true overlap heavily, which is why it works at all, but nothing in the process distinguishes them. There is no internal signal that fires on "I have no idea," so the output looks identical whether the model is reciting something well-established or assembling something it has never seen. Confidence is a property of the writing style, not a measurement of reliability. → Hallucination

This is why "are you sure?" is such a weak test. It often produces a revision, because a challenge makes agreement the plausible continuation, and that revision is no better grounded than the original.

It has no clock and no continuity#

A model does not experience time passing, has no idea how long ago it was released, and remembers nothing from your last conversation unless the product around it stores that and feeds it back in. Each request starts from nothing but what is in front of it. → Context windows

Knowledge stops at a fixed date, and the gap between that date and today widens every day the model exists. Everything after it is invisible: releases, prices, appointments, whether a company still exists, what the current advice is. → Training cutoff

It is unreliable about itself#

Self-report is the weakest category of all, and the reason is structural. The bulk of what a model knows came from material collected before it existed, so most of what it can say about itself is assembled from descriptions of earlier systems. Some information about a model does reach it afterwards, through later stages of training and through the standing instructions a product wraps around it, which is why an assistant reliably knows its own name and version. That is a patch over a gap rather than a fix for it, and it does not make self-report a form of observation: nothing gives a model access to its own training run, or to how it actually behaves, in the way it has access to text.

• Its own cutoff date is inferred rather than known, and the inference skews early.
• Its own capabilities are guessed from how such systems were described in its training data, which predates it.
• Its own reasoning is not reliably reported either. Models given a hint have been measured using it and then explaining their answer without mentioning it. → Thinking and reasoning

So "which model are you and what can you do?" is one of the least reliable questions you can ask, and it is asked constantly.

Exactness, counting, and character-level work#

A model does not see letters. It sees tokens, chunks of text that often span several characters, which makes anything operating below that level awkward in a way that has nothing to do with intelligence. Counting letters in a word, reversing strings, careful spelling games: these are structurally uncomfortable, not merely unpractised. → Tokens

Related, and more expensive in practice: anything requiring exactness rather than fluency. Arithmetic beyond the trivial, precise counts, tallies across a long document, strict reconciliation. The right response is not a better prompt but a tool: let it write the query, run the calculator, execute the code, and use the result. → Tool use

Questions about a whole collection#

Questions of the form "how many," "which ones don't," "what changed across all of these," and "list every instance" are much harder than they sound, and they fail quietly rather than loudly. A retrieval system fetches passages resembling your question, and no single passage resembles a question about an entire collection. → RAG and retrieval

Negation is the sharpest case. "Find the contracts that do not mention indemnity" cannot be answered by finding text similar to that request. You get a confident, plausible, incomplete list, with nothing to indicate that it is incomplete.

Long tasks, quietly#

Reliability decays as steps accumulate. Each step inherits the previous ones' mistakes as though they were established facts, so a very good per-step success rate still compounds into an unreliable run over enough steps. The arithmetic is worth doing yourself, because it is the entire argument: a 99% chance of getting each step right is a 90% chance of getting ten steps right, 60% over fifty, and 37% over a hundred. Nothing about the model got worse. The steps multiplied. → Agents

The related weakness is knowing when to stop. A model judging its own completion tends toward declaring success, so the characteristic failure of a long task is not a crash but a confident report on a job that was half done.

Consistency#

Ask the same question twice and you may get two different answers, both reasonable. That is a property of how text is sampled, not a bug, and it is the same property that makes the output feel natural rather than robotic.

It matters most where people least expect it: anything requiring reproducibility, auditability, or two people getting the same answer from the same query. If a process depends on identical output every time, that needs to be designed for rather than assumed.

Thinly-covered subjects#

Quality tracks how well-represented something was in training data. Widely documented subjects get strong performance; niche ones get weaker performance delivered in exactly the same confident register. Specialist domains, minority languages, small local knowledge, internal company practice, anything mostly discussed in private or in places the training data didn't reach.

The trap is that the register never changes. A model is not quieter about a subject it barely knows, which means the reader gets no signal precisely where one is most needed.

What it is not bad at, despite the reputation#

A limits page that only lists failures gives a false picture, so, briefly: these systems are genuinely strong at transforming text you supply (rewriting, restructuring, summarising, translating between formats), explaining concepts at whatever level you ask for, drafting a first version of almost anything, extracting structure from mess, and writing code with a human reviewing it.

The pattern worth noticing: they are strongest when the material is in front of them and weakest when they must supply the material from memory. Nearly every practical technique on this site is a variation on moving work from the second category into the first.

Sources#

This page describes mechanisms rather than quoting figures, so most of it is cited by the concept page it points at. Two claims here are empirical and belong to somebody, so they are named. That models given a hint can use it and then explain the answer without mentioning it comes from Anthropic's Reasoning models don't always say what they think, also cited on Thinking and reasoning. The compounding arithmetic above is just arithmetic: 0.99 raised to the power of the number of steps, which you can check on any calculator and which is why no citation is offered for it.

Added 12 August 2026. This page shipped without a sources block while every other page here had one, which is the sort of inconsistency that is easy to miss precisely because nothing on the page looks wrong.