Overconfidence and the illusion of skill

Overconfidence is among the best-documented findings in psychology and among the most expensive in investing.

The general finding

Across domains.

People consistently rate themselves above average at tasks where that is statistically impossible for most of them.

Confidence intervals given by experts are systematically too narrow, meaning the true answer falls outside the stated range far more often than it should.

And confidence correlates poorly with accuracy in many judgement tasks, sometimes negatively.

The investing evidence

Specific studies.

Analyses of retail brokerage accounts have consistently found that more frequent trading is associated with worse net returns after costs.

The classic finding that men traded more frequently than women and earned correspondingly less has been replicated and is generally attributed to overconfidence.

Investors who switched from telephone to online trading have been found to trade more and perform worse, attributed to the illusion of control created by information and speed.

And self-directed investors consistently overestimate their own past returns when asked, relative to their actual records.

The mechanisms

How it operates.

Illusion of knowledge: more information increases confidence considerably faster than it increases accuracy.

Illusion of control: the ability to act creates a sense of influence over outcomes that are largely random.

Self-attribution bias: good outcomes are attributed to skill and poor ones to bad luck, which prevents learning from the record.

Hindsight bias: past events appear more predictable than they were, which sustains the belief that future ones are predictable.

And survivorship in what we see: successful traders are visible and unsuccessful ones are not, which distorts the apparent base rate.

Where it costs money

Specifically.

Frequent trading, incurring costs and spreads.

Concentrated positions based on conviction.

Attempting to time entry and exit.

Chasing recent performance.

Using leverage.

Believing that research on a company confers an advantage over professional participants with far more resources.

And ignoring the base rate — that most active participants underperform after costs — on the assumption of being an exception.

The particular danger of early success

Worth noting.

An investor whose first decisions work out attributes it to skill, increases position sizes and frequency, and is then exposed to a larger loss.

Bull markets produce a great many people who believe themselves skilled, and the belief is tested only when conditions change.

Which is why the period after a sustained rise generates the most confident retail participation and the most subsequent damage.

Correctives

What helps.

Keeping a written record of every decision and the reasoning at the time, then reviewing it later — which is uncomfortable and is the single most effective corrective, because it prevents the retrospective editing that self-attribution bias performs.

Calculating your actual return, including timing of contributions, rather than relying on impression.

Comparing against a simple benchmark: what a broad index fund would have produced with the same contributions.

Considering the base rate before assuming you are an exception.

Reducing the frequency of decisions through automation.

Adding friction: a mandatory delay before acting on any idea.

And separating any speculative activity into a small defined portion of the portfolio, which contains the damage.

The role of a written policy

Which addresses several biases at once.

A statement of what you hold, why, what allocation you target, how you rebalance, and what would cause you to change.

Written when calm, referred to when not.

It converts decisions into a comparison against a pre-existing plan rather than a fresh judgement, which is where overconfidence operates.

And it creates a record against which the reasoning can later be assessed.

The uncomfortable framing

Worth stating.

The question is not whether markets can be beaten — some participants beat them.

The question is whether you have a reason to believe you are among them, and what that reason is beyond a general sense of competence.

Professional participants with substantial resources mostly fail to beat the market after costs, which is the relevant base rate.

And an investor who accepts this constructs a portfolio that does not require them to be exceptional, which is a considerably more robust position.

General information only, not investment advice. Investments can fall in value and past performance does not indicate future returns. Consult a regulated financial adviser.