I had a hard time remembering this1, almost as bad as Type 1 vs Type 2 (which are still the worst terminology I’ve ever seen).
Thinking of these by reframing them with actual statistical terms made them easy to remember.
A random variable (the estimator) is accurate if it has low bias.
A random variable is precise if it has low variance.
From these definitions, it’s finally clear to me why neither implies each other. Say your true distribution is a standard () Gaussian. Consider these cases for your estimator, also a Gaussian with the following parameters:
Perfectly accurate because it’s unbiased. Imprecise as hell.
Inaccurate. Very precise.
Inaccurate and imprecise.
Perfectly accurate, more precise than the true distribution because its spread is lower.
4 years until 10 minutes ago, when I finally thought about it properly. ↩