Selfish mining was described by Ittay Eyal and Emin Gün Sirer in 2013. A pool that finds a block keeps it secret and mines on top of it privately.
The idea
When honest miners find a competing block, the pool releases its hidden block, causing a race. If the pool stays ahead, it releases its private chain and honest miners' blocks become stale. The pool loses some blocks too, but honest miners lose more, so the pool's share of rewards can exceed its share of hash power.
Key numbers from the paper
With no network advantage, the strategy pays above about one third of total hash power. With a perfect network advantage, it pays at any size. The authors proposed random tie-breaking, which would raise the threshold to one quarter.