The researchers at King's College London were following adopted children who spent time in "hellhole" Romanian orphanages.
They grew up with brains 8.6% smaller than other adoptees.
The researchers said it was the "first and most compelling" evidence of the impact on the developing brain.
The appalling care at the orphanages came to light after the fall of Romania's communist dictator Nicolae Ceausescu in 1989.
"I remember TV pictures of those institutions, they were shocking," Prof Edmund Sonuga-Barke, who now leads the study following those children, told the BBC.
He described the institutions as "hellholes" where children were "chained into their cots, rocking, filthy and emaciated".
The children were physically and psychologically deprived with little social contact, no toys and often ravaged by disease.
The children studied had spent between two weeks and nearly four years in such institutions.
Previous studies on children who were later adopted by loving families in the UK showed they were still experiencing mental health problems in adulthood.
Higher levels of traits including autism, attention deficit hyperactivity disorder (ADHD) and a lack of fear of strangers (disinhibited social engagement disorder) have all been documented.
The latest study, published in Proceedings of the National Academy of Sciences, is the first to scan the brains for answers