Grandparents in earthquake-prone Chile teach children to identify load-bearing walls, and the Philippines has developed an internationally respected disaster management system. Do such low-cost, social adaptations increase community resilience to earthquakes, or are poorer countries forever doomed to large death tolls in small earthquakes? We attempt to answer this question by quantifying the vulnerability of exposed populations to a set of earthquakes recorded in the USGS PAGER system. We first remove the effect of strong shaking by statistically modelling published mortality, shaking intensity and population exposure data; unexplained variance from this purely physical model is dominated by, and its systematics therefore illuminate, the contribution of socio-economic factors to increasing earthquake mortality. We find that this variance partitions countries in terms of basic socio-economic measures and allows the definition of an Earthquake Vulnerability Index, which identifies both anomalously resilient and anomalously vulnerable countries. Unsurprisingly, wealthy countries perform well, while in general poor countries are more vulnerable. However some low-GDP countries rival even the richest in their ability to resist shaking, suggesting that social and political will can increase resilience. Until expensive engineering solutions become more universally available, the objective targeting of resources at relatively low-cost interventions might help reverse the trend of increasing mortality in earthquakes.