Drawing on qualitative research with refugees in and outside formal settlements, this article challenges characterisations of Uganda’s UNHCR-supported refugee settlement system as un-problematically successful. It shows that by denying refugees freedom of movement, the settlement system undermines their socio-economic and other rights. Refugees who remain outside the formal system of refugee registration and settlement are deprived of the refugee status to which they are entitled under international law. The article questions the conventional opposition between refugees living in and out of refugee settlements in the Ugandan context, revealing a more complex and interconnected dynamic than is often assumed.