While I'm not that keen on "demi-humans", dropping them outright removes 3/7 of the available classes.
In the campaign I'm going to start Real Soon Now, no really, elves and dwarfs were once subjects of the Vanir Dominion, destroyed in a sudden catastrophe. Scholars speculate that the Vanir were related to elves somehow, and perhaps dwarfs, but no Vanir survived.
The Vanir apparently had a caste system based on physiology and magical talent. All Dwarfs occupied the worker caste, building underground palaces for their masters that none of their masters survived to occupy. Theirs is an egalitarian society, for the most part, with each settlement choosing its own leaders (or rather, managers). Their egalitarianism, unfortunately, does not extend to the sexes; their dropping birth rate has led to most clans forbidding females from heavy labor and venturing into the dangerous outside world.
Elves, on the other hand, retain castes: royals, nobles, envoys, wardens, and commons. (The Elf class represents the envoys, the ones most likely seen outside Elf Realms.) As a whole, the elf aristocracy are stuck in the past, turning ever inward to their own dwindling lands. Wardens, guardians of the scattered and hidden Elf Lands, frequently attack strangers on sight ... even other elves. The "commons" who in times past performed all the unglamorous chores that kept elf society running, become less and less distinguishable from the Wild Fae.
Halflings, possibly distantly related to humans or Dwarfs, mostly stay in their hills. The Halfling Hills do not resemble the Shire so much as Fantasy Fucking Vietnam: traps to incapacitate or kill clumsy big folk, tunnels riddling the placid hills where halflings can move unseen, and diminutive, paranoid hunter-gatherers dressed in rabbit-skins and rough cloth. Only a few halflings have left the hills under unusual circumstances, and they call only a very few Big Folk friends.
This post is longer than I intended, but, for the TL;DR folks, I've put all demi-humans in isolated areas, and attempted to subvert some of the usual tropes. I could make Dwarf Freeholds, Elf Lands, and Halfling Hills disappear entirely without materially affecting the larger world.