I like it the way it is currently. It still pops up, but it doesn't -instantly- pop up. I took a low level PC with 10 constitution out against bandits, fought and beat about six of them, and was exhausted. Compared to the way it was before, where chances are the first fight I'd of been exhausted halfway through, what we have now seems a better balance.
As for the PCs with endurance running as a background, namely barbarians or the aforementioned Athasian elves, perhaps give them a bonus on their constitution that only applies toward exhaustion. Not sure the exact dynamics of that system, so can't say for sure what, just something reasonable. Remember though that they already cover more ground than an equivalent constitution character thanks to barbarian fast movement and the athasian elves natural sprinting ability.