I do not know if its intended, but drinking a lesser restoration potion while exhausted will remove your AB penalties until the exhaustion ticks back down to a different level from when you drank the lesser restoration potion.
Example: Being very exhausted gives a massive AB penalty. You can drink a lesser restoration potion to remove this penalty and it'll remain until you go from very exhausted to just exhausted.