So you recast invisibility after you cast confusion and the mobs were not close enough to attack you before you reapplied stealth?
I use this tactic frequently to thin the herd myself and I've never had an issue, but it does require that you recast invisibility quickly, run out of their detection range, and then stealth. If you're too close to the mobs when you reapply stealth, they never really lose you. I could be wrong, but I think that's more a limitation of the engine and knowledge of how to manipulate the stealth mechanics so that this doesn't occur. I'll test it out myself when I get a chance to log in later.