The only really "hard" part about changing familiars, is that you have to make 20 versions of each of them. With enough time, it's not difficult at all. You just have to understand the balancing of the game, and what sort of setting/level of ease/magic/what have you that you're committing to. Improved Invisibility may not seem like a big thing, but when you're playing a Sorcerer , for example, and every spell choice matters, not having to choose one while instead having your familiar do it can 'break things' just a little bit more in your favor, especially if it is not being RP'd , or minimally so.
Again, Changing familiars is not inherently difficult. It's 20 creatures per, + edits to the 2da as necessary. Then of course testing.