most annoying creature to kill

[Zechnophobe]

Cast Plague on Black Dragons, it pulls them down a peg or two pretty nicely. Especially effective if you are undead.

In the early game, assassin's are annoying to fight. Chance to poison, good damage potentially at range. A great tactic is to move someone out, then put a trap behind them, and wait with the rest of your units. The Assassin's hop out to backstab, hit the trap, don't backstab, and don't return to where they started.

Giants are annoying but not very effective. Use air based units.