I've had it happen too and to me it seemed to happen whenever any of his vikings or your units moved from a spot that was "landmarked" to another spot that was also landmarked, I got around that by just placing the sides where he doesn't breath on them and neither spawn the landmark things, and shoot down those silly small viking stacks with rage+spells+shooters+black dragons, damage on him was entirely done by phantomed thorns, love those badass 8leadership with 4-6 damage range, only lake faeries outdamage them ( <3 lvl1 army with +damage items
)