I entered the Wizard's Tower and did my first fight against the Caretaker, then - since the second was too hard - I left to continue my game.
To my great surprise and annoyance, I found that my Assassin's accumulated bonuses with Find Weakness has gone, reduced back to 0. I guess since when the Caretaker takes away your troops, this counts as disbanding the troops for the game, and thus the Find Weakness you have accumulated is gone for good.
