Am I getting too far ahead of myself?

[KingImp]

So, I ventured into Verlon Forest with my Level 4 Paladin. I came across this Deserted Castle which was being guarded by some guy named Mark. He told me I had to retrieve something from the castle and bring it to him. So, I went in and was attacked by 3 stacks of Skeletons/Skelton Archers each containing 400-700 units.

How the hell am I supposed to beat this with my measly army so early in the game? I had like 20 Bowmen, 20 Priests, 30 Swordsmen, 4 Archmagi and 100 Peasants. I can't get anymore that that.