Player leadership vs enemy leadership

[Watchmaker]

I recently started new game in Red Sands mod. My idea was to only fight with Asmodeo and pet dragon to level them up quickly while using only few other units for their supporting abilities like Inquisitor rage giving skill or Archmages medal counting magic shield etc... but I noticed that while having only Asmodeo and few stacks of single units I fight much smaller enemy armies then in previous games.. so does leading extremely small armies makes enemy armies smaller as well? Or maybe that is only effect on randomness of each play through? Last time I was fighting with full army in the cellar of snake merchant on the first island I faced like 20-25 units stacks while playing only with Asmodeo and 4 Inquisitors enemy stacks were like 9-12.