From the translated fan
manual:
Elven Crown:
Grows: +1 after each fight.
Falls: -8 in army of the enemy were elves. -4 in army of the hero was undead.
Silver Horn:
Grows: +5 in army of the enemy were dwarves.
Falls: -10 in army of the enemy were elves.
Do you have undead in your army? That would explain the -3 (-4+1), at least for the crown.