Arrows don't always kill outright, but they wound. Look at enemy unit statistics after a volley and you'll see its wounded perventage has been increased (number of wounded warriors in the unit). Wounded are much less capable in melee and unit morale becomes lower. Eventually, losses from each volley will become more and more severe (since already wounded men will collapse from even a weak arrow hit that would only wound a healthy one).
If you fire from maximum distance, arrows have weak stopping power. If you have opportunity to fire closer, results will be more satisfying.
In addition, troops with shields can use them to, well, shield themselves from the incoming volley, unless you fire at them from a flank or rear. It opens another tactic - try to make enemy archers to fire at your poorest troops like irregulars from the front (since they have shields) and not at other more valuable units.
Also remember that you have pretty lousy archers at the start
