Haha, I'm sure there's no difference. It is an interesting question though!
Probably some genious would have to dig through the game's files to find out for sure.
I usually dropped any party members I had in my group for extra villagers, then went back and got my party members back, so all men and women made it to my villages
.