Well, what you described is the usual option. What you also can do is to send an invite to all your XBOX Live party members at once, by pressing Back. But well, you'll still have to invite them to your party
. And I'm not sure, but I also think only the XBOX Live party 'leader' can do this.