Baron Norge & Gostrim Ghost quests, is there a way to do both?

[Metroplex]

I tried searching if this topic was already posted, but couldn't find it.
My problem is that I first take the Ghost quest and then go to the castle, the baron then offers to give me money to kill the ghost and I accept, I go back to the ghost, tell him the baron wants him dead, he then offers me an extra 1000 gold coins to kill the baron, after I go back to the castle I have no choice but to kill the baron but this makes the 'kill the ghost quest' fail.
So has anyone done both? Is that even possible?
Or it is so that you can only do one quest?