My intuition tells me the bonus applies to your base stats.
You can test it by:
1) Save the game
2) Remove all items on wife.
3) Use the cheat code "Levelup 30". Pick defense whenever you can. Your final defense = A.
4) Save the game "X"
5) Use the cheat code "Clearwife". Your no-child defense = B.
6) Load the game "X"
7) Strip naked. Your naked defense = C
Use the cheat code "Clearwife". Your naked defense = D
Now if the 20% bonus applies to naked + equipment defense,
A = B x 1.2
Now if the 20% bonus applies to naked defense only,
C = D x 1.2