Taking a day off is all about relaxing, if you feel uncomfortable then you're probably better off working.
I agree with this totally as this is precious time you will never get back. Family first. I too am up at 3:30 am everyday .But there are days where I had to just put aside everything just to spent Quality time with the family. Trust me you will feel much better once you learn to do this.