I did some digging regarding this situation and it seems that there was a bug not that long ago in the console version regarding Set bonuses, here's a video that shows the bug:
This video shows how you get a 100 Vit bonus by wearing two pieces of a set, you logout and login and you get another 100 Vit increase due to the Set bonus, and even if you unequip the items, you still keep the bonuses in your base stats.
According to the console hotfix log from Blizzard, this bug was fixed in September of last year and it mentions that it just fixed the new ocurrences but did not fix the affected characters. All affected characters were set to the right statuses by the 1.0.2 Patch release 10/01/2013, these are the patch notes.
This is the closest thing I found or I could think of that could have affected your Witch Doctor, as it seems consistent enough, but given the timeframe, the only way this could still happen to you is if you hadn't added that patch and kept playing offline (which I've no idea if it's possible in the console version).