Yes I've built holes up with wood before, you don't really want to be putting loads of filler in. It doesn't really matter how rough it looks under the filler, you're never going to see it again.Good idea, though I've got no board! Maybe a little piece of wood would do? Thanks!
(unless you forget and think it's a great spot for a mirror in a couple of years...)