On the Guest (assuming Windows) "Explore" your computer (right-click Start Menu), then click the + next to My Network Places/Network, then the + next to Entire Network, then you'll see a VirtualBox option - click the + next to that.
If at any time you click on the item itself, not the +, it seems to mess up, and you have to close and start again.
That's one bit of VirtualBox that isn't as good as VirtualPC - where you can simply drag/drop either way!