Not sure if it can exactly, however my new neighbours smoke weed every other day. At first this wasn't really an issue, I'd only smell it if they did it on the street or with their front door open (Could not smell it in my room). However they have since started to smoke on their back door, or sometimes hanging out the bedroom window next to my window, and unfortunately this means the smell clings to the wall area at the back and seeps in through my windows as they have poor sealing and let in a fair bit of air. This now means I can smell their weed smell every time they've done it and have woken up several times this week as they are sparking up.
So although it may not be travelling through the walls, but could be travelling through poorly sealed windows / doors / vents. Have a word with them, a pleasent one, and try to come to an understanding. Don't do what my neighbour did and say 'It's my property I can smoke it wherever' yet know full well that smoke doesn't just sit where you smoke it and travels through the air.