I would actually recommend something like an Arduino. It's very simple, doesn't need a screen, doesn't have an operating system, but if you play with it (most especially of you program in C) it will give you a good base on how a small processor work.
And you can do cool stuff, like blink a LED. Quite frankly, blinking a LED programmatically with one instruction is probably as fun as anything else who is learning to program.
If you want to dig, read a bit the AVR (the processor that is the arduino) documentation, and see about writing a bit of assembler for it. It's the "lower level" under the hood thing...
The funny thing is, it *scales* -- once you understand the assembler of the AVR, you know 80% of the assembler of the x86, powerpc, ARM etc -- they are all similar. Of course there are heaps of complexity added, peripherals etc, but the base is there...
And once you understand /that/ you understand how the "language" you are currently using is munching your code into the lower-level equivalent, and therefore you immediately will know how to write better, more efficient code in that language. And that knowledge wills serve you forever...