Building *nix from scratch

Cheetah Designs · 4 Feb 2011 at 13:15

Basically, I am getting an O2 Joggler for an NAS/Print Server/Alarm Clock/Video & Music Player/Internet browser.

I thought this would be a good time to learn how to do things from scratch, but have no idea where to start.

Hardware specifications: (personally I think these are wrong but will see when it arrives)

Code:

Based on Intel's Poulsbo/US15W mobile chipset.
CPU: 1.3GHz Intel Z520 Single-Core, Hyper-threaded, Atom CPU
Chipset: Intel US15W (400/533MHz FSB)
Graphics: Intel GMA500 (with hardware acceleration for H.264, MPEG2, MPEG4, VC1, WMV9)

Storage:
512MB RAM (NOT user upgradable - not easily anyway)
1GB internal filesystems (soldered, non removable)
1MB EFI bootloader and nonvolatile parameters (socketed)

Network Features:
Wired Gigabit Ethernet. Realtek RTL8111C
WiFi B/G/N based on the Ralink RT2870 chipset (connected to the internal USB port).

Audio: IDT STAC9202X5 Audio with 2 speakers and a 3.5mm headphone jack

USB (High Speed USB 2.0 Ports):
1 x external.
1 x internal (+2 with soldering).

Power: 5v DC
Screen: 7in Sharp LQ070Y3LG4A 800x480 LCD, 16m colour (24-bit), LED backlit, capacitative touch-screen

Given that there is no real hard drive, the OS is going to have to be run from a USB memory stick so it needs to be as small as possible so thats why I thought I'd build from scratch, that way I'd only install what I need. I was thinking of running without a swap drive aswell so everything is in RAM.

I was hoping to get around 10-12MB/s for transferring files over my 100Mbps network.

Any pointers on where to start?

I assume it works something like this:
You find a base eg. Debian (this has a load of inbuilt drivers right?)
Install a UI like GNOME

....errr...and now Im stuck.

brendan · 4 Feb 2011 at 13:38

Cheetah Designs said:
Basically, I am getting an O2 Joggler for an NAS/Print Server/Alarm Clock/Video & Music Player/Internet browser.

Where are you getting your joggler from and how much? I thought stocks where all gone!

Cheetah Designs said:
Hardware specifications: (personally I think these are wrong but will see when it arrives)

Those are correct, why do you doubt? I've checked ;-)

Cheetah Designs said:
Storage:
512MB RAM (NOT user upgradable - not easily anyway)
1GB internal filesystems (soldered, non removable)
1MB EFI bootloader and nonvolatile parameters (socketed)

I'll add the 1GB of internal flash is actually seperated in 4 parts parts and is accessible from linux, but fairly useless because of this.

Cheetah Designs said:
Network Features:
Wired Gigabit Ethernet. Realtek RTL8111C
WiFi B/G/N based on the Ralink RT2870 chipset (connected to the internal USB port).

Correct, the wired ethernet has no hardware MAC, you have to set it on boot, it's a little weird. Also the chip is not great and will not really sustain 100Mb/s with the little CPU.

Cheetah Designs said:
Audio: IDT STAC9202X5 Audio with 2 speakers and a 3.5mm headphone jack

The 3.5mm audio output is very noisy and i'd really recommend a USB soundcard or use pulseaudio to output the sound to another computer via the native TCP module.

Cheetah Designs said:
Given that there is no real hard drive, the OS is going to have to be run from a USB memory stick so it needs to be as small as possible so thats why I thought I'd build from scratch, that way I'd only install what I need. I was thinking of running without a swap drive aswell so everything is in RAM.

You don't need a swap drive, but you don't need to use something that small. a 4GB drive is more than enough to run ubuntu for example, or better - meego.

Cheetah Designs said:
I was hoping to get around 10-12MB/s for transferring files over my 100Mbps network.

First the ethernet is not great, second you're going to have to connect through USB on a small CPU which is going to limit the bandwidth quite a lot. I haven't checked the throughput on mine but I wouldnt expect too much. I don't think it makes a good NAS.

Cheetah Designs said:
Any pointers on where to start?

The easiest is to grab the the ubuntu 10.10 image that works very well with the joggler. It's only 854MB.

If you want to get into some dirty linux building, there are a bunch of us on #meego running meego on the joggler with the EMGD driver. You can then make images with mic2 using OBS repos and can make very small optimised images with exactly what you want.

The main problem with this device is that the intel poulsbo driver is not really great and closed source. It doesn't get updated very often and often does not support the latest version of Xorg. There are actually two drivers, IEGD and EMGD. Better you read about them yourself, the two links below will explain a little more.

Here are some ideas of stuff I did with my joggler. Remote for XBMC and running XBMC natively.

Essexraptor · 4 Feb 2011 at 16:54

seems to be 7+ available on the popular auction site at the moment

Cheetah Designs · 4 Feb 2011 at 17:25

brendan said:
Where are you getting your joggler from and how much? I thought stocks where all gone!

Bought second hand

Those are correct, why do you doubt? I've checked

Didn't think it had wireless-n...or an gigabit ethernet port - even if it cant use the throughput.

Correct, the wired ethernet has no hardware MAC, you have to set it on boot, it's a little weird. Also the chip is not great and will not really sustain 100Mb/s with the little CPU.

Interesting about the wired ethernet. Also interesting that you think the CPU can't manage sustained 100Mb/s....didn't realise it was that poor of a CPU. Is it a CPU intensive task?

pingwing · 4 Feb 2011 at 17:39

Well the best iv seen in the last 5 years was an install of Slackware at a massive 12meg for a 486 with a 30meg hard drive....

But yeah building up from the base is a sensible option, might take a while if your compiling on the system though... (could be time to learn how to compile your system on another then transplant it over)

brendan · 4 Feb 2011 at 18:06

Cheetah Designs said:
Bought second hand

Didn't think it had wireless-n...or an gigabit ethernet port - even if it cant use the throughput.

Interesting about the wired ethernet. Also interesting that you think the CPU can't manage sustained 100Mb/s....didn't realise it was that poor of a CPU. Is it a CPU intensive task?

I think the wireless is N capable (i dont have an N router though), you're right the ethernet is 100mb/s not 1gb/s. The problem with throughput is that you are using USB for the disk, that takes quite abit of CPU, and the driver/chipset for the ethernet card (not sure who's at fault here) takes a lot of CPU when tasking.

Also due to the lack of USB ports you'll probably have to use a hub, reducing disk throughput even more. Either way i'd love to see your results :-)

The CPU isn't exactly a beast, it's slower than typical netbook CPUs.

BigglesPiP · 4 Feb 2011 at 19:59

Get a Debian USB stick and install, choose no additional components, not even a desktop at this stage.

Boot into the system and connect to the internet (use the ethernet cable).

Login on the terminal as root.

Code:

apt-get update
apt-get upgrade
apt-get install gnome-core gnome-network-admin samba nfs-kernel-server nfs-common portmap

Then reboot and login to your bare minimum desktop, install a browser you like, install a media player you like and get configuring your servers.

brendan · 4 Feb 2011 at 20:24

BigglesPiP said:
Get a Debian USB stick and install, choose no additional components, not even a desktop at this stage.

Boot into the system and connect to the internet (use the ethernet cable).

main problems with doing that is installing the EMGD drivers - you can use the framebuffer driver but you loose all kinds of 2D acceleration, GL/GLES support, and va-api...

Have a look here, it's much easier to use the mic2 tool to make yourself a bootable linux OS and use that.
After this weekend a newer image/howto will be up with a newer EMGD driver that will hopefully work a little nicer.

If you do want to do the debian from scratch, in order to get the network up you'll need something like this :

/sbin/ifconfig eth0 hw ether 00:00:de:ad:be:ef
/sbin/ifconfig eth0 up

Cheetah Designs · 5 Feb 2011 at 14:59

I've been doing some reading about the Intel GMA500 graphics card at it appears there are a lot of issues with the drivers for linux and the 2D/Video acceleration. It appears that the EMGD is the best at the moment.

MeeGo appears to have the best driver at the moment...I think Intel is is actually talking to them.

If I was to build from debian, is there anyway to get the MeeGo driver or are they not compatible?

Cromulent · 5 Feb 2011 at 20:21

Sounds like you want Linux from Scratch.

brendan · 5 Feb 2011 at 21:43

Cheetah Designs said:
MeeGo appears to have the best driver at the moment...I think Intel is is actually talking to them.

If I was to build from debian, is there anyway to get the MeeGo driver or are they not compatible?

I'd hope they where talking to the meego people since Intel is funding the project with nokia ;-)

.

For debian i imagine it can be done, the ubuntu images work so there must be a way of getting EMGD on debian. I have no idea though...

aln · 5 Feb 2011 at 22:50

http://www.linuxfromscratch.org/

Cheetah Designs · 7 Feb 2011 at 12:01

Received the device today and am doing some testing as we speak.

Preliminary results don't seem bad at all.

Transferring data to and from the device doesn't seem to send the CPU over 50% (normally around 30%) usage. This is just with a Ubuntu 10.10 image from here: http://joggler.exotica.org.uk/ubuntu/ and just right clicking on the folder and sharing it....is that doing Samba but with a UI?

Currently I am testing with transferring files from and to the USB stick that has Ubunutu on it, I will need to test with a USB hub too.

Anyone know how I can do some accurate speed tests?

EDIT:

System Monitoring shows about 11.6MiB in/out on Ubuntu....although the USB drive which has an LED indicator is doing some strange things when transferring the files which may indicate they aren't being read/written from the drive maybe? Can't see a massive change in RAM either.

brendan · 7 Feb 2011 at 12:20

Cheetah Designs said:
System Monitoring shows about 11.6MiB in/out on Ubuntu....although the USB drive which has an LED indicator is doing some strange things when transferring the files which may indicate they aren't being read/written from the drive maybe? Can't see a massive change in RAM either.

Thats not bad at all, although I suspect your system is using your ram as an IO buffer, check the throughput doesn't die after a few 100MBs. Also use cat /proc/meminfo to see your memory usage, Look at the "Cached" section.

I don't know about the UI file sharing, I guess it's making a samba share.

For a good tool to check disk writing, use iotop. I think it's in the default ubuntu repositories.

Cheetah Designs · 7 Feb 2011 at 12:32

brendan said:
Thats not bad at all, although I suspect your system is using your ram as an IO buffer, check the throughput doesn't die after a few 100MBs. Also use cat /proc/meminfo to see your memory usage, Look at the "Cached" section.

I don't know about the UI file sharing, I guess it's making a samba share.

For a good tool to check disk writing, use iotop. I think it's in the default ubuntu repositories.

Problem is that I only have a 4GB drive so can't test fully. Will have to get a USB hub to test.

brendan · 7 Feb 2011 at 12:34

Cheetah Designs said:
Problem is that I only have a 4GB drive so can't test fully. Will have to get a USB hub to test.

If you don't care too much for wireless, you can grab the USB port inside :-)

I ran like that for ages before I had a hub i could use all the time.

Cheetah Designs · 7 Feb 2011 at 12:37

How do I get inside it? (No pun intended)

Also - there aren't any differences between USB hubs are there? Or should I be looking for a particular kind?

brendan · 7 Feb 2011 at 13:21

Cheetah Designs said:
How do I get inside it? (No pun intended)

Also - there aren't any differences between USB hubs are there? Or should I be looking for a particular kind?

Try grab a powered hub, so that you have the option to give it extra juice if you have lots of devices trying to suck up the 500ma. Other than that... I have this one, and it works very well.

There is a sticker on the back with the serial number/mac address. Remove that, reveal 4 screws. Then I think there are a few more screws on the case, and then it's a matter of prying the case + LCD apart. It's not too hard to do, and both parts are fairly sturdy. There may be a better explanation with pictures/video somewhere if you are unsure.

Cheetah Designs · 7 Feb 2011 at 14:49

brendan said:
Try grab a powered hub, so that you have the option to give it extra juice if you have lots of devices trying to suck up the 500ma. Other than that... I have this one, and it works very well.

There is a sticker on the back with the serial number/mac address. Remove that, reveal 4 screws. Then I think there are a few more screws on the case, and then it's a matter of prying the case + LCD apart. It's not too hard to do, and both parts are fairly sturdy. There may be a better explanation with pictures/video somewhere if you are unsure.

I'm sure one without power will be fine. The USB hard drive has a power brick with it.

Cheetah Designs · 8 Feb 2011 at 15:55

Right, from my tests everything seems OK speed with me. All my clients are wireless and get half decent speeds.

What I want to do now is get the OS sorted. Here's what I want:

-Samba File Sharing
-CUPS print server
-SSH and remote desktop so I can sort any issues out if i need to.
-Alarm Clock to wake me up. - decent interface for it
-Ability to browse and play music - decent interface for it
-Ability to browser internet - decent interface for it
-Ability to play videos with hardware acceleration? - decent interface for it
-Some sort of way to keep up with sports scores - goal flashes etc....

I know the video acceleration may not actually be possible yet/ever.

Can another recommend me an OS based on what I said or should I try building one from scratch? Any ideas for the interface?

If I were to try building one from say a Debian base (purely because Ubuntu is based on it) is it possible to build everything on my main people and then try it out on the joggler. Ie build the image on my laptop then put it on USB and try and boot?

Thanks.