flipturn.org - stuff. things. other stuff.
dvorak | pantera | hacks

OpenRD-Ultimate

Gentoo on the OpenRD-Ultimate

I've ordered an OpenRD Ultimate ARM box to use as a media server (and hopefully other things like a CUPS server). This is my first attempt at using Gentoo (or linux for that matter) on a non-x86 box.

SPOILER: It's a bit different in the boot procedure, but other than that it's just linux.

Crossdev and distcc install on host amd64

Never having played with cross-compiling before and since the OpenRD is similar to the Sheevaplug, I'm looking at this guide.

  • Architecture: arm
  • Subarchitecture: armv5tel
  • CHOST: armv5tel-softfloat-linux-gnueabi
  • Profile: default/linux/arm/10.0

Earlier, I attempted to compile a complete development environment on my amd64 box. It failed, but I left the record of the attempt.

Hardware

Finally got my hardware. Also got a 640GB disk, partitioned:

/dev/sdl1               1          14      112423+  83  Linux /boot
/dev/sdl2              15        2626    20980890   83  Linux /
/dev/sdl3            2627        5238    20980890   83  Linux /uclibc
/dev/sdl4            5239       77825   583055077+   5  Extended 
/dev/sdl5            5239        5370     1060258+  82  Linux swap / Solaris
/dev/sdl6            5371       77825   581994756   83  Linux /data

Formatted the disk on my desktop before installing:

mkfs.ext2 -c /dev/sdl1
mkfs.ext4 -c /dev/sdl2
mkfs.ext4 -c /dev/sdl3
mkswap /dev/sdl5
mkfs.ext4 -c /dev/sdl6

Booting

Using a mini USB (not micro USB) to USB cable, plugged the mini end into the SYSTEM DEBUG port on the OpenRD, and the regular USB end into the host machine. Got this dmesg on the host after turning on the power to the OpenRD:

USB Serial support registered for FTDI USB Serial Device
usb 4-1: Ignoring serial port reserved for JTAG
ftdi_sio 4-1:1.1: FTDI USB Serial Device converter detected
usb 4-1: Detected FT2232C
usb 4-1: Number of endpoints 2
usb 4-1: Endpoint 1 MaxPacketSize 64
usb 4-1: Endpoint 2 MaxPacketSize 64
usb 4-1: Setting MaxPacketSize 64
usb 4-1: FTDI USB Serial Device converter now attached to ttyUSB0
usbcore: registered new interface driver ftdi_sio
ftdi_sio: v1.5.0:USB FTDI Serial Converters Driver

giving only the device /dev/ttyUSB0 rather than both USB0 and USB1 as it said in the openrd-client_user_guide.pdf. However, this post says not to worry about that and you can still get the serial console on USB0. That works. Tried moserial rather than minicom to get a nice GTK terminal emulator, but it doesn't handle vt100. Used minicom -s to set the params to 115200 8N1 no software or hardware flow control and got the bootup menu:

Fedora release 8 (Werewolf)
Kernel 2.6.22.18 on an armv5tejl

OpenRD login: 

and the root login worked. Reset the box to get the full boot screen. Can subsequently start minicom using minicom -D /dev/ttyUSB0:

Network

I hardcoded the MAC address in my router to return a static IP, so from the Fedora terminal I can get my network working using:

ifconfig eth0 down
dhclient eth0
ping -c 3 www.gentoo.org

where dhclient -r will bring down the interface.

Gentoo Stage 3

Started a typical stage 3 install

mkdir /mnt/gentoo
swapon /dev/sda5
mkfs.ext3 /dev/sda2 # because the fedore doesn't support ext4
mount /dev/sda2 /mnt/gentoo/
mkdir /mnt/gentoo/boot
mount /dev/sda1 /mnt/gentoo/boot
date
cd /mnt/gentoo/
wget http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3/armv5tel-softfloat-linux-gnueabi/stage3-armv5tel-20110117.tar.bz2
wget http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3/armv5tel-softfloat-linux-gnueabi/stage3-armv5tel-20110117.tar.bzS
md5sum -c stage3-armv5tel-20110117.tar.bz2.DIGESTS 
tar xvjpf stage3-*.tar.bz2
cd /mnt/gentoo/etc/
vi make.conf
cp -L /etc/resolv.conf /mnt/gentoo/etc/
mount -t proc none /mnt/gentoo/proc
mount -o bind /dev /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
env-update
source /etc/profile
export PS1="(chroot) $PS1"

I updated my portage tree by rsyncing from my local box.

distcc

In chrooted environment

emerge distcc
/usr/bin/distcc-config --set-hosts "192.168.0.9"
cd /usr/lib/distcc/
rm c++ g++ gcc cc
cat >armv5tel-softfloat-linux-gnueabi-wrapper <<EOF
#!/bin/bash
exec /usr/lib/distcc/bin/armv5tel-softfloat-linux-gnueabi-g${0:$[-2]} "$@"
EOF
chmod 755 armv5tel-softfloat-linux-gnueabi-wrapper 
ln -s armv5tel-softfloat-linux-gnueabi-wrapper cc
ln -s armv5tel-softfloat-linux-gnueabi-wrapper gcc
ln -s armv5tel-softfloat-linux-gnueabi-wrapper c++
ln -s armv5tel-softfloat-linux-gnueabi-wrapper g++

On amd64 host

From the Gentoo embedded guide, it says to use the latest unstable crossdev:

ACCEPT_KEYWORDS=~amd64 emerge -v crossdev

and in my make.conf, set:

PORTDIR_OVERLAY=/usr/local/portage

which places the cross-development portage directories in /usr/local/portage/cross-armv5tel-softfloat-linux-gnueabi/. I have /var/tmp/portage mounted as a ramdisk, so I moved /var/tmp/cross inside the ramdisk:

mkdir -p /var/tmp/portage/cross
ln -s portage/cross /var/tmp/cross

crossdev always puts the target binaries in /usr/armv5tel-softfloat-linux-gnueabi. Further notes on the web seemed to indicate that attempting to install anywhere other than /usr/armv5tel-softfloat-linux-gnueabi would fail with hard-to-track-down errors at any rate. Compiled crossdev to match the utilities in the arm stage3 that I downloaded and configured distcc to allow the openrd box:

USE="-*" crossdev -v --binutils 2.20.1-r1 --gcc 4.4.4-r2 --libc 2.11.2-r3 --kernel 2.6.30-r1 armv5tel-softfloat-linux-gnueabi
emerge -v distcc
/usr/bin/distcc-config --set-hosts "192.168.0.15"

Back on OpenRD Ultimate

nano /etc/make.conf # to add FEATURES="distcc"
cp /usr/share/zoneinfo/PST8PDT /etc/localtime 
nano -w /etc/locale.gen
cat >/etc/conf.d/net <<EOF
config_eth0=( "dhcp" )
dns_domain_lo="flipturn.org"
EOF
rc-update add net.eth0 default
emerge -v syslog-ng vixie-cron logrotate dhcpcd vim sudo portage-utils usbutils
rc-update add syslog-ng default
rc-update add vixie-cron default
rc-update add sshd default
vi /etc/rc.conf
vi /etc/conf.d/clock
vi /etc/fstab
visudo
passwd
useradd -m -g users -G users,wheel,audio,video,usb,portage,cron -u 501 rob
passwd rob

serial console

Note that you have to fix the serial console entry, otherwise the minicom session won't be able to log in after booting:

vi /etc/inittab # changed serial console ttyS0 to 115200 baud

Kernel

On my amd64 machine, and mostly following this guide on the OpenRD google group, I cross-compiled a kernel using linux-2.6.36-gentoo-r5. On 2.6.34 and earlier, you need to update the machine name database otherwise the OpenRD Ultimate is not supported:

cat linux-kernel-Marvell-OpenRD-Ultimate-machine-support.patch|patch -p1
# get new machine name database
wget http://www.arm.linux.org.uk/developer/machines/download.php; mv download.php arch/arm/tools/mach-types

But, on 2.6.36, it is supported out of the box. Compiled with:

make ARCH=arm kirkwood_defconfig
make ARCH=arm menuconfig
# edit kernel config here
make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- uImage

which produces some output and the following summary:

Image Name:   Linux-2.6.36-gentoo-r5
Created:      Wed Feb  9 14:37:15 2011
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    2392084 Bytes = 2336.02 kB = 2.28 MB
Load Address: 00008000
Entry Point:  00008000

and the u-boot image is shown to be in the correct binary format using file arch/arm/boot/uImage:

arch/arm/boot/uImage: u-boot/PPCBoot image

There were only a few modules selected, but still had to do a

make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- modules
make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- INSTALL_MOD_PATH=/tftpboot modules_install

(Note that I don't really expect the modules to be loadable over tftp, I'm just putting all the kernel stuff in one spot.) Trying a tftp server. On the amd64 machine:

vi /etc/conf.d/in.tftp
mkdir /tftpboot
cp /usr/src/linux-2.6.36-gentoo-r5/arch/arm/boot/uImage /tftpboot/
chown nobody:nobody /tftpboot
/etc/init.d/in.tftpd start

On the openrd:

Marvell>> setenv mainlineLinux yes
Marvell>> setenv arcNumber 2884
Marvell>> setenv ipaddr 192.168.0.15
Marvell>> setenv serverip 192.168.0.9
Marvell>> tftpboot 0x2000000 uImage
mvEgigaInit: egiga0 init -by ddb mvBoardPhyAddrGet()=0x0 , priv->port =0x0 by ddb
Using egiga0 device
TFTP from server 192.168.0.9; our IP address is 192.168.0.15
Filename 'uImage'.
Load address: 0x2000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #############
done
Bytes transferred = 2392148 (248054 hex)
Marvell>> setenv bootargs console=ttyS0,115200 root=/dev/sda2 rw
Marvell>> bootm 0x2000000
## Booting image at 02000000 ...
   Image Name:   Linux-2.6.36-gentoo-r5
   Created:      2011-02-09  22:37:15 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    2392084 Bytes =  2.3 MB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
OK

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

Error: unrecognized/unsupported machine ID (r1 = 0x0000020f).
                                                             
Available machine support:
                          
ID (hex)        NAME
00000b44        Marvell OpenRD Ultimate Board
                                             
Please check your kernel config and/or bootloader.

Looks like I have to update the uboot or edit arch/arm/tools/mach-types. Updating arch/arm/tools/mach-types seems less scary, so trying that first.

cd /usr/src/linux-2.6.36-gentoo-r5
vi arch/arm/tools/mach-types # swapping the value for 527 and 2884
make
cp arch/arm/boot/uImage /tftpboot/

It booted without setting setenv arcNumber 2884. Gentoo started, but appeared to freeze after Starting local... because I forgot to change the baud rate for ttyS0 to 115200 in /etc/inittab -- once I fixed that it booted correctly.

First boot log here. I don't know what's going on with the error messages saying Unknown HZ value! (57) Assume 100. and later Unknown HZ value! (68) Assume 100.. It seems to work despite those messages.

Booting from SATA

In the chrooted openrd, I copied the kernel to /boot and the modules to /lib/modules, and the following procedure (from the HowtocreateHDD_1.0.pdf document from the openrd Google group) is able to boot from the drive:

ide reset
ext2load ide 0:1 0x200000 /boot/uImage
setenv bootargs console=ttyS0,115200 root=/dev/sda2 rw
bootm 0x200000

Initially I was getting errors like:

** Bad partition 1 **

using various combinations of ide or sata with 0:1 or 0:0, but it turned out that I was forgetting to use the ide reset command.

The boot procedure can be automated. At the Marvell>> prompt in U-Boot, enter:

set bootcmd 'ide reset; ext2load ide 0:1 0x200000 /boot/uImage; setenv bootargs console=ttyS0,115200 root=/dev/sda2 rw; bootm 0x200000'
save
boot

Setting up networked filesystems

Using ssh to copy files to the openrd machine uses up all the processor, which is limiting the transfer speed to about 8MB/s.

emerge -v portmap nfs samba

NFS

NFS failed with Function not implemented when I forgot to compile the kernel with NFS server support.

rc-update add portmap default
rc-update add nfs default
vi /etc/exports
/etc/init.d/portmap start
/etc/init.d/nfs start

NFS transfers peak at about 45MB/s according to iftop, which is a vast improvement over the needlessly encrypted ssh transfers.

Setting up CUPS

Turned on remote administration by changing the cupsd.conf to include

<Location />
  Allow localhost
  Allow 192.168.0.*
  # Allow shared printing and remote administration...
  Order allow,deny
  Allow @LOCAL
</Location>
<Location /admin>
  Encryption Required
  Allow localhost
  Allow 192.168.0.*
  # Allow remote administration...
  Order allow,deny
  Allow @LOCAL
</Location>
<Location /admin/conf>
  AuthType Basic
  Require user @SYSTEM
  Allow localhost
  Allow 192.168.0.*
  # Allow remote access to the configuration files...
  Order allow,deny
  Allow @LOCAL
</Location>

Not enough entropy available to generate SSL keys

The web interface hangs when trying to go to one of the admin pages. /var/log/cups/error_log shows it hanging on Generating SSL server key..., apparently because the entropy limit is so small on the openrd box:

# sysctl  kernel.random.entropy_avail
kernel.random.entropy_avail = 31

A reference said to run find / . in another terminal, but this doesn't seem to create much more entropy and the creation still hangs. I finally was able to fill the entropy pool using an emerge command.

Added HL-2040 printer

Using the CUPS admin interface, I added the HL-2040 printer using the HL-2060/hl1250 PPD file from openprinting.org. The paper size defaults to A4, so had to change it to letter in the setting page.

Fax server

Attempting to turn the OpenRD into my fax server so I can stop wasting paper on all the junk faxes that I get. Have to use

ACCEPT_KEYWORDS="*" emerge -v hylafax

because the arm keyword doesn't exist in the hylafax ebuild.

References:

References not needed with new enough kernel (2.6.36):