Installing Solaris IOU in SPARC server

por | 29 enero, 2013

NetBSD supports Solaris/SVR4 binary emulation. This means that NetBSD/SPARC is able to execute Solaris/SPARC binaries – IOU, for example.

http://www.netbsd.org/docs/compat.html
http://netbsd.gw.com/cgi-bin/man-cgi?compat_solaris+.sparc+NetBSD-5.1

NetBSD/SPARC runs on qemu-system-sparc. Solaris emulation requires some real Solaris libraries. Libraries from any 32-bit Solaris/SPARC should work. Solaris 8 works. Solaris 10 does not work. Here’s a quick tutorial:

1. Compile qemu-system-sparc.
2. Install NetBSD/SPARC in qemu. A minimal installation is enough. Emulate a SPARCstation 20 and use the “nographic” option.

./qemu-system-sparc -hda /path/to/netbsd.img -cdrom /path/to/sparccd-5.1.2.iso -boot d -M SS-20 -m 2048 -nographic

3. Create necessary directorys on the new NetBSD/SPARC system.

mkdir -p /emul/svr4/usr/lib

4. Copy the requred libraries from a Solaris CD/ISO to the emulated machine. All libraries are located under “Solaris_<version>/Product”. The following libraries are required to run IOU and the wrapper:

SUNWcsu
usr/lib/ld.so.1

SUNWcsl
usr/lib/libdl.so.1
usr/lib/libnsl.so.1
usr/lib/libc.so.1
usr/lib/libmp.so.2
usr/lib/libsocket.so.1

Put all libraries in /emul/svr4/usr/lib/ on the emulated machine.

5. On NetBSD, increase the maximum data area size. Otherwise IOU crashes with a “not enough space” message.

ulimit -d unlimited

For some reason, the largest IOU:s (around 90 MB) crashes. The smaller IOU:s (around 40 MB) seem to be working better. Maybe this can be resolved by recompiling the NetBSD kernel whith higher memory limits. I’ll try that later.

Hi Members!
I think the time is now, to share my experiences of this journey … Why do we need NetBSD instead of any other solution … because, ofc, it runs NetBSD :DDDD thoughfully, every other solution is very cpu intesive, fortunately like other linuxs for sparc the NetBSD has cpu powermgmt for cpu idle. I won’t describe the entire process, how to get the “finally” solutions, these took a long time from my free time (more than 1 months).
What do we need?

1. Any linux OS, that runs qemu [I'm using debian 604]
2. Installation CD of NetBSD for example: http://ftp.pgpi.org/pub/NetBSD/iso/5.1.2/sparccd-5.1.2.iso
3. Solaris 8 libraries (NOT 9 or 10, any others will good)
4. IOU :D
5. Kernel recompilation to solve more memory/process issue
6. Setting up Virtualization environment (put the Sol 8 libraries to the /emul/svr4/usr/lib) <-- you have to create it with "mkdir -p /emul/svr4/usr/lib"
7. Add execution permission them, ofc for iou binaries too chmod +x FILENAME
8. Install required NetBSD packages you can find all of them here: http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/README-all.html
9. Create topology, test
10. iou2net.pl/py implementation

Step 1.
Required Qemu,  vtun, Bridge-utils (for TAP interfaces), Python, Perl, ia32-libs (if you’re using 64 bit host OS), uml-utilities, libpcap, libnet-pcap-perl, libvdeplug2, python-libpcap, zlib1g-dev , openssl
Set up those, Create qcow2/qcow file-format virtual disk
I experienced interesting things under qemu … I made some install, and the file format (qcow2) usually not good for newer the (development phase) qemu version 1.0.x, 1.1.x … so if you are using one of the those version use qcow format like this command: qemu-img create -f qcow ./net5.img 20G
1.0 or under I think it is stable, so there I used qcow2 format

For the first boot use this command

qemu-system-sparc -hda ./net5.img -boot d -cdrom ./sparccd-5.1.2.iso -m 2046 -M SS-20 -nographic -net nic -net tap,ifname=tap1,script=no -localtime

Step 2.
http://www.netbsd.org/docs/guide/en/chap-exinst.html
more than enough if you aren’t familiar with installation process. If you don’t have enough memory (at least 2gb) set the SWAP minimum 1GB

Step 3. Collect your Sol 8 libs from /lib (I tar all of them Posted Image)

Step 4. One of the key step is the kernel recompilation … I did a lot of to find the best solution. so if you know there are a lot of unnecessary thing in the GENERIC config file, just uncomment them, if you don’t know what do them mean just leave everything … the REQUIRED modification are these:
Uncomment this line

Hidden Content

apc*                    at sbus? slot ? offset ?

This will solve the cpu utilization

The next one is to modify

Hidden Content

“maxusers 32” line to “maxusers 512”

this shit is calculating the memory utilization based on this value :DD omfg

Finally, put the end of the config file these lines:

Hidden Content

options MAXTSIZ=1073741824
options DFLDSIZ=1073741824
options MAXDSIZ=1610612736
options DFLSSIZ=1073741824
options MAXSSIZ=1073741824

Cool … compile the kernel, see the instructions from the official guide, it is very very good
http://www.netbsd.org/docs/kernel/

Your VM may be won’t survive the reboot, so kill the process and start again xD

Now, you will not see anymore unnecessary cpu utilization on your host OS Posted Image
great.

Do Step5 and 6

Step 7 notes: here are the required utils (output of pkg_info), a few perl module won’t be installed by pkg_add … so in this case you will need manual install, with this command (after gunzip) “perl Makefile.pl && make && make install clean”

bash-4.2nb1              The GNU Bourne Again Shell
bison-2.5nb3            GNU yacc(1) replacement
libpcap-1.1.1      System-independent interface for user-level packet capture
libtool-base-2.2.6bnb5 Generic shared library support script (the script itself)
m4-1.4.16nb1            GNU version of UNIX m4 macro language processor
nano-2.2.4nb1      Small and friendly text editor (a free replacement for Pico)
ncurses-5.9              CRT screen handling and optimization package
netbsd-kmod-tools-5.1 NetBSD kmod tools
openssl-0.9.8w    Secure Socket Layer and cryptographic library
p5-CPAN-Meta-2.120630 Perl module for handling the distribution metadata for a CPAN dist
p5-CPAN-Meta-YAML-0.008 Perl module to read and write a subset of YAML for CPAN Meta files
p5-Carp-Assert-0.20nb3 Executable comments like the ANSI C library assert.h
p5-Carp-Assert-More-1.12nb3 Perl5 module providing convenience wrappers around Carp::Assert
p5-Config-General-2.50 Perl Generic Config module
p5-Danga-Socket-1.61 Event loop and event-driven async socket base class
p5-Exporter-Lite-0.02nb3 Perl 5 module providing lightweight exporting of variables
p5-ExtUtils-CBuilder-0.28.02.02 Perl5 module for compiling and linking C code for Perl modules
p5-ExtUtils-MakeMaker-6.62 Write a Makefile for an extension module from a Makefile.PL
p5-File-Copy-Recursive-0.38nb2 Perl extension for recursively copying files and directories
p5-Getopt-Long-2.38nb2 Extended processing of command line options
p5-IO-1.2500nb2  Perl module for various IO modules
p5-IO-Interface-1.06 IO-Interface - get and set interface characteristics
p5-IO-Socket-INET6-2.67 Perl object interface for AF_INET|AF_INET6 domain sockets
p5-IO-Socket-Multicast-1.12nb1 Perl multicast socket IO
p5-IO-String-1.08nb3 Perl module for I/O on in-core objects like strings and arrays
p5-IPC-Cmd-0.72  Perl module for finding and running system commands
p5-Locale-libintl-1.20nb3 Perl internationalization library
p5-Module-Load-0.22 Perl5 module to runtime require of both modules and files
p5-Net-Pcap-0.16nb3 Perl interface to pcap(3) LBL packet capture library
p5-Params-Util-1.04 Simple standalone param-checking functions
p5-Parse-CPAN-Meta-1.4402 Perl5 module to parse META.yml and other similar CPAN metadata files
p5-PathTools-3.33nb1 Perl module for portably manipulating file specifications
p5-Scalar-List-Utils-1.23nb2 Perl subroutines that would be nice to have in the perl core
p5-Socket6-0.23nb2  Perl5 module to support getaddrinfo() and getnameinfo()
p5-Sub-Uplevel-0.2200nb2 Apparently run a function in a higher stack frame
p5-Sys-Syscall-0.23 Support lib for perlbal
p5-Test-Exception-0.31 Test exception based code
p5-Time-HiRes-1.9724 Perl5 module for high resolution time, sleep, and alarm
perl-5.14.2nb3    Practical Extraction and Report Language
py24-libpcap-0.6.2nb1 Python wrapper for libpcap
python24-2.4.6nb1   Interpreted, interactive, object-oriented programming language
tcpdump-4.1.1      Network monitoring tool
zlib-1.2.3                General purpose data compression library

To boost up your installation, put everything in 1 dir, and execute this command :

for x in $(ls) ; do pkg_add $x; done

If everything is fine, test images, it will run Posted Image for newer images the -m 240 options needed, by default they needs 256 ram, but the OS doesn’t want to allocate this value :DD not a problem, tested 10 routers in same time.

To create connections between the VM and the real world, we will need 2 different NETMAP file … it was a big s*ck to till I get this solution, only one older image worked with the NETMAP files that contains the @ character … no comment, so here it is some kind of solution:

Hidden Content

We need two different NETMAP file, there are several approach to do this

1. use the iou2net.py -n option and define the netmap file

2. modify inside the iou2net.py this line:

my $netmap_file = "./NETMAP";
to this one for example:
my $netmap_file = "./NETMAP.pl";

3. use iou option to define there the NETMAP file or use the NETIO_NETMAP os variable

The basic NETMAP file for IOU instance like this:

Hidden Content

1:0/0 22:0/0

NOTE!!!! You need define your localhost hostname in /etc/hosts file and DONT use localhost space (127.0.0.0/8), define with real one like this command

echo "192.168.1.100 yourhost" >> /etc/hosts

and for the perl instance:

Hidden Content

1:0/0@netbsd 22:0/0@netbsd

Fortunately, I can confirm, that both (UDP and PCAP) mode are working well, further test will be made, but the basic connectivity are flawless. The author of that script wrote, that TAP mode won’t work, so I didn’t/won’t test it … btw it is absolutely unnecessary, because qemu is using “real” le0 interface inside the BSD.
On host linux os use always sudo (even if you are the root) perl … like this

root@roarr:~/debio# perl -w ./iou2net.pl -i tap1 -p 11
iou2net.pl, Version v0.5, 20-Sep-2011.
Can't create IOU pseudo socket
root@roarr:~/debio# sudo perl -w ./iou2net.pl -i tap1 -p 11
iou2net.pl, Version v0.5, 20-Sep-2011.
Forwarding frames between interface tap1 and IOU instance 2, int 0/0 (MAC: 0E:00:00:00:02:00) -  press ^C to exit

At the other side (BSD)

bash-4.2# perl -w ./iou2net.pl.new -i le0 -p 22
iou2net.pl, Version v0.5, 20-Sep-2011.
Forwarding frames between interface le0 and IOU instance 1, int 0/0 (MAC: 0E:00:00:00:01:00) -  press ^C to exit

everything fine now Posted Image)
That’s all folks … enjoy the power of NetBSD, we don’t need anymore Solaris x Posted Image

NOTE TO QEMU: based on my experiences, the best qemu is the

QEMU PC emulator version 0.12.5 (Debian 0.12.5+dfsg-3squeeze1)

Currently, other newer (1.x) qemu might be failed through poweroff or boot process. not sure, may be just local error, for example i can’t reboot the NetBSD without freeze, with older qemu this isn’t problem, working fine