Hi there!
I've been developing umka for quite some time. It's not finished yet but can be useful already.
What is UMKa?The idea is to make userspace UNIX tools that use as much unchanged
KolibriOS source as possible to test architecture-independent parts of the
kernel in your favorite developer environment. It is a common project for a
set of KolibriOS developer tools which are based on original KolibriOS code
wrapped and hacked as to run in the UNIX programming environment.
I.e. umka is not actually an emulator. It embeds some kernel code of KolibriOS and runs
it natively in the userspace.
The idea is
not new.
The most similar project I'm aware of is
LKL.
What I Can Do with UMKa- Run some parts of KolibriOS kernel code in Linux userspace, without emulators like QEMU or Bochs.
- Debug kernel code conveniently (e.g. with gdb).
- Write and run reproducible automatic tests of kolibri kernel code, including unit tests.
- Collect line and branch coverage of kolibri kernel code transparently, i.e. without additional instrumentation code.
What I Can Not Do with UMKaYou can't run KolibriOS userspace code (programs, libs) and hardware dependent kernel code (e.g. device drivers).
Which Parts of the Kernel Can Be Compiled with UMKa?- Block layer
- Underlying devices are replaced with regular files -- disk images.
- Since media is accessed via uniform callback interface (thanks CleverMouse), block layer code doesn't care if it reads a floppy drive, ATA disk or a file.
- Examples: disk cache, MBR and GPT scanning, adding and deleting disks.
- FS
- All FATs, NTFS. EXTs and XFS are supported.
- ISO9660 (Joliet?) code accesses hardware directly and thus isn't compiled in umka.
- LFB graphics
- Instead of the linear framebuffer UMKa uses a simple memory buffer.
- You can dump the buffer to a PNG image to e.g. compare against the reference image.
- A number of small self-contained functions.
- May be some other code later.
What Is Already Done- A couple of tests for XFS. Can be used to write tests for other file systems.
- An example test for graphics (test/016_#f01_#draw_all.t): pixel, line, rectangle, image, text (including smoothing), button, skin, related internal structures (win_stack, win_pos).
- A tool to collect line and branch coverage on test runs.
How It Works- No
more changes to KolibriOS code are required to use UMKa! Preprocessor magic. - umka.asm includes those kernel files that can run in userspace, adds stdcall wrappers, provides a few stub functions, does a number of fasm preprocessor hacks to compile it all to umka.o.
- Tools are built around interfaces of umka.o.
- Coverage is collected via LBR. Slowdown is about 100x.(
What Tools Are Availableumka_shellA shell that reads commands from stdin or a file, can run interactively.
Example session:
Code:
$ ../umka_shell
/> disk_add ../img/xfs_v4_ftype1_s05k_b2k_n8k.img hd0 -c 0
/hd0/1: xfs
/> disk_add ../img/xfs_short_dir_i8.img hd1 -c 0
/hd1/1: xfs
/> ls70 /hd0/1/sf_empty
status = 6 end_of_file, count = 2
total = 2
----f .
----f ..
/> ls70 /hd0/1/sf
status = 6 end_of_file, count = 5
total = 5
----f .
----f ..
----f d0000000000_
----f d0000000001_x
----f d0000000002_xx
/> disk_del hd0
/> disk_del hd1
^C
umka_fuseFUSE based file system, operates on disk images using KolibriOS kernel FS code.
The difference with umka_shell is that requests to KolibriOS code are not written by hand. Instead, they are translated from FUSE calls when you walk mounted FS using cd, ls, stat and other standard UNIX utilities.
Example session:
Code:
$ mkdir /tmp/m
$ ./umka_fuse /tmp/m kolibri.img
$ ls /tmp/m
hd0
$ ls /tmp/m/hd0/
1
$ ls /tmp/m/hd0/1
3D CALENDAR DEMOS END GAMES ICONS16.PNG KUZKINA.MID MADMOUSE MYKEY RTFREAD SKINCFG TEST VMODE
ALLGAMES COLRDIAL DEVELOP ESKIN GMON ICONS32.PNG LANG.INC MAGNIFY NETWORK RUN @SS TINFO @VOLUME
APM CONFIG.INC DISPTEST EXAMPLE.ASM HACONFIG INDEX.HTM LAUNCHER MEDIA @NOTIFY SCRSHOOT STRUCT.INC TINYPAD ZKEY
APP_PLUS CPU @DOCKY FB2READ HDD_INFO KBD LIB @MENU NOTIFY3.PNG SEARCHAP SYSMON TMPDISK
ASCIIVJU CPUID DOCPACK 'File Managers' HOME.PNG KERNEL.MNT LOADDRV MGB @OPEN SETTINGS SYSPANEL TOOLBAR.PNG
BARSCFG CROPFLAT DRIVERS FONTS @HOTANGLES KF_VIEW LOD MOUSECFG PCIDEV SETUP @TASKBAR TXTREAD
CALC DEFAULT.SKN EASYSHOT FSPEED @ICON KPACK MACROS.INC MOUSEMUL RDSAVE SHELL TERMINAL UNZ
$ ls /tmp/m/hd0/1/LIB
ARCHIVER.OBJ BUF2D.OBJ CONSOLE.OBJ ICONV.OBJ KMENU.OBJ LIBIMG.OBJ LIBIO.OBJ NETCODE.OBJ PIXLIB.OBJ RASTERWORKS.OBJ TINYGL.OBJ
BOX_LIB.OBJ CNV_PNG.OBJ HTTP.OBJ INPUTBOX.OBJ LIBGFX.OBJ LIBINI.OBJ MSGBOX.OBJ NETWORK.OBJ PROC_LIB.OBJ SORT.OBJ
$ fusermount -u /tmp/m
Unlike umka_shell, umka_fuse is more like a PoC at the moment.
How to BuildCode:
$ KOLIBRI=/path/to/kolibrios make
/path/to/kolibrios is where you checked out `svn co
svn://kolibrios.org`.
How to Run TestsFirst, generate disk images.
Code:
$ cd img
$ make
Then run tests.
Code:
$ cd test
$ make -B
../umka_shell < 001_#f70_#f70s1_#xfs_#s05k_ls_all_dir_types_ftype1.t > 001_#f70_#f70s1_#xfs_#s05k_ls_all_dir_types_ftype1.out.log
../umka_shell < 002_#f70_#f70s1_#xfs_#s05k_ls_all_dir_types_ftype0.t > 002_#f70_#f70s1_#xfs_#s05k_ls_all_dir_types_ftype0.out.log
../umka_shell < 003_#f70_#f70s0_#xfs_#s05k_read_without_holes.t > 003_#f70_#f70s0_#xfs_#s05k_read_without_holes.out.log
../umka_shell < 004_#f70_#f70s5_#xfs_stat.t > 004_#f70_#f70s5_#xfs_stat.out.log
../umka_shell < 005_#f30_#xfs_cur_dir.t > 005_#f30_#xfs_cur_dir.out.log
../umka_shell < 006_#f70_#f70s0_#xfs_#s05k_read_with_holes.t > 006_#f70_#f70s0_#xfs_#s05k_read_with_holes.out.log
../umka_shell < 007_#f70_#xfs_#lookup_all.t > 007_#f70_#xfs_#lookup_all.out.log
../umka_shell < 008_#f70_#f70s1_#xfs_#s4k_ls_all_dir_types_ftype0.t > 008_#f70_#f70s1_#xfs_#s4k_ls_all_dir_types_ftype0.out.log
../umka_shell < 009_#f70_#f70s0_#f70s1_#xfs_#s05k_#xattr.t > 009_#f70_#f70s0_#f70s1_#xfs_#s05k_#xattr.out.log
../umka_shell < 010_#f70_#xfs_#lookup_#xattr_all.t > 010_#f70_#xfs_#lookup_#xattr_all.out.log
../umka_shell < 011_#f70_#f70s1_#xfs_#lookup_#unicode_all.t > 011_#f70_#f70s1_#xfs_#lookup_#unicode_all.out.log
../umka_shell < 012_#f70_#f70s1_#xfs_#v5_#s05k_ls_all_dir_types_ftype1.t > 012_#f70_#f70s1_#xfs_#v5_#s05k_ls_all_dir_types_ftype1.out.log
../umka_shell < 013_#f70_#xfs_#v5_#lookup_all.t > 013_#f70_#xfs_#v5_#lookup_all.out.log
../umka_shell < 014_#f70_#f70s0_#xfs_#v5_#s05k_read_with_holes.t > 014_#f70_#f70s0_#xfs_#v5_#s05k_read_with_holes.out.log
../umka_shell < 015_#f70_#f70s0_#xfs_#v5_#s05k_read_without_holes.t > 015_#f70_#f70s0_#xfs_#v5_#s05k_read_without_holes.out.log
../umka_shell < 017_#f70_#f70s0_#xfs_#s05k_read_above_4GiB.t > 017_#f70_#f70s0_#xfs_#s05k_read_above_4GiB.out.log
../umka_shell < 018_#xfs_#coverage_all.t > 018_#xfs_#coverage_all.out.log
../umka_shell < 019_#xfs_deep_btree.t > 019_#xfs_deep_btree.out.log
../umka_shell < 016_#f01_#draw_all.t > 016_#f01_#draw_all.out.log
all tests passed
How to Collect CoverageCode:
$ cd test
$ sudo modprobe msr
$ make -B COVERAGE=pew
...
$ ../covpreproc ../umka.lst 0x34 coverage.* > ../umka.cov
Coverage file looks as follows.
Code:
3716 - 844/0 844 : jz @f
3717 - 0 : test eax, eax
3718 -- 0/0 0 : jnz .dyndisk_cleanup
3719 - 0 : dec esi
3720 : @@:
3721 844 : cmp byte [esi], 0
3722 835/9 844 : jnz @f
line 3716: covered, branch (844 times taken, 'not taken' uncovered)
line 3717: uncovered, not a branch
line 3718: uncovered, branch (both uncovered)
line 3722: covered, branch (835 times taken, 9 not taken)
Troubleshooting- umka_* tools are 32-bit. On 64-bit platforms they need 32-bit libraries. Just in case, here is libfuse3.
- There is an issue with documentation in our project, umka is not an exception. Feel free to ping me.
SourceCode:
$ git clone https://baravy.by/r/umka.git
UMKa runs on Linux, therefore I don't think it should be committed to KolibriOS repository.
I didn't test it under Windows. In theory, it should be possible. Patches are welcome.