I'm going to FOSDEM, the Free and Open Source Software Developers' European Meeting

I’m actually even presenting!

Just uploaded ooo-build OOo 3.2 rc2 mac builds here.

Just a brief notice, took caff+tls, pulled changes up to latest debian version from svn, and hacked it into working state: script is here, suitable .caffrc is there.

Both power consumption and imminent cease of Debian security updates for the sparc32 platform (I was running a trusty SparcStation 10 still) made me look for a worthy replacement. Some requirements: three or more ethernet ports, very low power consumption, must run Linux (platform with all-open drivers gives extra points), and a RS232/RS485 jack for the solar power inverter read-outs.

I settled for GlobalScale’s openRD-client box, an ARM kirkwood-based design, which typically takes about 6 watts, has ample interface connectors, including two gigabit ethernet ports (yeah, I wanted three. well, turns out that brings you into a completely different price range – so I’ll simply use an USB-based NIC for my 3rd). Hardware arrived some 6 weeks ago, but there was no pressing need yet.

Well, while staying at the GSoC mentor summit, things suddenly went south, the SparcStation blew its scsi controller (and the attached disk’s interface as well), I lost ~3 days worth of email, and I had to hurry things a bit.

Here’s what I did to have stock Debian stable running on the box:

  • I upgraded u-boot to the latest version (it’s kind of the bios on those arm machines, first thing that gets run after a reset), as I needed support for booting off of sd cards:
    • grab latest image from the list at http://code.google.com/p/openrd/downloads/list (http://openrd.googlecode.com/files/openrd_uboot.zip currently)
    • put on toplevel dir of a random usb stick, insert into openrd-client, reset, then at the u-boot prompt:
      usbstart
      fatload usb 0:1 0x0800000 uboot.bin / ext2load usb 0:1 0x0800000 uboot.bin
      nand erase 0x0 0xa0000
      nand write 0x0800000 0x0 0xa0000
      

      If that turns out to be succesful, issue a reset.
      After that, you should see this:

       ** MARVELL BOARD: OpenRD-Client LE 
      
      U-Boot 1.1.4 (Oct 27 2009 - 21:57:24) Marvell version: 3.4.19
      
    • Ready to actually install now. I did it as outlined here
    • Most important difference: we’re not setting up a Sheevaplug (which is also quite nice), so in the above howto, issue a
      setenv arcNumber 2361
      

      instead. Memory setup, I/O register etc. are all hard-coded on arm, so picking the correct machine magic number is paramount for having a working setup (and not accidentally frying your hardware, because some I/O pins are mapped differently. You have been warned).

    • I then decided to build a custom kernel, as the existing lenny image was not supporting all the openrd-client features yet:
      • cloned repo
      • applied this patch on top of it
      • stripped down config (you want to select OpenRD client from the ARM Kirkwood section), make sure mmc drivers are built-in if you want to boot w/o initrd
      • make uImage doesn’t even take that long to build, on the actual machine (1.2GHz, FWIW)
    • You can then flash your kernel image to the internal rom, like this (assuming the image is less than 4MB in size):
      mmcinit
      ext2load mmc 0:1 0x00800000 uImage
      nand erase clean 0x00100000 0x00400000
      nand write 0x00800000 0x00100000 0x00400000
      

      This assumes the kernel image resides on the second partition of your sd card. Since u-boot’s mmc support is still a bit flaky, and sometimes fails to properly initialize the sd card after a reset, booting from internal flash comes in extremely handy – boots kernel from flash, kernel then sets up mmc nicely & grabs everything else from there). Which ends up with the following tweaked u-boot boot setup for me:

      setenv bootargs 'console=ttyS0,115200 root=/dev/mmcblk0p2 rw mtdparts=orion_nand:0x400000@0x100000(uImage)'
      setenv bootcmd 'nand read 0x800000 0x100000 0x400000; bootm 0x800000'
      saveenv
      

      (my root fs lives on the 2nd sd card partition)

    Really happy with it, now that it’s working:

Decided to put 11.2 on some spare HD bits of my MacBook last week. Here’s the log of things I did:

  • Grabbed newest refit, install docs here.
  • Wanted to install via the network install image (around 150MB) from a usb thumb drive, it’s such a hassle (and a waste) to go & and burn CDs for that. So I went for latest grub2 to make EFI/rEFIt boot from usb. More docs about grub2 efit booting:
    • usb boot on mac: german blog
    • grub background info
    • my grub2 setup (on a x86_64 system):
      • ./configure --with-platform=efi --enable-efiemu --enable-mm-debug --enable-grub-emu --enable-grub-fstest --enable-grub-emu-usb --prefix=place_you_have_write_permissions --target=i386
      • then make; make install
      • cd into place_you_have_write_permissions
      • cd into lib/i386-grub/i386-efi there
      • run ../../../bin/i386-grub-mkimage -v -d . -o grub.efi part_gpt hfsplus fat ext2 normal sh chain boot fixvideo appleldr loadbios loopback configfile fs_file fshelp halt handler help iso9660 linux ls minicmd probe reboot search video scsi msdospart bitmap blocklist fs_uuid kernel memrw part_msdos parttool ext2 extcmd hexdump minicmd read sfs xnu xnu_uuid 2>&1 | tail
        this should result in something like "grub-mkimage: info: writing 446 bytes of a fixup block starting at 0xa000"
        Note that in theory, you could just use ../../../bin/i386-grub-mkimage -d . -o grub.efi *.mod, only that this was simply hanging during boot for me – and I didn’t have the patience to bin-search those upteen modules to find the incriminating one. Thus the explicit list, which worked for me.
  • Prepare the usb stick:
    • you should be fine with using both ext2 or fat partitions on it
    • copy your openSUSE-11.2-NET-x86_64.iso into the toplevel dir of the stick
    • from place_you_have_write_permissions/lib/i386-grub/i386-efi, copy grub.efi and *.lst to a efi/grub/ dir on the stick (that’s where refit looks for bootable stuff)
    • put this into efi/grub/grub.cfg on the stick:
      menuentry "openSUSE install" {
       fix_video
       fakebios
       root (hd0,1)
       loopback loop /openSUSE-11.2-NET-x86_64.iso
       linux (loop)/boot/x86_64/loader/linux initrd=initrd splash=silent showopts install=file://openSUSE-11.2-NET-x86_64.iso?device=sdb*
       initrd (loop)/boot/x86_64/loader/initrd
      }
      
  • Put stick into macbook, select usb medium in refit boot screen, boot install image. For me, the installer was falling back to manual setup, so grab the next mirror to you from http://mirrors.opensuse.org/list/11.2.html, determine IP address for the host (“host <mirror_name>”), enter that into the field you get when selecting “http method” for the installation image. Path on that server usually is something like /pub/opensuse/distribution/11.2/repo/oss/ .
  • Proceed with normal installation. Easy as pie.
  • More general info for openSUSE on macbook:

Misc stuff I did:

  • most important first thing to do, to prevent 2nd degree burns on knees: echo -n 2400 > /sys/devices/platform/applesmc.768/fan1_min (you want to put that line into your boot.local script or something. and it’s nothing Linux is at fault, this is entirely broken on Apple’s side of things)
  • get headphones jack to actually output music: run alsamixer in terminal, press F6 and select “HDA intel”. Press F5 to show all mixer controls, un-mute “Speaker 1″.
  • for the iSight built-in camera: install isight-firmware-tools, mount your (hopefully still-existent) OSX partition (something like mount -t hfsplus /dev/sda2 /mnt, then extract the firmware via ift-extract -a /mnt/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS/AppleUSBVideoSupport. Reboot, and voila, camera is working.
  • everything else just worked out of the box.

Spent an intense last week in Orvieto, Italy. First two days had the 2nd odf plugfest; glad to see so many enthusiastic people from the odf universe again, or for the first time in person – and of course witnessing big corporation representatives like Doug and Rob sitting on the table, striving for better odf interop.

img_1630

Following that had three days of OpenOffice.org conference, with many interesting talks (including those from my excellent colleagues Petr, Noel, Cedric, Fridrich, Kendy, Rodo and Kohei).
I found the outlined ideas around an "odfkit" (the similarity in name to webkit is not by accident) quite remarkable – if that ends up in more code reuse across the place (and ideally also reusing existing code in their implementation), this is Good ™.

img_1716-up

Also great to meet the ever-energetic Chris Noack again; it was mostly due to him that I attended a few UX-related talks – and had at least the occasional feeling that approaches there were a bit by-the-book, maybe leaving a few peculiarities and potential synergies available in software generally and FLOSS specifically out of the equation. Reportedly, there are also students working on UX topics, so it would be really awesome to see them join the education project – in an attempt to tear down the wall between coders and interaction designers, that at least I perceive is existing in OOo.

Been at the mentor summit for the first time, and it was even exceeding my high expectations. Awe-some. If there’s another event that pulls together so many friendly high-profile FLOSS people, I’d like to know immediately.

Group Photo - everyone including photographers

Many kudos to Google for the event, and warthog9 for the nice group pic!

With the mentor summit coming up, it’s high time to wrap up this year’s Google Summer of Code for ooo-build. We started the term with 6 students, of which one was sadly missing in action right from the beginning (to set the record straight, he returned all funds).

All participants initially had to fight with OOo’s inherent complexity; I can only stress the importance of getting used to build system, installation & debugging peculiarities before the actual coding starts.

Progress until midterm was good to excellent, except for one case, where we had to make the very hard decision whether to continue with a student showing insufficient results – in the end, the agreement was to drop him, pretty much according to all best practices and lessons learned from other organisations. What I find most encouraging, and open-minded, is the fact that he’s now returning to OOo, and even bringing a few other students from his university for a joint project. Welcome back, Jon!

Which leaves us with a total of four completed projects:

  • Cross-building OOo for win32 (Jesus Corrius, the code (basically all crosswin32-prefixed patches))
  • Impress slide layout rework (Dona Hertel, the code)
  • Writer document comparison (Tzvetelina Tzeneva, the code)
  • Writer document navigation buttons (Maja Djordjevic, the code)

For the last two, let me just reference Tzvetelina’s and Maja’s respective blog postings; I couldn’t possibly explain and advertise it better.

The idea for the cross-building stems from the fact that compiling OOo on win32 is dog slow. With OOo consisting of some 90% c++, compilation is highly i/o-intensive, with an access pattern that apparently makes the Linux buffer cache shine, and the Windows one fall flat on its face. Additionally, of course you’d free people from buying and relying on proprietary software. Additionally, having something cross-buildable usually makes a project’s build system cleaner and more orthogonal.

With the Impress slide layout project, the aim was to get rid of the many hard-coded places where slide layout kinds were handled, following the paradigm of having pure data – the general layout – in a data structure, and not implicitely in code. Of course, an extra benefit is that changing or adding layouts now no longer needs re-compilation of OOo. Specifically, Impress now loads an autolayout configuration file on startup, and exposes the necessary data structures via OOo’s component model to the lower-level subsystems, for import and export.

This was my second GSoC term as a mentor, and my first one as an organisation admin. I’m more than happy with the results; our students did really well, and I hope we’ll see continued contribution from them over time. I also wish to thank my fellow mentors for their time and enthusiasm, and of course Google for making all of this possible in the first place!

2009-summer-of-code-logo-final-r3-01

Finally, there’ll be a panel presentation of the GSoC results at the annual OpenOffice.org conference; we’ll hopefully manage to get most of the students and mentors there. Will keep you posted.

The motivations and actual demographic groups the result’s responders recruited themselves from set aside: 53% pro UI-overhaul seems to be pretty low; at least this would make keeping the old UI optionally available a priority for me (and not an afterthought).

As mentioned by several, last week saw Novell’s HackWeek IV. I had two important vectors along which I wanted to advance, namely bringing slideshow a bit closer to using the new drawinglayer primitives, and improving on the built-in testing facilities in ooo-build.

I started with doing an inventory of things to do for the slideshow upgrade, a pleasant surprise was the fact that Armin already provides a factory for retrieving XPrimitive2D for a given Impress XShape – so technically, the slideshow can retrieve all that’s necessary via pure UNO. Looking at the renderer implementation (that outputs XPrimitive2D on the XCanvas render substrate used in slideshow), I found a bunch of loose ends, and got side-tracked fixing those.

The first thing I actually tackled was the way canvas implements gradients (I seem to develop a knack for those recently). The initial interface for those was rather rigid, and to add insult to injury, extremely tied to ODF’s weird gradient definitions. So I changed the XParametricPolyPolygon2DFactory into a generic XMultiServiceFactory instead, that takes the following parameters:

   Gradients - all gradients need to support two construction
   parameters, "Colors" being a sequence of a sequence of doubles
   and "Stops" being a sequence of doubles. Both must
   have the same length, and at least two elements. See
   http://www.w3.org/TR/SVG11/pservers.html#GradientStops for
   the semantics of gradient stops and colors.
   Required gradient services:

   * "LinearGradient" - the gradient varies linearly between
     the given colors. without coordinate system
     transformation, the color interpolation happens in
     increasing x direction, and is constant in y
     direction. Equivalent to svg linear gradient

http://www.w3.org/TR/SVG11/pservers.html#LinearGradients

   * "EllipticalGradient" - this gradient has zeroth color
     index in the middle, and varies linearly between center
     and final color. The services takes an additional
     parameter named "AspectRatio" of double
     (width divided by height), if this aspect ratio is 1, the
     gradient is circular. If it's not 1, the gradient is
     elliptical, with the special twist that the aspect ratio
     is maintained also for the center color: the gradient will
     not collapse into a single point, but become a line of
     center color. If "AspectRatio" is missing, or equal to 1,
     this gradient yields similar results as the svg radial
     gradient

http://www.w3.org/TR/SVG11/pservers.html#RadialGradients

   * "RectangularGradient" - this gradient has zeroth color
     index in the middle, and varies linearly between center
     and final color via rectangular boxes
     around the center point. The services takes an additional
     parameter named "AspectRatio" of double
     (width divided by height), if this aspect ratio is 1, the
     gradient is quadratic. If it's not 1, the gradient is
     rectangular, with the special twist that the aspect ratio
     is maintained also for the center color: the gradient will
     not collapse into a single point, but become a line of
     center color.

Further massaging the code to share the same gradient texture transformation setup across drawinglayer, canvas, and cppcanvas module, this not only finally gave a nicely uniform rendering of gradients (especially the canvas had some glitches before, with one or the other canvas implementation even missing a gradient type), but also much cleaner code (in terms of comprehensibility, and in terms of implicit duplication of functionality). Final patch has 1237 insertions(+), 1016 deletions(-), containing new functionality (see below) plus substantial commentary (amounting, in total, to about 150 lines). Which gets me quite close to one of my unstated goals of getting less code with more features, when doing refactoring. ;-)

Kudos to Armin btw. for abstracting away the legacy gradients pixel-by-pixel into the usual matrix plus texture function concept!

If you’ve read closely the parameter description for the new gradient factory, you’ve noticed that svg gradients are already fully covered by the canvas now (the little twist is to omit the AspectRatio parameter, and instead scale the gradient anisotrophically via the texture transform, to get the normal svg way of isobare gradient rasterization). Possibly one of the next useful additions would be an optional color transformation function, like the sigmoidal gradient that’s often used in MSO docs – with that, we should get fairly close to implementing the superset of all relevant gradient concepts in office land (well, still missing gdiplus-like path gradient, but that’s rather arcane and seldomly used))

‘Nuff said, in the end, I used most of the week on this gradient rework, but it was time well spent. I used Friday to hack up slideshow to use the primitive renderer for XShapes, which actually works, but of course needs much more changes to e.g. properly support attribute animations (like changing fill color, font size etc.). Full patch is here and here. The more relevant achievement of Friday was the addition of ~convenient convwatch support in ooo-build:

On a known-good version, do this (ooconvwatch is here):

   "cd ooo-build/build/install/program; source ooenv; ooconvwatch -c \
    -d path_to_testdocs"

Wait.

When ooconvwatch finally returns, rebuild with your dubious patches, then:

   "cd ooo-build/build/install/program; source ooenv; ooconvwatch \
    -d path_to_testdocs"

Check /tmp/allfiles.txt for list of docs processed, check /tmp/_filename_.prn.diff.jpg for the graphical diffs.

Todo: collect sensible set of test docs. Throwing huge amounts of random docs at this is a waste. I’m pretty sure I can put everything that will possibly break on a handful of Impress pages. Times the number of odf/sxi/ppt/pptx format versions, and one still has a smallish number. Explore the database usage of convwatch – someone willing to share a description of that feature?

Update: in case it’s not painstakingly obvious, the set of test docs of course will not be closed; instead, after each regression not found by convwatch, it needs to be updated to henceforth catch the problem (referring to the “I have a good idea about what will possibly break in Impress”). ;-)