One conference to bring them all and in the summer bind them: LCA End of Days

My final LCA06 report from the land of Dunedin, where the shadows lie. Although I’m here until Monday, I doubt my Internet will be working for much longer.

Started the morning at the usual time, which gave me an extra hour of bumming around (reading my LPI book…) before the (10am) keynote of the day.

And what a keynote it was. Mark Shuttleworth (The Ubuntu guy for those who’re playing along at home) gave a presentation about how collaboration works in successful projects, how it doesn’t work in unsuccessful projects, how to get projects to cooperate with each other, and how to get localisation from people who aren’t programmers. And he packed it in hard, leaving a fairly long period for questions. There were of course a few fairly long questions asked, so this was all for the best.

Amongst Mark’s major points were that the barriers to entry for things like bug tracking systems and translations are too high. Anything that isn’t debbugs needs you to register to post a bug, and you lose 50-70% of hits at that stage. Translation systems that aren’t Gnome currently require you to email around .po files, requiring the translator to edit them in plain text, and some even muck about with more esoteric tools. Gnome provides a neat program, and apparently so does KDE, to handle all that for you.

Mark’s other major point was that distributed revision control systems will turn the language of development from mailling lists to patches. A question from the audience pointed out that mailling-list review of patches (eg. linux-kernel mailling list, and we do this on FreeRADIUS-devel too) would probably suffer if people were just branching things, working on them, and then pushing them back upstream… It’s an interesting puzzle, but between this and the translations stuff, I’m the closest I’ve ever been to looking at contributing to Ubuntu… Scary.

For the one scheduled session of the day I went to a talk from Matthew Garrett about projects dealing with the loss of maintainers or pvioltal people, and how they can deal with it. I can relate, since I am both a sucker (I was kind-of the FreeRADIUS 1.0.0 Release Manager), and soon after moved, changed jobs, and basically stopped posting to (but kept reading what I could of) the FreeRADIUS-devel mailing list. I’ve been lucky that FreeRADIUS is a good, stable package, and so I don’t think my long periods of inattention have caused the Debian package to suffer.

After the barbecue lunch and ceremonial shaving of the luminaries (including Rusty’s moustache!), I attended one of the best-of session, about The RepRap, an attempt to produce a low-cost von Neumann Universal Constructor. And not only is this cool as heck stuff, it was heartwarning to see a geek from a different field, and know that much as our interests diverge, we are united in geekness.

It was also good to see that there are good jobs for geeks, doing geeky things for the betterment of humanity. Especially when one of those things is building the world’s largest nanobot.

At afternoon tea, I scored some Ubuntu 5.10 i386 CD sets, which I think I’ll take back to work and put on a desk with “Free! Try Linux without risking your data.” sign on it or something.

This reminded me that I haven’t checked WhirlPool in a week and a half, and I haven’t yet picked up Omiyage… Still, I will have tomorrow and Monday morning to do so…

Also sitting around the link, reminded again to try WorldForge and Thousand Parsec… Also to find the mesa 6.4 packages and see if the r300 driver’s been updated.

And a note about Blender that slipped my mind for yesterday… It’s awful without a mouse. In fact, a two-button mouse would be pretty awful, but shift-F11-touchpad is particularly not an easy thing to do with only two hands.

Went to the panel discussion, which was largely focussed on patents, trademarks, and how best to promote linux onto both desktops and SMEs.

Finally, was the conference close, where prizes were awarded for the programs for drawing the raffle, then the raffle was drawn, and during demonstrations of some of the other programs submitted, my ticket was drawn. >_<. Although I dunno if I really wanted one of the shirts, it looked like someone’d scribbled all over it, and it’d need some heavy-duty washing…

The total money raised for the John Lyons Chair was just over AU$48k, which is a sterline effort. We also raised NZ$1800 or so from the raffle for the NZ version of Kid’s Help line, which is an excellent cause. Hack fest winners were announced, much thanking of those who have sacrificed so much to get this going, including Mike Beattie’s second standing ovation (The only presenter I saw get a standing ovation was Van Jacobsen), which was well deserved.

Mike mentioned afterwards that the photoblog prize had been completely forgotten, so hopefully they’ll judge it anyway, and we’ll have something pretty to look at on the CD. There’s also supposed to be a written-blog prize, but I suspect my abuse of the “Excerpts” box may count against me. Unless the judges share my sense of humour. ^_^

I must say I have had an absolutly brilliant time. I’ve been rushed off my feet, exhausted, tired, I sat my second LPI exam tonight without having read the last three chapters of the Exam Cram book (It still only took twenty-two minutes by the wall clock… I hope I passed!) and on top of all that I’ve been coughy and snuffly when I wasn’t writing in pain from my throat. And I’d do the whole thing again at the drop of a hat. Or preferably a spiral.

Of course, like (with which I’ve been involved on the other side a couple of times, but which is only one day, rather than six) these things take so much time and effort to prepare, that managing one every year is quite impressive, even though it’s a new team each year. It’s been suggested that 2008’s LCA location be selected more than twelve months out to give them more time to prepare, but for me, my eyes are firmly fixed on LCA07, Sydney. (Their site’s not up yet as of this writing, but yesterday it wasn’t even in DNS. ^_^)

Unless I’m in Japan, of course. I was actually looking around yesterday afternoon in the link, at all these people, these luminaries of my industry and my passion, and wondering if Japan is the right choice… I mean, there’s CBIT now, and I’d hate to leave that. But it’s not what I want to do… And there’s the other project I may have mentioned, which I will be having a meeting about next week, with luck… And yet, having come this far, how can I not go further? Maybe I should be searching harder for Open-Source jobs in Japan, and combine my interests. Sure, I love teaching, and I love teaching languages… But it’s so scary!

I guess it comes down to Feel the fear and do it anyway. And I suspect if I do get to go on JET, as long as I have an Internet connection, I’ll be able to spend more time on my hobbies than I do now. Or maybe not.

A final thought on LCA06… I hope everyone realised the sense of humour it must have take to aware LCA06 to a New Zealand team… ^_^


Passing a penguin through the eye of a needle: LCA Presentations Day 2

Due to spending most of the day off wireless while I tried to get dscape working, today gets one LCA post… Which is a shame, because it was a spectacular day.

Damian Conway’s 14 lessons from the Perl 6 development process was of course both incisive and entertaining. The Perl6 design process has taken about six years so far, and is in fact almost complete, as has been previously reported.

The first presentation of the morning was a Blender tutorial, scheduled only late on Thursday night. It was interestined, but sadly the r300 driver had a few display issues, and having a more recent version of Blender (2.40) than the presenter (2.37) meant that I ended up with a very large and blocky gingerbread man…

Xen on POWER was next.

It started with an overview of Xen, particularly the problems it faces and the solutions it has chosen.

He then discussed the various current Power chips. Power 970 w/Hypervisor Extensions is the current target.

The Hypervison Extensions are: an IOMMU (Like an MMU for IO), a Hypervisor privilige level above the kernle privilige level, real-mode (MMU-less, used for exceptions) base+limit registers: keeps domains locked in their memory even in real mode while allowing direct-to-domain exception delivery.

The PPC Linux port supports wide hardware types, so many hardware interfaces (eg. interrupt controller) were already function pointers filled in by subarch-specific code. Other areas already supported PHYP (IBM’s Hypervisor) so port was generally easy

Currently can boot Dom0, DomU boots (with caveats). One binary runs PHYP, dom0, domU, direct to hardware thanks to OpenFirmware.

Still to be done: SMP, domU timekeeping, grant tables (Xen shared memory method), virtual IO, domain memory outside realmode area.

The main bottleneck to their work is Xen management tools portability… They rely on a hypervisor/userspace translation that i386 gets for free, but for PowerPC is very expensive without the MMU (eg. in Real mode, where the hypervisor runs…)

Long-range plans are: SMP, 970MP, live migration, non-hypervisor mode 970, cell CPU and the non-hypervisor mode 32-bit PPC (eg. my PowerBook, and my G4 tower. ^_^)

Lunch: Finally had time to write down the crashdump from the dscape modules on my PowerBook… On actual paper. It just fit in the blank column on the schedule sheets floating around. Jelmer sacrified his for my cause. ^_^

Avahi: mDNSSD. Interesting project, but it sounds like it wouldn’t be particularly helpful for wider-area service discovery (the multicast is only sent with a TTL of 1, seems kind of like Bluetooth for a LAN, based on the link-local IP address range). On the other hand, the program itself is very well done, and it’s IPv6-enabled. I’ll have to give this some further thought before I can see how to apply it. (In the enterprise world, DNS SRV and Kerberos/Active Directory are prolly better solutions anyway. This would make a network like Fenner Hall’s quite fun…). On the gripping hand, I saw a bug in one of the files he pulled up. I’m miscredited, but it’s close, and I have lots of witnesses. ^_^

Van Jacobsen: Wow. I guess it’s one of those things that just stares you in the face… The immediate upshot is I have to go whack irqbalance from all my SMP servers. The longer term upshot, if this sort of thing gets pushed successfully into Linux, is deleriously high-speed low-CPU networking… I wonder if Van Jaconbsen can drop the CPU usage of timidity like that?

I implemented a lockless ringbuffer into one of the projects I did for TransACT which has not to my knowledge seen the light of day yet. I’m glad to know that it was a good thing. I wish I’d had more time to look at Van’s code for that. I’ve prolly got a model version in a textbook at home, anyway.

Rasmus VS The Internet: Some very good advice from Rasmus regarding Internet security. It boiled down to “Don’t trust user input” which is really just as true for non-Internet applications… He also look at opcode caching briefly, and I linked to APC for his last presentation.

Rasmus took the opportunity to perform cross-site scripting attacks on sites provided by the audience, to the delight of said audience. Man, I’m alomst tempted to go back into Web Programming… almost

Perl BOF: Paul Fenwick gave a quick blurb about PerlNet, designed to be an AUS/NZ Perl community portal. Definately a project to watch, although it’s under CC-BY-SA-2.0-AU, which is a problem for code. I have to pull up the Debian discussion of this for Paul ASAP, so we can get it changed… For that matter, I need to reread the Artistic License. Sam Vilain gave a quick presentation forging on from Damian’s keynote (I ended up chatting briefly to Sam’s boss from Wellington at the dinner) and then Damian Conway demonstrated some of his favorite cool toys from CPAN.).

Interestingly, Creative Commons have put up plain-language versions of not just the license as used by but also plan-language versions of the GPL and the LGPL. This should make my argument easier to make… I wonder if they have one of the Artistic license…

In fact, a little light reading raises a question… The CC says that the SA term specifically applies to derivative works… I need to check the legalese to determine what applies to exact and direct copies… Not that it’s relative here. Taking anything from the site is a derivative work. I guess I need to talk to Paul further and see exactly what he’s trying to achieve… The important point is that any code on the site needs to be free enough to take into a GPLd or Artistic Licensed program, but both the GPL and Artistic License seem to be bound up in the code… Maybe a three-way disjoint license on the site, CC-BY-SA/GPL/Artistic. So you can make derivative works of the site under either CC-BY-SA (which means you must attribute and license it under CC-BY-SA), GPL (The aggregate must be under the GPL so everything must be GPL or looser) or the Artistic License (Roughly, if you change this apart from public- or the author-source changes, you either distribute it to the general public, only use it privately, or mark clearly what you’ve changed… Erk. Bad bad bad for a website.) And just to make things funner, one of the Perl6 RFC responses was the Artistic License 2.0. Something else I need to look up.

Penguin Dinner: Food was fine. Auction was entertaining. They ended up selling the Ian Ross signed book (link this!) for $10k, Linux Australia matches that with $10k, and USENIX matches _that_ for another $20k. The overall total needed is $500k, so I think we did our part. And the book was won by UNSW students and former students, so will be on display at UNSW next year at LCA07.

Damn, Keitarou’s down. No IRC for me, although I could drop irssi onto my laptop temporarily… Also no email until I get back home on Tuesday.

A late-2005 Perl6 release timeline which is still give-or-take valid Sam’s presentation at the Perl BOF had this in one corner, but HTML betrayed him and made it too small to see.

It is easier for a camel to pass through the eye of a needle than to finish Perl 6 In case you were wondering about the post title; also from Sam’s presentation

Edit: Actually link to previous post, rather than leaving myself a note saying “Link to previous post”…

IPv6 uptake suffers, 802.11g on the rise (and fall, about 2.4 thousand million times per second)

Well, the good news is that I got my wireless working with the latest dscape stack…

Up until now, any dscape stack more recent than the 060102 drop would bug when bcm43xx tried to send a packet. The stacktrace looked something like this:

Vector: 300 pc=e2157440: iee80211_master_start_xmit + 0x6c/0x4bc [80211]
            lr=e2157400: iee80211_master_start_xmit + 0x2c/0x4bc [80211]
msr 9032 sp df853d70 [df853cc0] dar -08 dsisr = 40000000
current=c1a8f240 pid = 1365 comm=udevd
[df853ab4] 00000008
[df853ab4] c01c449c qdisc_reestrt +0x100/0x1f0
[df853ab4] c01b2ddc dev_queue_xmit +0x160/0x2b0
[df853ab4] e27da328 mld_sendpack +0x2d4/0x3ec [ipv6]
[df853ab4] e27daa74 mld_ifc_timer_expire +0x21c/0x278 [ipv6]
[df853ab4] c0033507 run_timer_softirq +0x14c/0x1d0
[df853ab4] c002ea50 __do_softirq +0x68/0xf4
[df853ab4] c002eb24 do_softirq +0x48/0x60
[df853ab4] c00068dc timer_interrupt +0x1e0/0x1f4
[df853ab4] c0004584 ret_from_excet +0x0/0x1c
exception 901 [df853ab4] fedb9f0
[df853ab4] 1002000
[df853ab4] 10004474
[df853ab4] 0fe848ac
[df853ab4] 0fe849f4
[df853ab4] 00000000

Anyway, today I decided to actually track this down. Must be all the PowerPC presentations I keep going to…

First try was that low-hanging fruit, Ipv6. It gets dragged in somewhere during the boot process, despite not being listed in /etc/modules. Luckily, the easy solution is the rename the module, so modprobe can’t find it. ^_^

And lo and behold, the dang thing worked!

So I finished off the dscape-based packages I based on rene’s softmac-based packages and uploaded it to

So LCA report for today will be late, since I’m currently at the Perl BOF and will have to dash off to the Penguin Dinner after.

Mew's new drama launched: Gachi Baka!「ガチバカ!」

Mew in her new role


Yes, I know I used it less than an hour ago regarding Web 2.0. Seriously, it’s true again. Not only did Mew’s new TV series start last Thursday (I reckon they posted it late to her site. I’m sure I visited it last Saturday!), but a fansub of the first episode is already doing the rounds, with episode two not even available as a RAW yet.

「ガチバカ!」‘s official website is of course in Japanese, but have of course launched their own reference page for Gachi Baka!. JDorama’s cast list looks kinda short, compared to both the cast and character lists on the official website.

Mew’s character is named 牛島理恵, and there’s an interview and some character background there, again in Japanese. I’ll add it to my pile of Mew-related stuff that needs translating. ^_^

So to all those naysayers who said Mew wouldn’t do any more acting (this includes Mew herself, as I recall)… Yay Mew!

Edit: How could I forget to include a picture?

Edit: And then the alt tag?? My fangirly joy must have flooded my HTML neurons.

The PowerPC and the PHPassion: LCA Presentations Day 1 Afternoon

Lunch: Spent so much time editing the morning’s post, didn’t get to eat, and then went to two wrong theatres and missed the start of the 32/64 comparison with x86 and Power by Olof Johansson. And they were out of pies so I bought sushi instead…

The 32/64 bit presentation was interesting. I’d seen discussion on Debian-Devel along the lines that PPC64 is slower than PPC, and therefore a pure 64-bit Debian port (ala. AMD64 port) was a poor idea (as opposed to AMD64). Olof presented benchmarks (which supported this) as well as a discussion of the way the both x86 and PPC’s ABI’s differ between 32 and 64 bits. It seems that the PPC64 ABI has actually added more overhead to function calls, and hasn’t taken advantage of an ABI change to improve existing issues, while x86_64 has actually used the ABI change to make more use of CPU registers to improve function calls significantly.

An audience member mentioned that newer toolchains than those tested use a different function-call structure or something and should be more efficient. So maybe PPC64’s problems aren’t insurmountable. Mind you, PPC64 does seem to be mainly targetted at specific applications, rather than overall system support ala x86_64… I wonder how MacOS X on G5 does it? I presume its a 32-bit userland from this…

Following on neatly there’s a PPC presentation on next from Paul Mackerras… I wonder if this issue will be addressed then too?

Either way, it was a quick session, and happily I was _just_ able to follow the stuff that was going on and the following audience discussion… ^_^

And the short session meant I got to eat my sushi before the next presentation

Paul Mackerras (Paulus)’s presentation on Recent Developments in PowerPC

Merging 32-bit ppc and 64-bit ppc64 architectures in the Linux kernel: Lots of duplicate code, ensures both architectures benefit from features, cleanups, bugfixes and drivers etc. So instead, there’s one architecture powerpc which is now the default arch.

This explains why recently building kernel modules on my PowerBook G4 failed, unless I prefixed the make with ARCH=ppc… >_< I noticed rene’s bcm43xx packages did the same thing in debian/rules, so it’s not just me.

Interestingly, Paul said that powermacs are no longer supported in ARCH=ppc as of 10th January. So 2.6.16 I guess…

It’s been a fairly fast project, having been agreed to last July, first hit the tree on 29th August, and by 19th November the PPC64 stuff was completely gone.

During a question from Dave Miller about the PPC handing of OpenFirmware VS Sparc64’s handling I was passed a card to sign for Paul Mackerras’s birthday. I can’t imagine how the people sitting directly in front of him signed it without him seeing…

Further discussion of memory layouts (64kb hardware page support means lots and lots of addressable memory!) followed by a quick round of Happy Birthday rounded out the rather technical session nicely.

I spent the interlude before the PHP5 presentation (ie while everyone else was off getting afternoon tea) finishing the morning’s posting, and jealously guarding my powerpoint.

Rasmus Lerdorfs’s PHP5 talk was quite technical, fast and codery in nature, so I’ll just be noting the cool points I wasn’t previously aware of. The version posted on his site right now is slightly older (marked 2006-01-25) than the one he’s displaying on the screen (marked 2006-01-26).

PHP’s main aspects: Fail Fast, Fail Cheap, Be Lazy. The greatest efficiencies come from solving problems you will never have.

Rasmus said that PHP has the unix philosophy of solving one small problem and solving it well. It’s not a framework, or the complete solution, but it does what it does very well.

AJAX makes this even better because the layout stuff gets moved to the local browser side, meaning that PHP only really needs to serve up the data in a format like JSON or XML, and the client-side UI code is actually run on the client site.

AJAX appears (now that I’ve seen it on the code-side) to simply be javascript in the web browser sending requests to the server with updates and whatnot and dynamically changing the UI without refreshing the entire page. This is good stuff, and I probably would have understood it earlier if I had actually listened to anyone whom I’d talked to about it before now. I wonder if we’ll be talking about this at the Perl BOF tomorrow…

I now also know what MVC means, and why it’s not actually sensible for web applications. But we use AJAX to do something similar, seperating presentation and data (oooh, like XHTML!).

Oooh! A very very useful thing in PHP is the <<< construct. Like the << shell function, and it expands PHP variables that appear inline.

Coming soon from Yahoo!, some kind of useful library thing for AJAX and stuff. ^_^

Hmm. JSON is apparently a Javascript self-extracting object. Neat. And I didn’t even know Javascript had an eval() function…

Man, the CBIT Internet customer manager interface is gonna be sooo cool when I’m done with it. ^_^

PDO is a new PECL extension for PHP5, and part of PHP5.1 which looks a lot like Perl’s DBI. It lets you bind parameters which will also save a whackload of crappy coding on my part… Mind you, I didn’t use the binding stuff in Perl5, but I should be doing so.

Things in PHP5 that make it good for AJAX stuff: XML uses libxml2 and dom now. This gets us XPath and XSL… and SimpleXML! This gives a PHP native object from the XML file….

item as $it)
echo "link}'>{$it->title} {$it->description}

Huzzah. Two instruction RSS aggregator… I can’t imagine why Planet takes so much code… ^_^

And with the demonstration of and unsafe search (I wasn’t quick enough on the URL, and the only thing Google will find is an old version of it, Rasmus’s one had images. Images!) it was obvious that this is the future of the web.


Boy’ve I’ve been looking for a place to use that last one. I can’t find the website I saw it on, but as soon as I saw it, I know I had to write it somewhere. ^_^

And then just to finish off (and whet our appetites for tomorrow’s “Fast and Secure websites” presentation) Rasmus pointed us to APC, a PHP Opcode Cache which Yahoo! is happily running on their servers. One server he pulled up has taken 87 million hits in the past few hours, with 20k cache misses. O_O

And then there’s the XHTML problem which I really need to sit down and unblithe my pages about…

BOFs… VoIP. I visited PowerPC, but on consideration this one’s actually more important to me.

VoIP BOF links for my own reference: MacOS X softphone, Platronics nice USB headset which can be bought from Harris Technology.

Interesting things pulled off Planet LCA2006: Gumstix – Small, small small computers; CryoPID – Freezes processes so they can be migrated across ttys and even machines. CryoPID was actually described at the lightning talks at LCA05, and received many oohs and aahs at the time at the idea of screening processes after they’d been started.

After the BOF, went to do my LPI 101 exam. As mentioned, I’ve been studying from LPIC 1 Exam Cram 2, which the LPI’s Books page describes as …the most up-to-date publication reflecting recent changes in LPI exam objectives.. Mind you, it fails to mention that they’ve since (as of October 1st 2005) merged the DEB and RPM exams… Mind you, I hadn’t checked the errata since I was actually only going along to ask a question about the exam proceedure. Having since checked the errata, there’s six pages of corrections, and there’s still some whoppers missing. The above change to the exams, and the fact that Linux doesn’t have an “Archive” bit…

Either way, it took 20 minutes of the 90 allotted, and I feel really good about it. Still worried about the second exam, will prolly put that off until Saturday and push hard to finish the book first.

And another episode of S.O.S watched, as this site was being slow for some reason… It’s so wrenchingly painful to watch, but I can’t possibly stop until I know what happens…

Either way, an excellent day topped off with way too much Japanese food. ^_^

Oh yes. If I get a chance, I must ask Dave Miller at the presenter’s panel about the IEEE 802.11 stacks…

Niagra and network stacks, TCP and talloc: LCA Presentations Day 1 Morning

Overnight interlude: I spent all evening installing WordPress 2.0, and fixing up a few old posts for XHTML compliance. The new WYSIWYG editor is neat, but will lose chunks of unparsable markup (ie. missed quotes and brackets). New posts will prolly be fine to use it for, but for the moment I’m sticking with writing straight HTML.

The whole AJAX interface thing is cool. I’m looking forward to the PHP5 talk this afternoon.

Of course, once I had that done, I decided to grab a new theme. This one’s pretty cool, although the whole lense thing is a bit weird…

And I’m now appearing on Planet 2006, although because I use the excerpt in all my posts to produce a clarification (or declarification) in Chinese Kung Fu Novel Chapter Synopsis Style, my posts end up being quite short on the site, while long on my page.

This morning’s keynote by David Miller was interesting. He maintains the Linux networking stack, and also is the sole porter of the Sparc64 port. So he actually gave three presentations, an overview of the recent changes in the Linux networking stack, a presentation about the Linux port to the new Sun Niagra CPU line, and a brief talk about how to actually deal with kernel maintainers. Lack of wireless there meant I didn’t get my laptop out, so don’t have much more to say about it.

Well, I’ll talk about the new Sun chip, known as Niagra, UltraSPARC T1 or CoolThreads depending on who’s marketing department you ask. It’s a 8-core CPU, each core actually runs four threads in a round-robin fashion when they are able to be scheduled, and leaving them out when they’re waiting on main memory or the FPU or otherwise. This means that any task which can actually _use_ 32 threads for integer-only code will be able to run fast. Kernel compiles are a prime example (looking forward to the kernels-per-second numbers for comparison to the 128 CPU PowerPC G5 box talked about at LCA05. This would also be very nice for video encoding, I suspect. Mind you, the small Sun Fire T1000 Server (shipping March 2006) lists at US$3495, so I doubt I’ll have an array of these to play with anytime soon…. Imagine a Beowulf cluster of these things. ^_^

Morning tea interlude: Posters have gone up. There’s Thousand Parsec, WorldForge and FAI. I’ve looked at WorldForge and Thousand Parsec before, at LCA05, but if I have time tonight (Ha!) I might see where they’re up to these days. FAI on the other hand I’ve only been vaugely aware of, since I never seem to deploy more than one box at a time… But now it’s in my blog, so I’ll be able to find the link when I do want it.

Congestion Advancements with Ian McDonald. A technically-oriented delve into the new congestion control algorithm module structure for TCP, as touched upon by Dave Miller.

Ian presented both the work done recently to generalise and modularise the congestion control algorithms for TCP in the Linux kernel, which had originally been kind of ad-hoc and wide-raning in their touching. The interface they use is fairly simple (if you know TCP backwards, that is ^_^) and they turn out to be per-socket switchable. This will allow much easier use of different algorithms, which are optimised for various combinations of high and low bandwidth, high and low latency, and timeout vs loss vs congestion vs drop situations.

He then presented his current research project, which is a TCP-like protocol (I think… Or was it a congestion-control algorithm?) called TCP-Nice, which is designed to back off from congestion so that the rest of the network functions as if it wasn’t there, while it uses all the left-over bandwidth… I like this, I’d love to see BitTorrent ported to use it. Then I could give free TCP-Nice traffic, and lower my TCP quotas significantly. ^_^ A vast improvement over my previous Second-Class Traffic plan.

He then presented a futher, already live use of the modularised congestion control code in Linux, DCCP. This is a session-based congestion controlled (like TCP) unreliable (like UDP) protocol, mainly intended for multimedia traffic, where you want as much as possible to get through, to back off (somewhat) under congestion, while not doing retransmits and re-ordering since retransmitting live data is a pain.

It’s in the final call for the RFC, and he’s already gotten it working. It’s in the 2.6.14 Linux kernel, with a NAT fix to come in 2.6.16. However, they still haven’t gotten the perfect congestion control algorithm for multimedia streams… The TCP-like CCID2 isn’t very good, the smoothed and slower-falling version CCID3/TFRC didn’t help much, and the latest attempt, MFRC is currently too agressive, and needs tuning to avoid killing other traffic under congestion conditions. But it’s getting there, and shows a lot of promise.

Netem: A last-five-minutes gem… Introduces loss, delay, reorder and duplicate packets on an intermediate box. Can only currently work on output queues.

Finally for the morning, Rusty Russell presented Talloc. Talloc was touched upon by Tridge in his “non-junk” code tour at LCA05, but he didn’t spend too much time on it, looking mainly instead at tdb and ldb…

Basically, talloc is a hierachial pool allocator, which gives destructors, pools and hierachy to your memory allocation calls. This mean that managing your memory usage in C becomes sensible. It’s mainly been driven by Samba, which in fact produces huge whacks of memory allocation… Rusty showed a graph of it, I’ve no idea where to find it. (There was also a URL to the program to make such graphs, I missed that too. >_<) Anyway, it’s pretty impressive.

Andrew Bartlett pointed out last week that he’s using talloc in Samba 4 to trivially wrap the krb5-allocated blobs coming out of the kerberos libraries. This basically gets him free destructors, solving the nasty lifetime problems kerberos’s allocation and free activities otherwise bring.

nfsim uses talloc to simulate kmalloc, providing simple and easy kernel memory leak detection in the netfilter modules being tested. Also has a very neat graphical live talloc allocation tree display. I think that is really neat!

Now to find myself a project to use talloc on… That’s also what I said last year about tdb, as it happens. I actually have one for the latter…. I want to unbone FreeRADIUS‘s IP Pool module, specifically so I don’t have to kill FreeRADIUS to make changes to the pools. I just didn’t get it done in the last 9 months. Gah.

In the more general programming talk at the beginning of the talloc presentation, Rusty suggested that interfaces should be hard to misuse first, easy to use second. He also suggested the following list of tools as being of great importance:

  • distcc
  • ccache
  • ccontrol – This one’s new to me. In fact, I’m still not clear what it does…
  • Mercurial – Source control tool. I’ve not tried it, but Alan DeKok from FreeRADIUS uses it for his own development, and then breaks up the patches for shoving into CVS for the rest of us…

Interview with a Sambpire: LCA Tutorials Afternoon

Samba4 status presented by Andrew Tridgell, Andrew Bartlett, and Jelmer Vernooij.

Samba 4 Tech Preview 1 has been released. In the tradition of named released, it’s known as “Rigged Demo”. ^_^ It’s been packaged in Debian experimental for months, but you of course knew that…

The list of cool cool things is kinda long, but here’s some very quick notes: Javascript-based scripting API. Finally, actually scriptable remote AD administration. In fact, there’s Javascript embedded in the server-side of the new, AJAJ-enabled SWAT allowing remote web users to remotely administer other machines in the AD domain…

Which of course leads to the fact that Samba4 TP1 can already function as a AD Domain Controller. Tridge gave a live demo of Active Directory Vampire! Point it at a Win2k3 AD DC, and hit “exsanguinate”. Less than a minute later, a windows XP machine which was on the domain with the old DC can login without missing a beat.

Andrew B? was telling me on Friday night that Samba4 is a monolithic server, but is completely non-threaded… When a part needs to wait for a response, it runs the event loop again. If a message comes in for another part of the system, that part gets to run, simply growing the stack of paused subsystems. This follows on sensibly from Tridge’s talk last year about how bad multithreading is, how bad IPC is, and why can’t the various Samba4 components all just get along in one address space…

The Samba4 core team took a brief vacation earlier in the year to look at the new Windows Vista tech-preview, specifically to pull apart the new and completely undocumented SMB2 protocol MS has slipped underneath. Consistent with their drive to make everything a database, it apparently has some kind of support for database-like transactions and rollbacks. The good news is that Samba4 has client-side support for it, and will probably release a full SMB2 product before Microsoft does. Just another way in which Open Source stifles innovation™, I guess.

Also this afternoon, a presentation on L2TPNS by Brenden O’Dea, who presented the same software at LCA05… I don’t remember much about the presentation last year, but I had at the time just started playing with it for IPSec over 802.11, since I didn’t have any WPA-capable hardware and was sick of cables running along the roof at Shane’s place.

Since then, I’ve moved, I’ve got WPA-supported gear, and I’d forgotten who Brenden worked for. I told someone “a small ISP in Mildura” a couple of weeks ago. As it happens, he’s from Optus, and his software is supporting about 170k concurrent ADSL connections. So it’s good good stuff. As it happens, I’m back in the L2TP market just now, for both ADSL and hopefully a VPN solution, so I was glad this particular presentation was very very featureful, and talked frankly about the challenges involved. Also, Optus’s ISP serverside is totally Linux-based, and as other departments hand servers over to them to operate, they get Linuxed too. It’s a wonderful thing… Sounds like a great place to work.

Linux Australia AGM… Nothing to see here, move along people.

Keysigning… Not really entertaining, although I am equal 11th in the MSD, for those registered this year. This puts me out at the end of the unfashionable western spiral of the central cluster.

Well, it was faster than last year. Done in about 45 minutes. ^_^

So, dinner hunting time. WordPress 2.0 upgrade to do tonight. And LPI study. And more S.O.S to watch… And gotta get more sleep than last night. (No last-minute libmysqlclient15 bug hunting this time…)

(Edit: &tm; is not the HTML entity for the tradmark symbol. &trade; is)