Kernel Planet

I am one of those cavemen who reject gnome-terminal's tabs and use screen(1), mostly because of the muscle memory. Today I went to do some work on RHEL 4, from console. After having to remind myself several times that Alt-F1 is to be used here, I gave up and started screen. Might as well throw extra mingetties out from Upstart (well, inittab in RHEL 4 case, but anyway). BTW, copy-paste works without gpm: disable that too... Oh, and scrollback now works after switching back and forth. My word, I should've done it years ago.

October 06, 2008 10:46 PM

So far it produced not bad results. But not good either. I see locking messages and they are in the right order and file content is not damaged, but clients frequently give up on timeout waiting for lock to be granted. Since locking release process requires inode to be unlocked (so it could be found and locked by the thread, which received network packet), this indeed may take too long on slow media and disks, since locking has to wait until data is written, for example wait for writeback completion or page reading, if they were note in the cache yet.

I tested POHMELFS locks in Xen domains, where network speed is limited by 3 MB/s and writing one million (or ten millions, that may be the point) 8-byte entries at different offsets (sequential step of 128 bytes) took more than 50 seconds, so 5 seconds default lock timeout could be not enough.

That's the theory, in practice I need to test different timeouts and actually run on real machines, but here comes another problem. I have three quite fast SMP machines with lots of RAM connected over gigabit ethernet, which can be used whatever I like. But...
The first one was essentially killed by tbench regression testing. They all have long history of problems with disks or SCSI controllers, now it happens again: the first machine boots only with single 2.6.22 Debian kernel, anything else (including vanilla 2.6.22) fails to read data from the software raid partition, although disks are detected correctly.
Another machine is actively used by aforementioned tbench regression testing, it takes quite long time to boot it and run tests, so things are slow enough.
And the last one is used to control IPMI, since it is the only way I have to reboot them, so when I managed to freeze all three, I needed to contact people who needed to contact people, who needed to hard reset machines in datacenter and put them into BIOS, since existing KVM switches are stupid enough not to respond to keyboard when machine died.

So, I'm a bit forced to spread efforts in several different directions, but nevertheless there is a little bit of time for the new things:

$ ./elliptics -c ./elliptics.conf 
2008-10-07 01:03:39.430198 12778 Logging has been started.
2008-10-07 01:03:39.430559 12778 Successfully initialized 'sha1' hash.
2008-10-07 01:03:39.430641 12778 Node id: b551803fd74ff5590ed38f6ce8a10a2e577b2a9e
2008-10-07 01:03:39.431076 12778 Server is now listening at 127.0.0.1:1025.

$ cat elliptics.conf 
#
# This is a simple config file for the elliptics network.
# Note, that spaces are skipped before and after the '=' delimiter.
#

log = /dev/stdout

hash = sha1
id = This is id string

#numeric_id = 1234567890abcdefffffffffffffffffffffffa
 
root = /tmp
addr = 127.0.0.1:1025:2
#addr = ::1:1025:10

That will be an excellent project (maybe even my best one to date :), which will be used in... More details when things are ready.
I like the idea, so maybe it will give a name for my new site, like noelliptics.net. Not yet though. Comments (0)

October 06, 2008 10:00 PM

New DST release contains following changes:

• Keepalive messages to early detect failed nodes, which are sent if there is no traffic between the nodes.
• Listening socket reuses address now, which speeds up stop/start sequence.
• Fixed bug with wrong debug option, which could read uninitialized memory.
• Change module name from dst.ko to nst.ko, since the former is used by dvb card.
• Whitespace cleanup.

As usual patch is available from archive or via GIT tree.
Enjoy!

Asked for inclusion again. Let's make bets on number of comments for the patch :) Comments (0)

October 06, 2008 04:00 PM

Recent Rawhide (F10 Alpha and Beta) has one of the oddest features ever (Gnome Power Manager level of "odd" here). Whenever one runs ssh (including scp) for the first time, a dialog pops up, asking for a "password to unlock keyring".

Entering any kind of password fails: the dialog reappears again. I must click "Deny" to proceed. V.odd.

It's a little scary to realize that someone well-meaning interposed some pretty big chunk of [currently broken] software into something as fundamental as ssh. Who knows what else it does aside from contacting my X.

P.S. This can be disabled, right? Right?

UPDATE: Peter Robinson e-mailed that the missing package is seahorse. Also, there's apparently a stuck ~/.gnome2/keyring/login.keyring. Removing that gets it regenerated and functional, so gnome-keyring-manager can be used.

In case anyone is curious, GNOME hooks ssh by setting SSH_AUTH_SOCK to something like /tmp/keyring-HIZ5We/ssh. The socket is created by gnome-keyring. Apparently, ssh hooking cannot be disabled selectively.

I wrote before that it would be great if GNOME provided a central persistent keyring for applications such as Pidgin (which otherwise stores your Gmail password in plaintext). Unfortunately, they began with hooking OpenSSH, which has its own secure key management already, and left Pidgin out -- where this kind of capability would actually be useful.

October 05, 2008 08:07 PM

I recently discovered that a friend (yes, you, David) was having an enormous birthday party to which tout le monde was invited - except for me. The explanation was simple: he sent the invitation only to his friends on Facebook, because only degenerates and hermits are not on Facebook. (I may qualify as both.)

The thought of joining Facebook and re-re-connecting with all my friends fills me with weariness and despair. My first social network was Orkut (remember those days, geeklings?) and I was already exhausted before its initial fast burn through the programmer community concluded. Once I thought that OpenSocial would solve this problem by making it possible to export and import my connection data between social networks. Now I'm not so sure.

First, each social network is specialized; e.g., my only up-to-date network is LinkedIn, intended for professional connections. I'd never import that into Facebook, even if Facebook supported OpenSocial. (I'd complain about having to then connect to all the people I knew from high school, etc., but that's not a problem since I only went to public high school for one year part-time - and I truly, actually, literally, had zero friends there.) Second, a central appeal of joining a new social network is the ego gratification that comes with each new request for a connection. I posit this as a major cause of the continual rise and fall of new social networks. The challenge for a social network is to keep that ego gratification coming, which requires the ceaseless invention of silly new memes to send to your friends (the true purpose of social network applications).

I always worry that I'm beginning the slow slide into technological senescence - "These damn kids! Web 1.0 was good enough for me, it should be good enough for them!" - but in this case I feel confident that I'm just. Too. Tired. To join Facebook. Besides, apparently I will learn about anything important through the Real Life(tm) social network.

October 05, 2008 04:06 PM

"Cezares Illusion"
That's how my shower cabin and bathroom corner look now. It took me virtually years to make this, but practically a day to install the cabin, and still bathroom is not completed yet. I need to finish tile glueing and water hatch installation.

Also need to complete brick tiles glueing (roughly 7 sqare meters of walls in kitchen), which likely will be done with bathroom glueing.



There is a lot of work, but actually not that much as may look from the first view. I just need a special development mood to start doing it good and fast, which, as a muse, does not come on demand. Comments (0)

October 05, 2008 03:00 PM

I just returned from Paris, and the 2008 Netfilter Workshop. Just like last year it was a blast, and there were lots of interesting things discussed as well as inbibed.

On the first day there was a users day where presentations were made aimed at a more user oriented audience. It seems that just about anyone who was aware could attend and hear the talks. I gave one on multiqueue networking. You can find my slides and other info here.

Tuesday and Wednesday were the main workshop days.

Of greatest interest to me were the descriptions given by Patrick McHardy for his new filtering framework, where all the complexity is in userspace and the kernel just runs filtering scripts and lookup datastructures fed to it by the user tools. In short, I think this stuff is great, and unlike some folks I don't think this will decrease netfilter participation by other developers at all.

And frankly, iptables was absolutely too accessible to contributors. Look at how much stinking poo is in the patch-o-matic, oft called "crap-o-matic".

Patrick's work is a wonderful centralized framework, and in fact the scripting is generic that you can build any tool to create these filtering instructions and subsidiary lookup tables.

We also made some headway with the tproxy stuff. All but one of the core networking patches are in the net-next-2.6 GIT tree. Indeed, this is a feature which has been missing for 5 years :-) I have to hand it to the balabit guys for sticking to it and working so hard for so long to get this merged.

Pablo gave some interesting presentations (3 at once!), and he is exploring some ways to perhaps make use of bloom filters. This is something Patrick has devoted some exploratory brian power to in the past, but it is often hard to find a use case for these inexact matches, although they are very cool.

Jozsef Kadlecsik gave his IPSET state of the union, discussing new features such as support for ip-port-ip hashing and set lists (which are unions of the existing set type).

Yasuyuki Kosakai gave a presentation on the road blocks that exist currently for doing proper connection tracking for MIPV6 nodes. The basic problem is that the persistent addresses (ie. the ones we'd want to use for connection tracking) exist in various locations in the IPV6 packet and extension headers.

Jesper talked about all of the userland scalability improvements he made to the iptables utilities. He also described a set of scripts he wrote to build optimized rule table trees.

Stephen Hemminger discussed some of the user visible interface work that Vyatta has been doing. Essentially these are a set of templated bash shell scripts and descriptor files that present a Cisco IOS like interface to administrators. He also talked about the performance issues surrounding the way in which iptables does packet counters, as well as the global conntrack table lock.

Harald Welte gave a talk about the current state of GPL violation enforcement. Things seem to have been going quite well, but it is becoming more and more important to give Harald more facilities by which to make air-tight arguments that he has enforcement rights to code which has been violated. One way for that to happen is for significant contributors to sign over their rights to him so that he can make enforcements on their behalf.

It seems that this is a very common stall tactic by the defence in such cases, to try and bring up some doubt about the code property ownership situation.

Of course, aside from the workshop itself there were plenty of parties. Even for lunch we had quite nice French cuisine and beverages, and the dinners were even nicer.

Tuesday night even included a full 4 hour boat cruise on the Seine, with tons of champaigne, wine, small bite size delicasies of all types, and lots of sweets.

Overall a wonderful time, the netfilter workshop never disappoints. A big thank you to the official organizers this year, INL.

October 05, 2008 12:30 AM

This seems ridiculously cheap.
A number of times in the last week I've ordered stuff from amazon where the shipping has cost more than the item. (I have amazon prime, so get free shipping, but this doesn't work for 3rd party sellers using Amazon as a storefront).

Curiosity got the better of me, and even with the shipping making it ~$10, it still seemed like a great deal for 4GB, so I ordered one. The Fedora livecd-iso-to-disk script makes it pretty simple to create a bootable usb stick preloaded with Fedora. 4GB is a pretty usable amount of space. (It's that same that I had on my first eeepc, which proved more than adequate). Hopefully it's not something substandard with a low number of write-cycles, or slow speed or the like. I guess I'll know in a week. If this works out though, I think I'll never have to bother carrying a rescue CD with me every time I travel.

October 04, 2008 08:29 PM

Some of you might remember the famous blinkenlights installations of the CCC in Berlin at Alexanderplatz some years back. Basically they used a matrix of windows on a building for a low-resolution display to play pong and display all kinds of animations and text.

After a long break, they're back, even bigger with blinkenlights stereoscope, a massive installation spanning 960 windows of Toronto City Hall. The entire backend technology has been re-implemented based on OpenBeacon , specifically the WMCU and the WDIM units.

October 04, 2008 02:00 AM

Finger exercise

It’s been a busy couple of weeks since I got back from Plumbers. The merge work seems to be going well; we finally got an ack from Nick on the GEM stuff, so we should be able to push that for 2.6.28. Eric put together a rollup tree with everything we’d like to push (this will probably be the busiest release for i915 since it was initially written), including GEM, several bug fixes, the reworked vblank code (which also contains bug fixes), and support for MSIs, which can save us a lot of CPU time in shared IRQ configurations. The xf86-video-intel 2.5 release is slowly coming together as well, with several bugs fixed, including some of the nastier, EXA related ones. The related 3D and kernel releases are also maturing well, so hopefully this quarter’s release will be stable and feature filled. PCI has been pretty busy too; lots of reviews to do, lots of fixes streaming in and a few new features too (still hoping to get the I/O virtualization support in before the merge window opens too). Finally, the top secret soon-to-be-released platform work is going well; I finally managed to get my bits into a state I’m fairly happy about, so I’m looking forward to seeing it out in the wild.

Graphics merging

So the GEM merge, and more generally, some kind of memory manager merge, has been a looong time coming. It looks like the wait will finally be over once the 2.6.28 merge window opens. To recap, having a real memory and execution manager like GEM enables all sorts of good features: reasonable texture-from-pixmap support for our 2D/3D stack, in-kernel mode setting, removal of user level register access from our 2D driver, redirected & composited direct rendering, and probably several more that I’m forgetting. With the kernel bits in place we should be able to merge the kernel mode setting code finally too (it’ll be disabled by default so it should be fairly safe to merge early), and accelerate our pace of integration into the kernel. That means things like improving the panic stuff I demoed at LPC to be even better, and providing some additional features and generally polishing things. Yay!

xf86-video-intel 2.5

We’ve been closing bugs at a fairly good rate recently, but I still have some concerns about our stability on G4x machines. I should be getting mine soon though so I can help with that, and Zhenyu will be back from vacation next week to finish off some of the fixes he’s been working on, so I’m confident we’ll be in good shape by the time we release. Carl, meanwhile, has been doing some good work on EXA, closing out the notorious “EXA slower than XAA” bug at long last. With a recent X server, the driver is actually in pretty good shape, performance-wise. Carl is trying to track down a couple of more issues for the release, so if you’re someone who can easily reproduce EXA problems, please add your $0.02 to the various EXA bugs; I’m sure Carl would appreciate it.

On a related note, we’ve finally been able to release the IGD OpRegion specification, which should allow distributors and developers to more easily hack on the driver to support various platform features, like ambient light sensors and hot keys. And for pre-OpRegion platforms, I’ve been trying to add more support to our VBIOS support code to at least document the various structures and handshaking registers, I hope to be able to push that work out soon, though I doubt I’ll have time to writeup a full PRM like we did for the OpRegion spec.

PCI

The PCI arena has been fairly busy over the last couple of weeks. Several people got involved in the e1000e NVRAM fire drill, including me (since the X server was initially though to be to blame, and it just so happens that I also wrote the code the X server was using to bang on PCI registers). In the course of debugging the problem, we added several safety checks to various bits of code (the PCI sysfs layer, the I/O resource management layer, the e1000e driver itself, etc.) which turn out to be generally good to have regardless. On the patch management front things have been looking good; we’ve merged several cleanups and fixes (including at least one annoying regression) in the past couple of weeks. The PCI slot naming and SR-IOV support patches aren’t merged yet, but they’ve been getting a lot of review so I hope to be able to pull them in soon (and in time for 2.6.28).

Well that’s it for now. Back to doing ‘git pull’ every few minutes to see if the GEM bits have landed yet. :)

October 03, 2008 09:33 PM

LET'S TALK ABOUT AFGHANISTAN MY NICKNAME WILL BE SARAH PALIN

October 03, 2008 02:00 AM

I've completed a small rewrite of the distributed locks in POHMELFS. They can be byte-range, but since Linux VFS locks the whole inode during writing, I decided first to implement simpler apporach, so although clients send byte-range locks, server locks the whole object.

If there is a simultaneous writing to the object, only one writer is allowed at a time. Write locks are grabbed at write time, read locks at read time. Writing is still handled via writeback, so all caching facilities persist. Locks are 'cached', i.e. if inode was locked and no one else tried to update it, no new lock messages are sent between server and client. Lock release message (initiated by another client, who wants to start writing into the same file) forces inode writeback on the current lock owner.

I've started a testing process, so far quite trivial, but I plan to write a simple application, which will simultaneously write into the same file from different clients into different offsets (like first client writes each second byte, second client writes each third byte and so on) and check the result. If everything is ok, I will release a new version this weekend and start implementation of the really cool distributed facilities I plan to have in POHMELFS. It will be first implemented as a library, so that anyone could use it to create a distributed storage without patching a kernel (but with own API though, I do not want to mess with FUSE). Comments (0)

October 02, 2008 05:00 PM

It seems that foss.in this year is undertaking a major change in its focus—away from the traditional general conference and toward a developer-oriented working event.

Atul Chitnis today posted a detailed rationale, which has also been summarized by Sankarshan. It seems that inspiration was drawn from the recent Plumbers Conf, and also the strong desire to foster actual FOSS development in India.

A FUDCon is being held in conjunction with foss.in, which should also help attract developers. It's the closest upcoming FUDCon to me in geographic terms, and I'm working on attending for that at least.

From discussion with some of the folk involved in the wider event, it seems that many fine details are yet to be worked out, and while the emphasis is very much on Indian developers, I'd suggest that international developers who've been considering submitting a proposal this year definitely still do so.

October 01, 2008 03:50 PM

After I found a small fix for tbench regression over loopback, I decided to run some tests with it.

As was expected, turning off TSO/GSO does not fix the whole issue, performance was increased from 366 MB/s upto 381 MB/s, which is still less than 398 MB/s for 2.6.26-slub.

Another interesting issue I found, is SLAB vs SLUB difference. The former is always faster (about 5-7 MB/s difference): 366 vs 361 MB/s for 2.6.27-rc7 and 381 vs 374 MB/s when TSO and GSO are turned off. Pekka Enberg suggested to revert 5595cffc8248e4672c5803547445e85e4053c8fc commit, which could result in this performance degradation, but without this commit SLUB behaves a little bit slower: 372 vs 374 MB/s.

I will try to find out why there is a huge drop between 2.6.23 and 2.6.24 (54 MB/s) next. Comments (0)

October 01, 2008 01:00 PM

Interesting post on the new foss.in. I applaud the organizers for being willing to experiment with innovative ideas and "breaking eggs to make an omlette". Although, I would encourage using a more Indian friendly analogy. How about "you can't make tofu without crushing some soybeans" or "you can't make paneer without chopping some limes"? :-)

October 01, 2008 08:21 AM

At netfilter workshop, Patrick McHardy described an exciting new feature implementation of netfilter firewalling called nftables. This has the promise of reducing 100's of netfilter modules down to a smaller kernel footprint, and allow for optimization of rulesets. Eric Leblond's blog has more information.

October 01, 2008 03:07 AM

I'm currently at the netfilter workshop 2008 in Paris, France.

It's always sort-of a mixed experience for me. Obviously it's great to spend some time with all the great hackers who work on various aspects of netfilter/iptables (and now finally also its successor that is so-far called nftables). On the other hand, it's been about three years since I last actively contributed code to netfilter, which makes me sad. I'm still excited about it and have many ideas that I'd love to implement. But where to find the time for it?

October 01, 2008 02:00 AM

Elon Musk came pretty close to equaling the Russian record of blown-up rockets in a row, established with N-1. But instead, Falcon 1 successfully orbited a 150 kg chunk of aluminum today. I'm sure Robert Bigelow is quite relieved.

The significance of this event lies in the fact that nobody else but Musk and Bigelow is doing anything to establish permanent colonies in space, because it's impossible to sustain human presence with just government money. So, we have to have a business in space, and that is only possible with a reduction to launch costs. And that is what Mr. Musk has vowed to do.

September 29, 2008 03:04 AM

I've been invited to be the keynote speaker of the joint NLUUG autumn conference and Embedded Linux Conference Europe.

It is a great honor to me to be the keynote speaker, and I will certainly use this chance to provide some of my insights into Embedded Linux. I feel confident to have a thorough understanding about the market (and it's many problems) due to

• having a strong, 14 year FOSS community developer background
• knowing how hard it is to do FOSS-only embedded hardware development (for OpenPCD, OpenBeacon, Openmoko, ...) in todays secretive hardware industry
• having seen a wider range of embedded Linux products than most other people by reverse engineering hundreds of devices for gpl-violations.org
• and now even knowing the chip maker perspective, after becoming VIA's Open Source Liaison

So I'm trying to point out the various problems I see in the Embedded Linux world, and how they can be addressed.

If I know you and you're planning to attend the conference: Please drop me an e-mail in advance so we can meet up, chat, have drinks, meet for dinner or the like.

September 29, 2008 02:00 AM

Today, while reading IT mainstream magazine "c't", I stumbled across an article about GSM deployments (and popularity) all over Africa.

One of the interesting things in that article was that one Operator had modified their network in a way to only use four timeslots (out of the eight available timeslots) per frequency in order to extend the range of a single cell to something like 70 kilometers.

For those who are not as familiar with the GSM Um air interface: It uses TDMA (multiple devices each get one timeslot on a given frequency). So let's assume we have eight timeslots on one frequency, all the transmitters (handsets) need to be synchronized with regard to that timeslot. Radio travels at speed of light and not with infinite speed. Therefore, since the handsets can be at a lot of distance to the receiver (base station), they might send in the correct timeslot, but the signal arrives out of the timeslot. GSM uses what's called "timing advance" in order to compensate for that effect. The base station tells the handset how much time earlier than the actual timeslot it needs to transmit to ensure arrival within the timeslot.

Now in that African GSM network in question, it seems like even the maximum timing advance is not sufficient. The frame still arrives late, i.e. in the next timeslot. By allocating only every second timeslot, there cannot be any clash and thus the range of a single cell can be extended. This is actually a very cool idea, I would almost call it a "hack", and it is possible within the GSM spec without requiring any change to existing mobile phones!

I only wonder how much of such cool hacks we would see if GSM base stations were more open and available. If there was a full FOSS stack that many people could use on off-the-shelf hardware, it would lead to a lot more innovative experiments and thus innovation. There would suddenly be more than a handful of GSM experts with access to proprietary technology looking at what kind of good, useful, cool and/or creative things one can do...

September 29, 2008 02:00 AM

When you're using network sockets, there are these fiddling little practical details (usually relating to error handling) that aren't defined outright in the spec or the man page or even in UNIX Network Programming. The Programming UNIX Sockets in C FAQ usually answers my questions.

September 28, 2008 10:26 PM

It brought me back about 5% in Xen domain with 256 MB of ram in 4-clients tbench test:

current: 187 MB/s
patched: 194 MB/s

Patch is rather trivial: it disables TSO and GSO in loopback and generically on devices which are capable of scatter-gather (where it was automatically enabled by e5a4a72d4f8 commit, which I biseced to be guilty). Actually TSO disablement part provided more gain than GSO on SG devices.
Idea behind patches is clear: we create bigger packet, so we should have smaller overhead of its processing, but apparently GSO/TSO packet creation overhead dominates in loopback at least.

My all three (big) test machines died in various (apparently unbootable) bisections, so I tested it in small and very slow Xen domain. Because of that I did not run 2.6.22 kernel, since git operations and compilation take ages on this 'machine'. For example I was only able to perform about dozen or so git checkous/resets/bisections and compilations for the whole day.

I've posted patch to the netdev@, let's see the result.

Forgot to mention, that I wanted to sell this patch for the DST, POHMELFS or netchannels patch review next time I will post them :) Comments (2)

September 28, 2008 10:00 PM

When I began poking at OpenVZ, I thought that it would be neat to add libvirt support for it. The vzctl is fine and everything, but it's arcane and a new thing to learn (I absolutely hate to use --root instead of --private by accident). Today, Dan Berrange announced that he's done exactly that. OK, I lied. He added "Linux Containers", which are not specific to OpenVZ. But it will do.

Since we're on topic, I've been a bit overwhelmed by the sheer size of the diff. Running "git diff v2.6.26 master" in Parallels' 2.6.26 repo produces an 84 thousand line diff. At the rate I'm reading it, it's going to take me ONE MONTH just to complete one pass, to say nothing about understanding what it does. I was tempted to blog funny bits I saw, but it's premature. Most questions I have right now are likely to answer themselves.

UPDATE: Dan commented that libvirt people are hacking on a specific openvz driver too.

September 28, 2008 03:57 AM

It was reported, that starting from 2.6.23 Linux kernel has a continuous network-related regression, which results in more than 20% performance degradation. I checked it, and got interesting results.
It is better one time to see, than 1000 times to hear it.

Yes, we suck!
I decided to try to fix this issues, and started to bisect 2.6.22->2.6.23 and 2.6.26->2.6.27 on two identical machines, which have 4 logical CPUs (HT enabled) and 4 GB of RAM.

Result was quite surprising: second bisection in the 22->23 froze machine in the middle of the compilation, and first bisection in the 26->27 did not boot. Since I ran it remotely, no progress on this til tomorrow. Comments (0)

September 27, 2008 11:00 PM

Finally it looks like there are no killing bugs or noticebly bad features in the distributed storage, yesterday I pushed a change to drop wrong debug, which may resulted in a crash, also couple of comment cleanups are waiting to be pushed, and likely that's it. It will be the last release, if there will be no new feature requests or bugs found.

So, I switched back to the POHMELFS development from DST.
To be really cool in cache coherency collisions, POHMELFS requires new locking/coherency mechanism, which I implement similar to MOESI cache coherency protocol. Which basically means a floating lock for given object, which may be owned by only one client at a time not counting readers, they just receive a message, that theirs data is not valid anymore.

First, I changed userspace management of the inode cache: now there is only single tree of all objects, which were ever opened by any client. When client disconnects or drop inode locally, it is removed from the server's cache also.
Next, there will be a special command to acicure grab/release a lock, which is only being sent by writers. When writer starts its dirty job of damaging shared data, it sends a lock grab message to the server with requested range, which in turn is broadcasted to the other writers, only single writer is allowed to own given area. Then server proceeds with its usual tasks of cooking or waiting for IO. Eventually owner of the lock decides to release it, for example after above message from the server it can flush data to the server and send lock release message or just on its own. So server checks if given area is now free and sends lock comepltion message to the requester. New owner receives the message, mark inode as own and starts writing there. Any subsequent writing, if inode is marked as owned, does not end up with additional lock message.

So far looks doable, but I only completed what is called 'first' above :)
If there will be no major problems with other project, I plan to complete this part quickly and move furward. Comments (0)

September 27, 2008 07:00 PM

Okay, I have not yet used this argument in a discussion, but I guess I
soon will. With growing number of subnotebooks, Linux preloads grow,
and sooner or later they will find out that something is wrong... and blame it on Linux.

Then, the argument "XP work fine on this hardware, so Linux must be
broken" is used. Well, as you can guess, the argument is wrong: just
because your BIOS happens to work with XP does not mean your BIOS is
bug-free, and does not mean that Linux (or FreeBSD, or OpenSolaris, or
any other Windows versions than those you tested) will work
properly. You know, XP is not a BIOS testing tool, and Linux excersises different codepaths than XP].

[What is worse, I have seen cases of clear BIOS bugs -- like BIOS
returning battery current in 65A area due to integer overflow, where
BIOS people claimed that it is too hard to fix the bug ("we'd have to
retest all these Windows versions"), and wanted us to break the Linux
instead :-(...]

September 26, 2008 09:24 PM

Last week the Linux Plumbers Conference was held in Portland as a developers conference for those working close to the boundary between user and kernel space. Many have described it being one of the best conferences they have attended in a while and I have to agree. The talks were interesting and the people running them discussed their current activities rather than a one-way monologue on their activities over the last year or so (for the most part anyway). For myself, I met a number of people to iron out issues that have been bugging me for a while and got a number of small hacking jobs prototyped that have been on my TODO list for too long. After being working on large pages for some time, it was also a chance to get a quick tour of what is active in the lower levels of the Linux world at the moment.

One area of interest to me was the graphics layer which has had a history of hilarity working outside of the kernel tree. This is out of necessity as the people willing to test an X driver are not necessarily the same people willing to test kernels. Hence, the out-of-tree is required to build against a number of different kernel versions and the resulting ifdef trickery would have a hard time living in mainline not to mention keeping the API in sync. During the track, the possibility was raised that some of the interesting developments in the kernel, X and graphics drivers over the next year would the user to update userspace before enabling using certain kernel features. There was pressure to not require this update but the X guys seemed pretty insistent that a fully incremental effort would be a real pain and the end result ultimately worse.

In case a kernel feature requires an updated userspace in the future, I took a look at what was involved in building X these days. If nothing else, my laptop was using the vesa driver which broke when switching to a text console (fonts were the wrong size) and generally performed worse than what the hardware should have been capable of. The distribution drivers for the card were less than satisfactory for a number of reasons that I never got around to ironing out and I knew there was a lot of additional support for the ATI M56GL Mobility card in my machine added over the last year. Plenty of incentive to get this working.

I vaguely recall from years ago that building X from scratch was no fun whatsoever. Others must have had similar experiences as there is a general perception that X development is scary and building it from source is as much fun as a punch in the nose. X development may still be daunting, I haven't tried, but building from source is straight-forward today. Maybe I missed a pile of options and combinations, but getting the basics right involved an evening on the couch watching Flight of the Concords on DVD and poking buttons periodically - hardly a taxing event.

The X server, the modules and supporting infrastructure consists of a large number of git repositories. If you were to download and build them by hand, you'd be there for a few hours and maybe that turns people away. There is a build scripton the wiki, but it is a rushed hack by the looks of things and did not check that build of a module actually succeeded for example. I updated it to have some new smarts and it should be a fire-and-forget effort although you may need to add your graphics driver to its list. I'll update the wiki when I get back to Ireland (on plane at the moment) and have a
chance to test it on my other machines to make sure it works in general but Script for building X how it currently stands for those that are interested.

What did catch me was starting the new X properly. The site is very clear on starting X itself but I must have missed the instructions on how to give mesa the right paths. For the library paths, I added /opt/gfx-test/lib to /etc/ld.so.conf (it could also have been done in .bashrc with greater smarts but I was lazy). I then used a laucher script for X to load the kernel modules and
set LIBGL_DRIVERS_PATH to find the new DRI drivers. Without LIBGL_DRIVERS_PATH, the system DRI libraries get used resulting in some weirdness which I only spotted after setting some debug options. gdm was starting the old X server so I simply disabled it rather than fixing the init script. End result? One very satisfactory X desktop running very smoothly - nice one.

September 26, 2008 05:07 PM

During my testing I managed to beat following interrupts out of the chip:

[41773.200686] ipw2100: Fatal interrupt. Scheduling firmware restart.
[41773.200707] eth1: Fatal error value: 0x500185B8, address: 0x08004501, inta: 0x40000000
[41773.200810] ipw2100 0000:02:04.0: PCI INT A disabled
[41773.203110] ipw2100: IRQ INTA == 0xFFFFFFFF
[41773.224446] ipw2100: IRQ INTA == 0xFFFFFFFF
[41773.245781] ipw2100: IRQ INTA == 0xFFFFFFFF
[41773.249360] ipw2100 0000:02:04.0: enabling device (0000 -> 0002)
[41773.249384] ipw2100 0000:02:04.0: PCI INT A -> Link[C0C8] -> GSI 11 (level, low) -> IRQ 11
[41773.249426] ipw2100 0000:02:04.0: restoring config space at offset 0x1
	(was 0x2900002, writing 0x2900006)

This happens during PCI ipw2100 device disablement in the reset handler, so when interrupt handler sees that, it bails out. It should be generally ok, but I found a different thing: there is a race between interrupt handler (handler itself and related processing tasklet) and reset code. The latter disables interrupts before starting to turn adapter on, but interrupt handler can run right now on given cpu and can schedule the tasklet, so its disablement does not prevent parallel reading and writing of the various registers.
IRQ processing tasklet does register reading and writing under the lock with interrupts turned off, but reset tasklet does not protect initialization path against it, so I wonder, what may happen in this case. Since register reading and writing happens from absolute addresses (I meant there is no need to write address register first), this maybe not a problem, but still race exists and theoretically can harm the system. Similar unguarded accesses exist in ipw2100_wx_event_work() handler, and also there is unguarded status field setting in various places in the driver, which can harm the driver's behaviour too.

So, maybe I decided to blame firmware a little bit early, although found things may be harmless. I will try to figure this out later tomorrow. Comments (0)

September 25, 2008 09:00 PM

The Linux Plumbers Conference happened last week, and it was exactly the conference I've always dreamed of. I'm clearly biased and a shill for the conference, but I'll make one comment and then link to other people's comments on it.

The LPC organizers really, really wanted this to be a working conference where developers could meet and have in-depth discussions about on-going problems and - key point - come up with solutions. I knew we'd succeeded when a V4L developer told me, with misty eyes and a huge grin, "We have... *action items*!" This was the first time many of them had ever met another V4L developer in person, ever, and they quickly worked out some vexing design problems that had been simmering for years. Several other microconference runners also spoke of action items with smiles on their faces. When you have developers excited and thrilled about AIs, you know you've won.

Anyway, a few other words on the subject:

Peter Graner, Ubuntu kernel manager:

http://blog.redvoodoo.org/2008/09/linux-plumbers-conference-recap.html

Lennart Poettering (audio microconference runner):

http://0pointer.de/blog/projects/lpc-summary.html

We got to see an early Linux-based e-paper device with fascinating implications for X:

http://blog.mozilla.com/blassey/2008/09/24/fennec-on-e-paper/

The LWN summary (paid subscribers only until Sep. 30th - a subscription is $2.50/month at cheapest):

http://lwn.net/Articles/299763/rss

And of course, Greg K-H's keynote lived up to all my expectations for controversiality:

http://blogsearch.google.com/blogsearch?hl=en&num=10&c2coff=1&lr=&safe=active&ie=UTF-8&q=linux+plumbers+conference+greg+kroah+hartman&btnG=Search+Blogs

September 25, 2008 06:49 PM

While I was not looking, someone changed the way local mail worked in Fedora. No longer it is possible to execute your MTA's front-end and have it do the job. Instead, it contacts a daemon and passes the mail with SMTP, which seems like madness to me, even if we consider that several MTAs are packaged for Fedora. Why do they do it like that?

This came up in the context of mitsuki, which now has a freaking gazillion of sendmail processes across its VEs. The genius who came up with it was not content with wasting just one task and stack per system.

September 25, 2008 06:39 PM

You'd think that -oro mounts volume read-only?

You are wrong, at least in ext3 case. ext3 only pretends volume is read-only to the userspace, but it still does journal replay on it.

Results are not nice: (I have broken SD card where read-only switch does not work properly).

Sep 25 11:51:38 amd kernel: mmc0: new SD card at address 0002
Sep 25 11:51:38 amd kernel: mmcblk0: mmc0:0002 SD1GB 975872KiB (ro)
Sep 25 11:51:38 amd kernel:  mmcblk0: p1
Sep 25 11:51:47 amd kernel: JBD: Clearing recovery information on journal
Sep 25 11:51:47 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:47 amd kernel: end_request: I/O error, dev mmcblk0, sector 4017
Sep 25 11:51:47 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 486
Sep 25 11:51:47 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:48 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:48 amd kernel: end_request: I/O error, dev mmcblk0, sector 129
Sep 25 11:51:48 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 0
Sep 25 11:51:48 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:48 amd kernel: end_request: I/O error, dev mmcblk0, sector 137
Sep 25 11:51:48 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 1
Sep 25 11:51:48 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:48 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:48 amd kernel: end_request: I/O error, dev mmcblk0, sector 161
Sep 25 11:51:48 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 4
Sep 25 11:51:48 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:49 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:49 amd kernel: end_request: I/O error, dev mmcblk0, sector 3977
Sep 25 11:51:49 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 481
Sep 25 11:51:49 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:49 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:49 amd kernel: end_request: I/O error, dev mmcblk0, sector 262305
Sep 25 11:51:49 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 32772
Sep 25 11:51:49 amd kernel: lost page write due to I/O error on mmcblk0p1
Sep 25 11:51:50 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:50 amd kernel: end_request: I/O error, dev mmcblk0, sector 524417
Sep 25 11:51:50 amd kernel: Buffer I/O error on device mmcblk0p1, logical block 65536
Sep 25 11:51:50 amd kernel: lost page write due to I/O error on mmcblk0p1
...

Sep 25 11:51:51 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:51 amd kernel: end_request: I/O error, dev mmcblk0, sector 1048705
Sep 25 11:51:51 amd kernel: end_request: I/O error, dev mmcblk0, sector 1048713
Sep 25 11:51:51 amd kernel: end_request: I/O error, dev mmcblk0, sector 1048721
Sep 25 11:51:51 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:51 amd kernel: end_request: I/O error, dev mmcblk0, sector 1052537
Sep 25 11:51:52 amd kernel: mmcblk0: error -110 transferring data
Sep 25 11:51:52 amd kernel: end_request: I/O error, dev mmcblk0, sector 4017
Sep 25 11:51:52 amd kernel: kjournald starting. Commit interval 5 seconds
Sep 25 11:51:52 amd kernel: ------------[ cut here ]------------
Sep 25 11:51:52 amd kernel: WARNING: at /data/l/linux/fs/buffer.c:1186 mark_buffer_dirty+0x57/0x70()
Sep 25 11:51:52 amd kernel: Modules linked in:
Sep 25 11:51:52 amd kernel: Pid: 16292, comm: mount Not tainted 2.6.27-rc6 #379
Sep 25 11:51:52 amd kernel: [] warn_on_slowpath+0x5f/0x90
Sep 25 11:51:52 amd kernel: [] __find_get_block+0x82/0x190
Sep 25 11:51:52 amd kernel: [] __lock_acquire+0x181/0x930
Sep 25 11:51:52 amd kernel: [] __lock_acquire+0x181/0x930
Sep 25 11:51:52 amd kernel: [] journal_update_superblock+0x67/0xd0
Sep 25 11:51:52 amd kernel: [] mark_buffer_dirty+0x57/0x70
Sep 25 11:51:52 amd kernel: [] journal_update_superblock+0x6f/0xd0
Sep 25 11:51:52 amd kernel: [] journal_flush+0xa1/0x120
Sep 25 11:51:52 amd kernel: [] ext3_mark_recovery_complete+0x2c/0x80
Sep 25 11:51:52 amd kernel: [] ext3_fill_super+0x125e/0x1800
Sep 25 11:51:52 amd kernel: [] snprintf+0x1f/0x30
Sep 25 11:51:52 amd kernel: [] disk_name+0xb0/0xc0
Sep 25 11:51:52 amd kernel: [] get_sb_bdev+0x104/0x130
Sep 25 11:51:52 amd kernel: [] kstrdup+0x39/0x70
Sep 25 11:51:52 amd kernel: [] ext3_get_sb+0x20/0x30
Sep 25 11:51:52 amd kernel: [] ext3_fill_super+0x0/0x1800
Sep 25 11:51:52 amd kernel: [] vfs_kern_mount+0x43/0x90
....

September 25, 2008 02:54 PM

I thought there's a time to finally update my OpenEmbedded-based Zaurus C3000 (spitz): bluetooth caused crashes over suspend, sometimes screen got to zero brightness accidentally, SD card was very slow and font size varied over boot.

I did update to latest Angstrom, thinking that all this must be long fixed... well not so. Font size still varies.

I kind of expected mutt to be available for Angstrom; it is not there :-(. Moreover, alt-tab stopped working, and theres ugly nag screen each time you resume ("enter password"). There's "autologin" setting in the preferences, but it does not seem to work and does not make the nag screen go away.

Then I installed altboot so I could boot debian on hda3... and altboot survived for exactly 4 boots. Now it hangs after printing options 1-4 :-(. So I no longer have zaurus machine, I have a brick now.

September 25, 2008 09:59 AM

I was not able to force card not to send or receive packets with ping tests, although definitely was able to generate lots of fatal interrupt with completely different values and addresses.
Frequently card generates fatal interrupt with different values on the same address, like below:

eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x5000CEE4, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x5000CEE4, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x5000CEE4, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000
eth1: Fatal error value: 0x50018584, address: 0x61C00000, inta: 0x40000000

They did not follow one after another though.
Different error values likely mean, that there is no any correlation between values and addresses, so this information is useless.

I added power state changes to the reset function, so now it does something like that:

[  897.661002] ipw2100: Fatal interrupt. Scheduling firmware restart.
[  897.661021] eth1: Fatal error value: 0x30016C44, address: 0x601F7C00, inta: 0x40000000
[  897.664712] ipw2100 0000:02:04.0: PCI INT A disabled
[  897.712041] ipw2100 0000:02:04.0: enabling device (0000 -> 0002)
[  897.713549] ipw2100 0000:02:04.0: PCI INT A -> Link[C0C8] -> GSI 11 (level, low) -> IRQ 11
[  897.713595] ipw2100 0000:02:04.0: restoring config space at offset 0x1
			(was 0x2900002, writing 0x2900006)
[  954.646319] ipw2100: Fatal interrupt. Scheduling firmware restart.
[  954.646338] eth1: Fatal error value: 0x5000CF10, address: 0x61A00000, inta: 0x40000000
[  954.646429] ipw2100 0000:02:04.0: PCI INT A disabled
[  954.692041] ipw2100 0000:02:04.0: enabling device (0000 -> 0002)
[  954.692063] ipw2100 0000:02:04.0: PCI INT A -> Link[C0C8] -> GSI 11 (level, low) -> IRQ 11
[  954.692103] ipw2100 0000:02:04.0: restoring config space at offset 0x1
			(was 0x2900002, writing 0x2900006)
[  968.585409] ipw2100: Fatal interrupt. Scheduling firmware restart.
[  968.585429] eth1: Fatal error value: 0x5000C9D0, address: 0x57E00500, inta: 0x40000000
[  968.585517] ipw2100 0000:02:04.0: PCI INT A disabled
[  968.632037] ipw2100 0000:02:04.0: enabling device (0000 -> 0002)
[  968.632059] ipw2100 0000:02:04.0: PCI INT A -> Link[C0C8] -> GSI 11 (level, low) -> IRQ 11
[  968.632099] ipw2100 0000:02:04.0: restoring config space at offset 0x1
			(was 0x2900002, writing 0x2900006)
[  972.269514] ipw2100 0000:02:04.0: PCI INT A disabled
[  972.316041] ipw2100 0000:02:04.0: enabling device (0000 -> 0002)
[  972.316400] ipw2100 0000:02:04.0: PCI INT A -> Link[C0C8] -> GSI 11 (level, low) -> IRQ 11
[  972.316446] ipw2100 0000:02:04.0: restoring config space at offset 0x1
			(was 0x2900002, writing 0x2900006)

As we can see, fatal interrupts did not dissapear, and are actually as frequent as before.

Also got this lines:

[ 2032.560413] ipw2100: exit - failed to send CARD_DISABLE command
[ 2032.560449] ipw2100: exit - failed to send CARD_DISABLE command
[ 2032.560491] ipw2100: exit - failed to send CARD_DISABLE command
[ 2032.560593] ipw2100: exit - failed to send CARD_DISABLE command

One after another, which does not provide me any clue though.

I've started several big torrent downloads/seeds as a big load, maybe card somehow differentiates different flows, so this test should be more heavy than lots of pings. First time I noticed fatal interrupt problem with this kind of load, when card not only stopped to work, but also printed some goodbay message.

So far conclusion is not very optimistic: fatal interrupts happen always, no matter what magic is enabled in the reset, which already tells that firmware is broken.
Hopefully additional reset games with power management will allow card to work, even with those interrupts. Time will tell. Comments (1)

September 25, 2008 09:01 AM

Russell writes about the iphone. I think he's missing a few things.

The open nature of the PC wasn't inherently what brought it greater success. The open nature of the PC meant that it could spawn an ecosystem of third party hardware vendors, sure. It also meant that it could be cheaply cloned by other manufacturers, ensuring competition that drove down the price of hardware. The net result? x86 is ubiquitous, sufficiently so that even Apple use a basically standard[1] x86 platform these days. Low prices and the wide availability of software that people wanted to run bought the PC the marketplace, with Microsoft being the real winners. Apple hardware remained more expensive for years, and the compelling MacOS software was mostly limited to areas like DTP. Nobody else had any incentive to buy a Mac.

Now, let's look at the phone market. Third party hardware vendors? No real distinction between the iphone and anything else. Sure, anything remotely clever has to plug into the dock port, but developing something to work with that also gets you into the ludicrously huge ipod market. Other phone accessories are either batteries, chargers or headphones. That's really not going to be what determines market success.

Competitive cheapness? When you have a multivendor OS like Android or Windows Mobile, you might expect there to be more opportunity to compete to undercut each other, offering equivalent platforms for less cost. But that's missing something. In the same way that the home computer market has basically consolidated towards PCs, the phone market has already consolidated. Your smartphone has an ARM in, probably along with an off the shelf GSM chip and some 3D core (generally something from Imagination, though in Android's case Qualcomm seem to have come up with their own core - I haven't been able to find out if it's derived from something else). There's no realistic way to make a phone with equivalent hardware functionality and quality to the iphone and sell it for significantly less money. And if you figure out how to, Apple get to take advantage of the same price reductions in their next generation hardware. And, being Apple, they'll probably find some compelling wonderful design feature that costs them nothing extra but makes you want it more anyway. So hardware competition probably isn't going to be what determines market success.

Which leaves two things - advertising and applications. Apple are good at marketing. This is unfortunate, because I'd really rather live in a world where everyone running MacOS was running Linux instead, but we seem to suck in comparison. The good news is that Microsoft also seem to, so maybe we'll have our act together some time between now and Apple crushing us to death. So, assuming current trends continue, Apple's marketing probably isn't going to kill the iphone. Which leaves one thing: applications.

The obvious argument against the iphone's success is that, as a closed software distribution platform, it's less attractive to developers. I don't think that's true. If we look at the console market, the gp2x was hardly a PSP killer. Or a DS killer. You could possibly argue it was a Gizmondo killer, but only if you ignore the Finnish mafia. Being an open platform doesn't immediately result in you killing closed platforms. You need developers, and you need applications. Otherwise nobody's going to buy your hardware, even if it costs $10 less than an iphone and has a few extra bits of plastic. What attracts developers? An attractive development environment and a revenue stream. Android has one real thing going for it here - it's not tied to Objective C, and so there's probably a larger number of potential developers. But let's be realistic. If you're a competent developer, you can move from C++ or Java to Objective C without too much effort. And if you're an incompetent developer, you're not going to be deciding the future of a platform.

Apple have made it easy for people to write applications that share the iphone's delightful[2] UI. There's almost active encouragement to write beautiful programs that integrate well. Sure, the platform limitations bite you in weird ways (like the no background running thing), but Apple have come up with hacks to smooth most of those over. The iphone is a wonderful device to develop for. Sufficiently delightful that there was a huge developer base even before Apple had released the SDK. What does that tell you? Developers actively want to write for the iphone. In fact, they wanted to even before there was a real revenue model. Mindshare means a lot.

What are we going to see in response from Android? To begin with, uglier applications. I'm sure that'll get better over time, but right now the Android UI just isn't as well put together[3]. It's functional, even attractive. But it's not beautiful. And lowering the bar to developer involvement means the potential for more My First Phone Application. Windows Mobile and Symbian have huge numbers of applications. They're mostly dreadful lashups of functionality you'd never want and a UI that's ugly enough to make you want to stab out your eyes, coupled with a nag screen asking for a $10 donation to carry on using it assuming it hasn't crashed before it got that far. To be fair, a lot of iphone stuff isn't much better. But proportionately? Right now the Apple stuff has it. I never want to see another listing of Symbian freeware.

At the moment, Apple wins at providing compelling applications. They may be a gatekeeper between the developer and the user, but right now that's not causing too many problems. Well. It wasn't. The recent fuss about Apple dropping applications because of perceived competition with their own software is an issue. If a developer is going to spend a significant amount of time and money on an application, they want a reasonable reassurance that they're going to be able to ship it. And, right now, Apple's not giving that. It remains to be seen whether this has long term consequences, but there's some danger of Apple alienating their developer base. If those developers move to another platform, and if they create compelling software, Apple might stand some real competition. At the moment? Apple has the hardware, the OS and the applications. They have the potential to take over broad swathes of the market. But they also have the opportunity to throw it away. And that's what's going to decide the success of the iphone - a closed platform is not inherently a problem, but it gives the vendor the option of removing one of their key advantages. If Apple get through this with their developer popularity intact, I don't see the open/closed distinction as having any real-world importance at all.

The relevance to Linux? We're not going to succeed by being philosophically better. We have to be better in the real world as well. Ignoring that in favour of our social benefits doesn't result in us winning.

[1] It's slightly more legacy free than a "genuine" PC - there's no i8042, and things like the gate A20 hack aren't implemented. But it'll boot DOS (given enough effort), so hell, it's a PC

[2] And yes, I genuinely do think that the iphone's UI is better than anything else on the market. There's no reason someone else, including us, couldn't have got there first. But we didn't, and now everyone gets to play catch up. Shit happens

[3] I have no idea where Apple gets its UI engineers from, but someone needs to find the source and start waving huge piles of money to pick them up first.

September 25, 2008 02:48 AM

At OpenExpo, Swisscom Research had it's own small stand (wouldn't call it a booth) to demonstrate thei research and evaluation work based on Openmoko. This is definitely exciting news, first of all since it is the research department of an established carrier, i.e. Openmoko is considered seriously even by them.

Secondly, they have many interesting ideas, some of which they have implemented. They have created a much more simplified UI, as well as an interesting input method based on gesture recognition. They've also been working on some crypto and security related bits.

I can now also disclose the fact that both Rasterman and myself have been (and stilll are) providing a bit of consulting and R&D services for Swisscom.

September 25, 2008 02:00 AM

For quite some time there has been a project called Wireless3G4Free. I suspect it is little known outside a certain academic community. So what is it all about? Creating a FOSS based test platform for wireless 3G systems. Yes, this is the so-called baseband side. The parts that are usually very carefully locked away in the proprietary stack of cellular handsets and other equipments

Even though the project, funded by the European union and implemented at EURECOM in France, is 'finished', it is not as easy as to just use that software and get UMTS connectivity.

First of all, it implements the 3.84MChip/s TDD variant of the physical layer (layer 1), whereas most commercially deployed UMTS systems for cellular access use the FDD variant. For those not as familiar with 3G technology: There's three different layer 1 options: the 3.84MChip/s TDD, the low-chip-rate Chinese TDD variant, and the FDD variant. The layer 1 is separated in two parts, one that is TDD/FDD independent, and the other part that is shared.

Secondly, the Wireless3G4Free project uses IP on the layer 3, as opposed to the actual layer 3 protocol of UMTS (which borrows a lot from the layer 3 protocol for GSM, which in turn borrows a lot from Q.931 / Euro-ISDN).

So if one was to make that code interoperate with UMTS cellular networks, the lower half of layer 1 need to be rewritten for FDD, and layer 3 needs to be implemented.

What is exciting about 3G compared to GSM: GSM uses proprietary ciphers (A5/1, A5/2) for the actual air interface. Those ciphers have leaked quite some time ago, and they're no longer secret (and thus the GSM security is no longer existing), but still people are not supposed to know how it works.

In the 3G world, the corresponding cipher is public. This means that in theory, it should be possible to implement everything in Free Software based on publicly known information. Yes, it is a lot of work. But it definitely can be done.

Before actually using this on any official network, it would obviously need to be certified. Certification for this kind of protocol is a time-consuming and expensive process. It requires development cycles of going to a certified test lab, obtaining test results, going back to actually fixing the problems, re-running the test lab tests, and so on. Nevertheless, Free Software has already proven that this can be done. The isdn4linux project did a full EDSS1 certification some 10 years ago. ELSA, a maker of passive ISDN cards, sponsored that effort. And if you used an unmodified code version, then you were certified. As soon as the source code was changed, you were running an uncertified version. I don't see any big problem why the same scheme should not work for a 3G baseband software stack.

One important question though, is the question of hardware. None of the existing commercial vendors of 3G chipsets will ever provide you with the hardware documentation that you would need or want to run that kind of code on their hardware. It is their business to sell their proprietary 3G stack along with their chip, so they would only loose money if there was any FOSS implementation in competition.

Sure, you can use something like the USRP or USRP2 or any other software defined radio platform. But while that would be ok for a proof-of-concept, it is too large, expensive and power-consuming to be used or 'ported' to any actual handset-type product.

So any possible real-world plan of making this happen would probably go as far as to implement everything based on the USRP, then have a proof-of-concept prototype and then do a modem design based on existing, openly documented RF components and ADC as well as DSP+Processor combination that is suitable for low-power operation.

Sure, I'm just daydreaming. But sometimes you have to dare to dream in order to make things happen. Anyone wanting to turn this idea into a business, let me know ;)

September 25, 2008 02:00 AM

I've had the honor to join other experienced FOSS hackers on the Jury for the Hackontest. The idea was to let the community collect a number of work-items for teams (of 3 developers) working on a particular project, then pick some of those work items and see how far each team gets within 24 hours.

I think it was a very interesting concept. Something that at least I have not yet seen anywhere else before. The organizers did a great job with the preparation. Setting up the website for project proposals, collecting community votes on the individual tasks, putting together the jury.

The participants of the contest then were placed into a container (yes, the kind of containers used for international shipping) with a fridge, beer, snacks, Internet, power, a projector and some other gear. They had vnc running on all of their systems, to enable a large public screen at the trade show where people would be able to follow whatever happens on-screen right now on a system of a random developer participating in the context. 'reality-tv for hackers' ;)

The results and the progress were quite different between the individual projects. I don't want to reveal the internal discussions we had in the jury, but one thing that basically everyone agreed to was the improper use of revision control systems. None of the teams was setting a good example on how to use them. Either the granularity of the commits, or the changelog texts, or the time when they committed was wrong. You shouldn't commit unrelated changes, never do cosmetic and functional changes in one commit, etc. This is what makes your work reproducible, readable and understandable to others, like the jury and particularly your own user and developer community. It is one aspect that affects a lot how easy it is for others to contribute to your project or to contribute to it.

September 25, 2008 02:00 AM

Some things I'd really love to see to improve the usefulness of bugzilla (and other bugtrackers)

• Comment nesting.
  Basically multiple "Reply to this comment" buttons, one after each post, instead of one "post a new comment" button.
  
• A "collapse this thread in *my* view of this bug".
  Useful for ignoring inconsequential stuff that people may post to the bug. "HEY! I SAW A KERNEL OOPS ONCE TOO!"
  
• (going a bit crazy now). Comments Points/Moderation/Scoring.
  Vote down a comment so it gets auto-collapsed in everyones view of the bug.
  

(This is beginning to sound like the bastard child of bugzilla & digg/slashdot, so I'll stop here).

I've had these thoughts a few times over the last few years triaging kernel bugs for Fedora. I think they would be worth additions especially for the instances of a single bug that gets many people jumping on it. Especially high profile issues like the current e1000e scare that has a bunch of people worried.
(Though the Fedora bug tracking that problem is quite tame in comparison to the ubuntu one right now. I actually pity those guys having to wade through all those comments, with so much not-directly-useful-or-relevant commentary).

But we've had issues in the past in Fedora where we've easily had over 100 comments. Trying to keep up with what's going on when there are 16 different people all with slightly different (or even completely different in some cases) problems is nearly impossible after a point. On a number of occasions, I've just closed the bug with a comment along the lines of "guys, this is madness, please file individual bugs, and we'll work through them".

Part of the problem is context. If all I had to do was look at that one bug, and work on it, it wouldn't be such a big deal. Due to there being more bugs than developers (in any distro), time-slicing occurs, and regaining 'state' when loading up a bug again takes a while. When there's pages and pages of comments, it's sometimes impossible to regain the full picture just due to the amount of noise.

September 24, 2008 10:54 PM

At kernel summit.

Check new ones! Comments (0)

September 24, 2008 07:00 PM

This is a maintenance release, which contains following changes:

• Use idr to manage minor numbers. Now create/remove/create sequence does not produce new minor, but uses previous one, which is now freed.
• Added cache name to the node. It is possible to have freed node still being alive while we register new node with the same name, so its cache name should be different.
• Wait during node removal until there are no pending transaction, so node would be freed in process context and not in the receiving threads itself.
• Warn user if there is no security permission config file during export node initialization. No client will be allowed to connect without explicit security association.
• Tune default size of the page pool for crypto processing a bit.

I want to thank Remy Ritchen (remy.ritchen_gmail.com) for his excellent tests and analysis.

As usual, DST is available from archive and via git tree. Comments (0)

September 24, 2008 05:00 PM

As many people have pointed out to me, the posting of the Linux Plumbers Conference keynote on Google Video makes it kind of hard to watch using "free" software. So I tried to work out how to convert the original file to .OGG format.

And I failed.

So, any hints? Someone did this last time around for me for my talk at Google, which can be seen in the fancy new "media" directory on kernel.org right here. I'll be glad to put up the keynote talk, and some other videos that I have of talks if I can get them converted.

Update: Lots of people have pointed me to the excellent ffmpeg2theora tool, which I'm now using to convert the videos. Thanks for all of the help, I really appreciate it. I'll have copies of the videos up soon...

September 24, 2008 03:17 PM

I've uploaded man-pages-3.10 into the release directory (or view the online pages). This is a fairly light release; conferences, and learning and changing my workflow for git have take a bit of time lately. Notable changes in man-pages-3.10 are:

• The clone(2) and getpid(2) pages add some notes and details of the consequences of the PID caching performed by glibc's getpid() wrapper function.
• Various changes to services(5), including removing various out-of-date pieces of text.
  

September 24, 2008 12:14 PM

I managed to compile small enough kernel, which boots on my laptop (do not know how long it took, since fell asleep), and managed to bring fatal interrupt error just after several seconds of ping -f 192.168.1.1 -s 8192 on freshly booted machine. 192.168.1.1 is my gateway address.
Here is the result with the patch I posted to the mail lists, which was not acked, replied and commented though (well, I have to admit, that if I would send it couple of mails earlier, it could probably find its way into the tree, but I still believe that it would not result in anything, since everyone knows about this bug, it just is not fixed by some reasons). Intel developers (at least those who maintain the driver) continue to keep silence.

[  613.960164] ipw2100: exit - failed to send CARD_DISABLE command
[  624.456033] eth1: no IPv6 routers present
[  690.721534] ipw2100: Fatal interrupt. Scheduling firmware restart.
[  690.721554] eth1: Fatal error value: 0x5000C97C, address: 0x100E201C, inta: 0x40000000
[  690.721580] ------------[ cut here ]------------
[  690.721587] WARNING: at drivers/net/wireless/ipw2100.c:3188
	ipw2100_irq_tasklet+0x8fe/0x9b0 [ipw2100]()
[  690.721736] Pid: 0, comm: swapper Not tainted 2.6.27-rc7-mainline #2
[  690.721744]  [] warn_on_slowpath+0x5f/0x90
[  690.721763]  [] up+0x11/0x40
[  690.721773]  [] release_console_sem+0x190/0x1d0
[  690.721786]  [] enqueue_hrtimer+0x72/0xf0
[  690.721795]  [] printk+0x1b/0x20
[  690.721805]  [] ipw2100_irq_tasklet+0x8fe/0x9b0 [ipw2100]
[  690.721831]  [] hrtick_start_fair+0x157/0x170
[  690.721844]  [] enqueue_hrtimer+0x72/0xf0
[  690.721855]  [] snd_intel8x0_interrupt+0x1d7/0x250 [snd_intel8x0]
[  690.721875]  [] tasklet_action+0x46/0xb0
[  690.721886]  [] __do_softirq+0x75/0xf0
[  690.721897]  [] do_softirq+0x37/0x40
[  690.721906]  [] do_IRQ+0x40/0x70
[  690.721917]  [] getnstimeofday+0x37/0xe0
[  690.721927]  [] common_interrupt+0x23/0x28
[  690.721937]  [] sys_setpgid+0xd8/0x190
[  690.721955]  [] acpi_idle_enter_simple+0x15a/0x1c1 [processor]
[  690.721980]  [] cpuidle_idle_call+0x7b/0xc0
[  690.721991]  [] cpu_idle+0x46/0xe0
[  690.722000]  =======================
[  690.722006] ---[ end trace 70268f59a00d957c ]---
[  695.271318] ipw2100: Fatal interrupt. Scheduling firmware restart.
[  695.271337] eth1: Fatal error value: 0x50014148, address: 0x60207E04, inta: 0x40000000

writing this note and starting over

[ 1520.709136] ipw2100: Fatal interrupt. Scheduling firmware restart.
[ 1520.709156] eth1: Fatal error value: 0x5000C96C, address: 0x538E7E40, inta: 0x40000000
[ 1550.954315] ipw2100: Fatal interrupt. Scheduling firmware restart.
[ 1550.954334] eth1: Fatal error value: 0x5000C99C, address: 0x08418004, inta: 0x40000000
[ 1592.175473] ipw2100: Fatal interrupt. Scheduling firmware restart.
[ 1592.175492] eth1: Fatal error value: 0x50018588, address: 0x57E77A00, inta: 0x40000000

So, this fatal error value and address numbers do not tell me anything, but since they are always different on different addresses, I think firmware just loses its mind and stops responding.
The first line, where ipw2100 fails to send a command, was obtained during ifdown of the interface. I never saw it before, but do not think it is related though.

So, I need to move to the office and want to make some distributed storage changes, namely fix an issue with name collision (kernel already has a dvb card, which module is called dst.ko), and implement better minor number allocation scheme for the imported devices, since right now after node was created and distroyed, new one will not get the same number, but continuously increasing one, which looks confusing and may bring a sysfs initialization error (when system tries to register kobject with existing name).

I will continue ipw2100 experiments today's night if will not fall asleep again because of jetlag. Stay tuned! Comments (0)

September 24, 2008 09:00 AM

During the least couple of days I've had some pretty intense conversations with a number of people on various aspects of GSM, leading me to [re]reading some of the interesting bits of its specification.

There are a number of observations that I don't want to talk about right now, and which will likely be part of my work during the next couple of months.

One thing that ever so often gives me the creeps is STK (Sim Toolkit). To those people involved with GSM, it is no news that with STK an operator can basically remote-control your phone. He can, among other things

• make your phone send SMS
• initiate outgoing calls without your interaction
• initiate outgoing calls and terminate any existing call
• open data connections (GPRS/EDGE)
• launch a browser to any URL
• play tones on your speaker
• access and modify any information (contact, SMS, dial history, even IMSI) stored on the SIM

And the worst thing of it all: You don't even know which of those features your phone implements (most likely all of them). I'm happy to still use a SIM that predates the GSM11.14 (STK) specification.

Now in the advent of projects like OpenBTS, where we can emulate a GSM network side, and in combination with either supplying your own SIM card (or emulating it using a PC), we will finally see a faint possibility of actually testing (and demoing) the never-ending security nightmares caused by this evil monstrosity.

September 24, 2008 02:00 AM

The video for my keynote has been published on Google Video, hopefully the other talks get posted there soon as well.

September 23, 2008 09:42 PM

At kernel summit.

Last couple of photos were made at Linux Plumbers Conference (filesystem bof). Not all of them got into the gallery though, I need to try to find missing bits.
I needed to get a real flash instead and do not use build-in one, which sometimes mangled the images...

Got a look? Comments (3)

September 23, 2008 05:00 PM

I've just arrived in Winterthur (Switzerland) for OpenExpo where I will present on the various reasons and implications of the fact that 99.9% of the makers of commercial embedded Linux products have not understood the slightest bit about Embedded Linux or rather, embedded FOSS in general.

Those people who've ever tried to exercise their freedom to create and run modified versions of an embedded product with pre-installed Linux will definitely know what I'm talking about.

September 23, 2008 02:00 AM

for real.

I couldn’t have been happier with how the Linux Plumbers Conference went last week. I went back and looked at the original proposal that we had Arjan, Greg, and Randy present to the Linux Foundation, and we seem to have hit all our original goals. From conception we wanted this to be a “working” conference - and from the conversations in the hallways that I overheard, to the discussions in the microconfs that went on, I could see that people were indeed getting together, discussing issues and solving problems. Conferences require a lot of time, effort, and money to do right, and it’s gratifying to feel that something useful will come out of this.

I think that now I can go back to blogging about duck poo and vegetables.

September 22, 2008 09:50 PM

After reading some of the comments at lwn about my sessions last week at kernel summit/plumbers conf about writing a 'make initrd' target for the kernel that would work on every distro, I feel some clarifications are in order.

What I am not doing.

• Dropping the Fedora mkinitrd script into the kernel.org tree and hoping other distros will make it work for them.
  
• Obsoleting the existing mkinitrd from Fedora for a considerable amount of time. (Think RHEL7 timeframe).
  

What I am doing.

• Right now, nothing but decompressing. Last week was pretty intense, with a lot of feedback to take in.

What I've done

• I've looked at other distros initrd's & creation tools. They all suck equally. Really. If you disagree, you either haven't looked hard enough, or you have emotional attachment issues. There is really nothing amazing about your distros tools over any others. (Which is why I'm of the opinion that "do it right, once, upstream" is the right answer.
  
• Written very little code. A few dozen lines of shell on the planeride home. I'm not posting anything until it at least is limping[1] along.
  

What I'll do next

• Yes, I'm starting afresh. I'll be borrowing some bits from various distros initrd's as I stumble into "hmm, how _does_ this currently work" territory, but this is going to be done with baby steps, first of all just booting off simple setups with /dev/[hs]da, no root on nfs/iscsi/nbd etc.
  
• At least in part, I'm starting with something completely new for political reasons. It's the only way to get anyone to agree on how to make progress. With everyone so invested in their current working solutions, "just use the debian one" or "just use the fedora one" isn't going to get us anywhere.
  

How is this going to work?

• For Fedora systems, we'll be building an additional initrd. Currently we only build it at kernel RPM install time, tailored for that system. Going forward, a one-size-fits-all initrd will be made during the package build process. Which gets used will depend on an /etc/sysconfig/kernel/ setting. This way, we can continue to use the crufty mkinitrd until this thing is ready, and the bleeding edge lunatics and people actually wanting to work on this can set the variable and have the new hotness.
  
• The very first step however is recovering from some of the fallout of the boot/init session of plumbersconf. Reviewing our CONFIG options. It's likely that use of 'make initrd' will imply that certain options should be set certain ways.
  

Once I'm feeling a little more on my feet[2], I'll make some more posts on what I intend to do, and how. Until then, continue to speculate wildly.


[1] Trapped sciatic nerves are the _worst_.
[2] Seriously. Screw flying anywhere for a while. I'm staying home.

September 22, 2008 07:38 PM

It was nice to see the large response from my Linux Plumbers Conference talk, and there seems to be a few common themes of questions about the talk that I figured I'd clear up here.

I have seen the video of the talk, and the video team from the Linux Plumbers Conference is working to put it up online somewhere, hopefully soon. I'll link to it when it is available.

First off, my numbers for the binutils development was completely WRONG. Kees and I sat down and tried to figure out exactly why I didn't count his valid contribution, and it turned out that binutils puts a ChangeLog into each subdirectory, the top-level one is not the summary of all of the individual parts of the project. So I apologize about that one, Canonical really did have one binutils patch in the past 3 years. Not that this really affects any of the main points of my talk though...

All of the other numbers for the other projects are still correct, from what I can tell. If anyone thinks I got them wrong, please let me know and I will be glad to review them. Feel free to review the changelog and svn and git trees of the different projects if want to verify.

One main question that I saw a lot, and was even asked about during my talk, was "what about Canonical's work on the desktop/Gnome/KDE"? I really don't know if they have contributed a lot of effort back upstream on these projects, that wasn't my point here.

Remember, this was given at the Linux Plumbers Conference a gathering of developers of the low-level plumbing of Linux. This wasn't a group of desktop developers, so remember the audience that this was addressed to please.

If Canonical has contributed a lot to Gnome/KDE, that's great, I'm sure someone will post the numbers soon to verify this. Either way, please remember that this was not the audience that I was addressing.

I sat down with Matt the day after my talk, as he described, and hopefully the Canonical kernel developers will work to become more of a valid part of the community, which is what I am sincerely hopeing will happen here.

Oh, and Amanda, I have given this very same kind of talk to Amazon, a number of months ago, as well as many other companies over the past 1 1/2 years, so it's not like I am ignoring them at all.

And this response brings me back to my main point of my talk, which most people seem to have missed as they were upset at me pointing out Canonical's lack of upstream contributions. And that point was, and still is:

The market right now is just too good for individual developers who have experience in writing open source software for Linux, especially the low-level plumbing of Linux, to waste their time working for companies who do not allow them to contribute back, if they want to.

This was a developer conference. I am a developer, talking as myself only, and not as a representative of any company (note the total lack of any corporate branding on my slides), to other developers who I totally respect and want to see be as happy in their day-job as I am in mine.

I would like to point out that lwn.net's summary of the talk did get this correct, which was great to see.

Hopefully this helps clear things up, if not, let me know and I'll be glad to address it.

September 22, 2008 05:11 PM

I'm sorry, I'm not going to waste time on this if you keep acting this dishonest; welcome to my mail filter...

If we pretend to not know about the problem, problem comes and hits us out of stand. Comments (6)

September 21, 2008 10:00 PM

Actually I think I like this city much more than when just saw it. It is small, but still alive, it has parks and river as long excellent coast to walk at (without access to water though, since it is navigable). There are interesting buildings and lots of places where to take a seat like restaurants, cafes and pubs. Once heared live jazz music from the street, but was suggested to visit Ostin: capital of the live music in the USA.

So, couple of photos of Portland I made (without any artistical attempts). Several KS and Plumbers photos are pending. Comments (5)

September 21, 2008 09:00 PM

Just sent a patch to zillions of maillist (netdev@, linux-kernel@, linux-wireless@) and to lots of developers because of its Fatal interrupt. Scheduling firmware restart. problem.

Let's see if Intel folks will do anything.

Also added couple of jokes about conspiracy theories (like bug fires because Intel forces us to buy a new adapter by this error) to make it a little bit more flameable and to bring attention. I really hope Intel does not do it intentionally. Comments (3)

September 21, 2008 06:00 PM

Only to be greeted with a leak in the water main from the street to my house. Nothing a few trips to the hardware store, some digging, gluing and clamping couldn’t fix though (my patch has been holding so far at least). Hopefully next month’s water bill won’t include over 20,000 gallons of leaked water…

But I digress. Plumbers turned out to be a great conference. Kristen & co. had some ambitious plans for the event, but they didn’t disappoint. The conference was extraordinarily informative and productive, with many of the “talks” being more like working sessions, where everyone shared information and fixed or solved problems on the fly. The highlights for me were the Audio, Boot and Graphics tracks.

At the Audio track I learned a lot about how ALSA works from Takashi-san; I think we all agreed that some of the ALSA bits should be pushed upstream a little more aggressively, to support new hardware and fix configuration problems. Also I got Lennart to promise me that mixer settings would suck at least 100% less in future versions of Pulseaudio, which although better than straight ALSA is still lacking in some areas. There was also some discussion of adding virtual output devices to the ALSA library plugin API. I think this makes a lot of sense, since when I associate my bluetooth headset I want to be able to see it in application drop downs immediately.

The Boot sequence track was interesting because we got to see a demo of Arjan & Auke’s work to reduce boot time to 5 seconds on an Eee PC running Moblin. Some of their work was added to Fedora and Ubuntu immediately following the talk, and if nothing else I think they’ve given the distros something to shoot for. Dave and Kyle talked a bit about initrd and startup services; hopefully we’ll see some unification (mkinitrd in the kernel tree basically) sometime soon, assuming Dave makes it home.

Maybe I’m biased, but I thought the Graphics track the most interesting. Dave and I started out with an update on the kernel mode setting work. I demoed panic under X (and it worked!) and got some applause, which was nice. Most of our time was taken up with an argument with Linus about our development process, however. The discussion went something like this (paraphrasing here):

Linus: “You guys are idiots, you should have developed this incrementally and pushed it upstream a long time ago”

Us: “No, you’re a pinhead, we’ve done the incremental development and this is where it took us. We’ll be pushing upstream soon now that we have APIs we can actually support.”

Linus: “No, you guys don’t get it, you should have done the development like this, x, y, z”

Us: “Oooh, we see what you mean now. Yeah we tried that, it sucked real bad.”

Linus: “You’re dumb.”

Us: “Well you’re a pinhead.”

And then we agreed.

Ultimately, after some offline conversations I think we came to the conclusion that yes, developing out of tree for as long as we have sucks, but on the other hand there aren’t any great alternatives with sane kernel APIs that we can support along the way. All of this means that we should see kernel mode setting upstream soon; even if it’s not fully cooked the interfaces are in good shape so all that’s left is bug fixing.

The other topics we discussed were interesting too: Carl went over what happens when you try to draw text to the screen, and all of the ways it can end up being slow (which it has done for a long time now with EXA), then Jaya talked about some of the unique aspects of E-paper graphics support. It was a really interesting topic, with some unique problems (some of which we found solutions to on the fly during Jaya’s “open issues” time). Ian concluded with an overview of OpenGL 3.0 features and what it’s going to take to get Mesa to support them. All in all, good stuff and a great conference.

But with all of that out of the way, it’s back to the hard work of getting GEM, kernel mode setting, and GTT mapping merged, along with fixing bugs in xf86-video-intel prior to release, managing the PCI queues, and doing some power on work for an upcoming mobile platform. No problem.

September 21, 2008 02:47 AM

Today was the last day of the Linux Plumbers Conference, which overall seems to have gone really well. Certainly it exceeded my expectations, which were already pretty high. In my view, the conference was distinctive in that it was totally developer-focused and collaborative, with no thinly-disguised marketing talks.

The atmosphere was relaxed, and not overly structured, which allowed for a lot of useful ad-hoc discussions between developers working in different areas of the OS. An example was Arjan's talk on achieving a five-second boot, which itself was very interesting and entertaining, but was also followed by a lunch session with a bunch of distro maintainers to work out various specifics. It seems that a small arms race has been launched between Fedora & Ubuntu on who can first get the default install to a five second boot.

I was interested to catch up on the latest file system developments, and caught the updates on btrfs and crfs by their respective authors, Chris Mason & Zach Brown. The disk format for btrfs will be locked in before the end of the year, according to Chris, to encourage more developers and users to start playing with it. crfs is looking increasingly impressive as a small-scale, fast, reliable and sane networked file system: I grabbed a photo of the slide comparing it with other network filesystems:

Other photos I took at the conference are here.

It was really great to catch up with so many people I work with over the net, and also finally meeting some people I must have known for more than a decade but still never met in person -- possibly due to this being the first Linux conference I've attended in the US.

During the closing, Kristen Accardi did a brief survey on several aspects of the conference, and it seems that virtually everyone was happy with it. I think the conference has a bright future, as it seems to have filled a now obvious need for a place where a cross-section of mainline Linux developers can meet up specifically to solve problems.

September 20, 2008 01:22 AM

navit quickly approaches usable state. It can display vector map (for example openstreetmap), it can route, it can provide directions, and it can even search for place.. provided towns are properly marked with is_in=(country name). Not even Prague is marked like that :-(.

Situation with streets is even more interesting... navit assumes city/town/village has certain area (based on if it is city or town or village) and puts nearby streets into that town. I guess is_in should be an improvement there...

September 19, 2008 09:30 PM

Fedora, like quite a few Linux distributions, aims for the technical challenge of being self-hosting, that is, one should be able to recompile each of the packages in Fedora 10 using a copy of Fedora 10, and get exactly the same result.  For Open Source and Free Software developers, this is a critical feature which ensures that when someone has a program in binary form, they can be assured that the source code purporting to be that of the binary is, in fact, the same corresponding source.

One challenge to self-hosting a project the size of Fedora (now with about 6200 source packages) is dealing with the interdependencies between packages.  When a major component, such as the compiler or an often-used library, upgrades to a new version, you should rebuild all packages that depend upon that major component, to ensure they continue to work.  Often, simply re-compiling or re-linking each package using the updated compiler or library is all that is needed. In some cases though, applications which once built, no longer do - bitrot has set in.

For a couple years now I have been rebuilding all of the packages in the Fedora "rawhide" tree (the code that will make up the next major release), using only the code in rawhide to do so.  This tends to find bitrotting code relatively quickly, which I then report to the package owners to resolve.  I run these builds roughly monthly, striking a balance between annoying package owners with build failure reports, and timeliness of detecting such failures.  At the far end of the spectrum, openSUSE's build service cleverly rebuilds all packages "higher" in the dependency chain whenever a component "lower" in the chain is modified.

Borrowing a page from Debian's playbook, "Fails To Build From Source" (FTBFS) bugs are considered serious in Fedora as well.  When GCC 4.3 landed in rawhide back in early 2008, about 560 packages no longer built, and needed review and fixups.  As of this week, all but 4 have been resolved, the rest are pretty tricky and are being actively worked.  Thank you to all the Fedora packagers for fixing these up!

So, now it's time for the next pass.  Several changes to rpmbuild have been made recently, including disallowing patches to be applied with fuzz (126 packages tripped by this), and automatically building python egg-info content which must be included in the package.

For all packages failing due to patch fuzz, now is a good time to review what the latest version your upstream developers have released, verify that the patch is still necessary, and send the patch upstream ASAP if it hasn't already been applied upstream.

September 19, 2008 02:54 PM

Ok, so that was the end of the Linux plumbers conference. Really interesting. I think I've gotten a bunch of info and ideas that'll be key to solving various problems I have encountered. Wow, I can't believe how fast the past 4 days have disappeared. Time to crash because I stayed up all night writing slides...

September 19, 2008 03:44 PM

Just when investors and financial journalists are asking themselves "Can't SEC chairman Christopher Cox do anything right?", he answers the question by proposing a complete ban on short sales.

If manipulative short selling is an issue, why not simply reinstate the uptick rule or require the large funds to pre-borrow shares they short?

After all, Cox only abolished the uptick rule last July and we've seen the results. Going from one extreme to the other makes little sense, when we had a perfectly working middle ground.

read more

September 19, 2008 02:53 AM

[Update: 19 Sep 2008: when I first attempted the git import, I didn't import the subversion tags into git. I've updated and expanded this post to include the required details to do that.]

When I inherited man-pages, there was no version control system (VCS) in use. To help myself keep track of changes, I've been running a private subversion repository since I took over as maintainer (i.e., since man-pages-2.00), but I never got round to hosting it on a public server so that people could pull from it (requests for such a facility were only occasional). Instead, people wanting to send patches would just grab the latest tarball from the downloads directory, patch the required source file, and email me the patch.

Somewhat more frequent requests for a public repository, and the fact that the Linux world is nowadays mostly oriented around the git distributed version control system, have gradually created a pressure to change things. So, I'm taking baby steps towards using git for man-pages. Here goes...

Importing from subversion