Kernel Planet

Michael Kerrisk (manpages): man-pages-3.41 is released

Thu, 17 May 2012 06:46:00 +0000

I've released man-pages-3.41. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

The most notable changes in man-pages-3.41 are as follows:

A new get_robust_list(2) page, written by Ivana Varekova, documents the get_robust_list() and set_robust_list() system calls.
Three new pages written by me—mallinfo(3), malloc_info(3), and malloc_stats(3)—document various interfaces used to monitor the state of the malloc implementation.
The madvise(2) page adds documentation of the MADV_DONTDUMP and MADV_DODUMP operations.

James Morris: Reminder: CFP for the 2012 Linux Security Summit closes in 1 week!

Wed, 16 May 2012 21:45:17 +0000

A reminder for folks planning to submit proposals for the upcoming Linux Security Summit in San Diego — the CFP closes on the 23rd of May, a week from now.

LSS is one of eight co-located developer events at LinuxCon this year, including the Kernel Summit and Plumbers. It’s shaping up to be an epic event!

Pavel Machek: Multi-monitor setup using notebook

Tue, 15 May 2012 13:36:32 +0000

I'm right handed. I'd like to use notebook with big monitor, docking station, external mouse and keyboard.

Question is, what is the reasonable setup? For now, I have notebook to the left of the big monitor, with X set up to use both monitors, and notebook "to the right" of the big monitor. Yes, it is usable, but moving mouse right to get to the display that is to the left of the screen is strange.

If I put notebook to the right of big monitor, I'll have no place for the mouse.

So... is there clever solution?

(I'd still like to undock and be able to use the apps I've opened.)

Dave Airlie: ripping the X server a new driver API

Mon, 14 May 2012 17:05:58 +0000

So I've been slowly writing the hotplug support v3 in between all the real jobs I have to do.

[side note: When I started out on hotplug. one of my goals was to avoid changing the server API/ABI too much so I could continue side by side testing.]

how did I get to v3?
v0.1: was called dynerama it failed miserably and proved that using Xinerama as the plugging layer was a bad plan.
v1: was the first time I decided to use an impedance layer between some server objects and driver objects.
v2: was the a major rebase of v1.

v2 was trucking along nicely and I managed to get the design to the stage where PRIME offloading intel/nouveau worked, USB device hotplug with udl worked, and GPU switch worked between two drivers. However v2 duplicated a lot of code and invented a whole new set of API objects called DrvXRec, so DrvScreenRec, DrvPixmapRec, DrvGCRec etc, this lead me to looking at the pain of merging this into the drivers and the server, and my goals of avoiding changing the API/ABI was getting in my way.

So before starting v3 I decided to rework some of the server "APIs".

The X server has two main bodies of code, one called DIX, and one called DDX. The DIX (device independent X) code and the DDX (Device dependent X code). In the X.org tree the dix lives up in the top level dirs, and for X.org server the DDX lives in hw/xfree86. The main object with info about protocol screens and GPUs is called ScreenRec in the DDX and ScrnInfoRec in the DIX. These are stored in two arrays, screenInfo.screens in the DIX and xf86Screens in the DDX, when code wants to convert between these it can do one of a few things.

a) lookup by index, both structs have an index value, so to go from ScrnInfo to Screen you look at screenInfo.screens[scrninfo->scrnIndex] and other way is xf86Screens[screen->myNum]. This is like the I didn't try and make an API, I just exposed everything.

b) ScrnInfo has a ScreenPtr in it, so some code can do ScrnInfo->pScreen to get the pointer to the dix struct. But this pointer is initialised after a bunch of code is called, so you really can't guarantee this pointer is going to be useful for you.

c) XF86SCRNINFO uses the DIX private subsystem to lookup the Scrn in the Screen's privates. This is the least used and probably slowest method.

So also screenInfo.screens contains the protocol screens we exposed to the clients, so this array cannot really change or move around. So I'd like to add screeninfo.gpuscreens and xf86GPUScreens and not have drivers know which set of info they are working on, however (a) totally screws this idea, since the indices are always looked up directly in the global arrays.

Now lots of the Screen/ScrnInfo APIs exposed to the drivers pass an int index as the first parameter, the function in the driver then goes and looks up the global arrays.

So my first API changes introduce some standard conversion functions xf86ScreenToScrn and xf86ScrnToScreen, and converts a lot of the server to use those. Yay an API. The second set of changes then changes all of the index passing APIs to pass ScrnInfoPtr or ScreenPtr, so the drivers don't go poking into global arrays. Now this is a major API change, it will involve slightly messy code in drivers that want to work with both servers, but I can't see a nicer way to do it. I've done a compat header file that will hopefully allows to cover a lot of this stuff where we don't see it.

I've ono other API introduction on the list, Glyph Pictures are another global array indexed by screen index, I've gone and added an accessor function so that drivers don't use the index anymore to get at the array contents directly.

Once this stuff lands in the server, a team of people will go forward and port the drivers to the new APIs (who am I kidding).

LPC 2012: Submission Deadline Approaching

Fri, 11 May 2012 22:45:57 +0000

We have a great set of submissions for the 2012 Linux Plumbers Conference! But we are missing one thing, namely a submission from you. There are still a few more days to send in your submission, please visit http://www.linuxplumbersconf.org/2012/participate/. Submissions are due at 11:59PM Pacific Time on Tuesday, May 15th.

We hope to see you in San Diego for Plumbers this August!

Dave Jones: Weekly Fedora kernel bug statistics – May 11th 2012

Fri, 11 May 2012 16:42:50 +0000

	15	16	17	rawhide
Open:	311	583	76	141	(1111)
Opened since 2012-05-04	3	37	14	0	(54)
Closed since 2012-05-04	3	43	12	1	(59)
Changed since 2012-05-04	10	61	15	4	(90)

F16 bug count is out of control at this point. We’re still managing to close them faster than they’re coming in, but the sheer number of bugs still open is overwhelming. There still exist a lot of older bugs that may have been fixed but the user hasn’t updated the bug. As we’re coming across these bugs that have been in needinfo state for a while we’re closing them out if we have a reasonable expectation that the bug is fixed.

There’s also still a huge number of duplicate bugs, where it’s not immediately obvious that they are duplicates. The end frame in the call chain may be the same, but how we got there differs for example. Or we see multiple instances of a particular trace, from various hardware. (A good example of this case being the numerous irqpoll bugs open for which we still have no real explanation. Likewise softlockup reports).

A curious observation: We’ve had a slight uptick lately in the number of bugs that seem to be caused by bad memory or other hardware problems. Sometimes these stick out like a sore thumb (a single bit flips changing an address from ffffffff81c8bca0 to ffefffff81c8bca0 for example). Other times, they aren’t as obvious.
I have been wondering though if we see more of these in Fedora just due to the extra debug options we leave enabled even in the production builds (like linked list debug, which is pretty much free).

Something that has been interesting though when we’ve found a user with a bad ram report, is searching for other bugs that user filed. Suddenly “weird” oopses make a lot more sense.

Weekly Fedora kernel bug statistics – May 11th 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...

Related posts brought to you by Yet Another Related Posts Plugin.

Pavel Machek: mbank.cz: insecure by default

Fri, 11 May 2012 11:12:01 +0000

So you want to get a debet card. It comes by email, with instructions, that you need to activate it over the web. So you do activate it. Then you realize that all limits are way too high... like $50000 per day for payments over the web. Oops. So you go to change it quickly. At this point, authorization SMS fails to come, so you can't. Nice.

What about having reasonable limits by default, dear mbank?

Harald Welte: Some follow-up on the Osmocom Berlin meetings

Mon, 07 May 2012 02:00:00 +0000

We've now had the first two incarnations of the Osmocom Berlin User Group Meeting. The start was great, and we had probably something around 10 attendees. Some were the usual suspects like the various Osmocom developers living in Berlin. But we also had a number of new people attending each of both of the meetings, which is good.

To my big surprise people are even flying in from other parts of Europe in order to be able to attend. Last time from Sweden, and for the next meeting some folks from the Netherlands have announced themselves.

To an even bigger surprise, the attendee from Sweden announced that he is working for an Ericsson research lab, and apparently they are using OsmocomBB quite a bit inside that lab. They think it's a great tool, and apparently nothing else with the same flexibility (i.e. full source code) is at their hands that can compete.

On the one hand it is surprising to see such a large traditional Telco supplier to start to use such amateur tools like OsmocomBB, which definitely have not had even a fraction of the testing (particularly with various operators in various countries) like the commercial protocol stacks.

On the other hand, if you think more about it, Ericsson is entirely a network equipment supplier today. They have spun off their baseband processor business to become part of ST-Ericsson, they have pulled out of Sony-Ericsson, sold their TEMS product line to Ascom and other bits and pieces to Tieto. So right now, if they need a MS-side protocol stack or engineering phones, they probably have to obtain what is available on the market. And that's unfortunately not all that great, as the products are either

Measurement devices aimed at mostly L1 testing / QA (Racal, Agilent, Rohde-Schwarz)
Trace mobiles primarily aimed at field testing (TEMS, Sagem OT) and while they provide traces they don't permit you to send arbitrary data or behave spec-incompliant
Mobile Phone development platforms (Qualcomm, MTK, Infinenon, ...) which don't necessarily give you the full source code to the stack, and are only available if you actually intend to build a handset

So all in all, the more I think about it, it is actually not too surprising that they ended up with OsmocomBB. It's free (as in free beer) and they get the full source code with it. You need a lot of skills and time to get it running and find your way around how to use it, but I guess if you're working in cellular protocols and embedded systems, it's not that hard.

Dave Jones: Weekly Fedora kernel bug statistics – May 4th 2012

Fri, 04 May 2012 16:16:48 +0000

	15	16	17	rawhide
Open:	311	580	78	142	(1111)
Opened since 2012-04-27	2	36	13	6	(57)
Closed since 2012-04-27	2	20	7	11	(40)
Changed since 2012-04-27	9	61	13	11	(94)

Weekly Fedora kernel bug statistics – May 4th 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...

Related posts brought to you by Yet Another Related Posts Plugin.

Pavel Machek: 17.98 km/h average over 5 kilometers

Fri, 04 May 2012 11:07:57 +0000

16.37 km/h average over 10 kilometers. Yes, my fjord horse just likes to run. And no, I can't compute, which meant missing the deadline by 15 seconds, and not doing too well in 20km race.

Matthew Garrett: Anatomy of a Fedora 17 ISO image

Wed, 02 May 2012 15:46:47 +0000

My spare time has been limited lately, so progress on producing Mac-bootable Fedora install images has been a bit slow. Thankfully it looks like everything's gong to land in time for Fedora 17[1] so hurrah for that - all that's missing right now are the last couple of patches that make sure the boot picker displays useful labels on the drives.

So how is this accomplished? Here's a hex dump of the first few K of the image.

00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00  |ER..............|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  33 ed fa 8e d5 bc 00 7c  fb fc 66 31 db 66 31 c9  |3......|..f1.f1.|
00000030  66 53 66 51 06 57 8e dd  8e c5 52 be 00 7c bf 00  |fSfQ.W....R..|..|
00000040  06 b9 00 01 f3 a5 ea 4b  06 00 00 52 b4 41 bb aa  |.......K...R.A..|
00000050  55 31 c9 30 f6 f9 cd 13  72 16 81 fb 55 aa 75 10  |U1.0....r...U.u.|
00000060  83 e1 01 74 0b 66 c7 06  f1 06 b4 42 eb 15 eb 00  |...t.f.....B....|
00000070  5a 51 b4 08 cd 13 83 e1  3f 5b 51 0f b6 c6 40 50  |ZQ......?[Q...@P|
00000080  f7 e1 53 52 50 bb 00 7c  b9 04 00 66 a1 b0 07 e8  |..SRP..|...f....|
00000090  44 00 0f 82 80 00 66 40  80 c7 02 e2 f2 66 81 3e  |D.....f@.....f.>|
000000a0  40 7c fb c0 78 70 75 09  fa bc ec 7b ea 44 7c 00  |@|..xpu....{.D|.|
000000b0  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e  |....isolinux.bin|
000000c0  20 6d 69 73 73 69 6e 67  20 6f 72 20 63 6f 72 72  | missing or corr|
000000d0  75 70 74 2e 0d 0a 66 60  66 31 d2 66 03 06 f8 7b  |upt...f`f1.f...{|
000000e0  66 13 16 fc 7b 66 52 66  50 06 53 6a 01 6a 10 89  |f...{fRfP.Sj.j..|
000000f0  e6 66 f7 36 e8 7b c0 e4  06 88 e1 88 c5 92 f6 36  |.f.6.{.........6|
00000100  ee 7b 88 c6 08 e1 41 b8  01 02 8a 16 f2 7b cd 13  |.{....A......{..|
00000110  8d 64 10 66 61 c3 e8 1e  00 4f 70 65 72 61 74 69  |.d.fa....Operati|
00000120  6e 67 20 73 79 73 74 65  6d 20 6c 6f 61 64 20 65  |ng system load e|
00000130  72 72 6f 72 2e 0d 0a 5e  ac b4 0e 8a 3e 62 04 b3  |rror...^....>b..|
00000140  07 cd 10 3c 0a 75 f1 cd  18 f4 eb fd 00 00 00 00  |...<.u..........|
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

This is the boot sector. One of the fun things about ISO9660 is that it doesn't have a boot sector in the usual x86 sense - firmwares are expected to read enough of the filesystem that they can find the bootloader in it and run it directly. x86 isn't really smart enough to manage that, so it uses something called El Torito[2]. In any case, this is usually empty space on an x86 CD. But with an increasing number of systems shipping without optical drives, real CD installs are decreasing in popularity. We use a piece of software called isohybrid that allows the building of ISO9660 images that can be written directly onto a USB stick. Asking a machine to boot off them will execute the boot sector located here, which then loads a real bootloader that's capable of reading ISO and booting the OS. It'll just be ignored when it's on a real CD.

There's one difference here, though. The isohybrid bootsector has been modified so that the first 32 bytes are just noops, and this has been inserted:

00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00  |ER..............|

45 52 ("ER") indicates the presence of an Apple partition map. This will become important later. 0800 indicates that it's using 2048-byte sectors. The 9090 is a lie to convince the firmware that it's not a zero-sized disk without representing dangerous code. We need an Apple partition map because some older Macs won't boot off CD unless the CD has one. But it's sitting right at the start of the boot sector, and this code is going to be executed. Thankfully, the entire Apple header decodes to

00000000  45                inc bp
00000001  52                push dx
00000002  0800              or [bx+si],al
00000004  0000              add [bx+si],al
00000006  90                nop
00000007  90                nop
00000008  0000              add [bx+si],al
0000000A  0000              add [bx+si],al
0000000C  0000              add [bx+si],al
0000000E  0000              add [bx+si],al
00000010  0000              add [bx+si],al
00000012  0000              add [bx+si],al
00000014  0000              add [bx+si],al
00000016  0000              add [bx+si],al
00000018  0000              add [bx+si],al
0000001A  0000              add [bx+si],al
0000001C  0000              add [bx+si],al
0000001E  0000              add [bx+si],al

which is completely harmless - we can execute all these instructions without any interesting changes in state. They'll all be undone by the rest of the boot sector.

000001b0  14 05 00 00 00 00 00 00  4c 1e ed 76 00 00 80 00  |........L..v....|
000001c0  01 00 00 3f a0 89 00 00  00 00 00 50 14 00 00 fe  |...?.......P....|
000001d0  ff ff ef fe ff ff a4 00  00 00 70 04 00 00 00 fe  |..........p.....|
000001e0  ff ff 00 fe ff ff 44 05  00 00 c0 08 00 00 00 00  |......D.........|
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|

This is the MBR partition table. The aim here was to produce media that would boot via BIOS just as well as it does via EFI, and it turns out that there are some systems that refuse to BIOS-boot off GPT media. So we need an MBR partition map. The first entry covers the entire disk and is of type 0. This is important, because some EFI implementations are strict about MBR parsing - if there's an MBR with overlapping partitions, the entire MBR will be ignored. Inconvenient. Fortunately, we've got the source code to this code, and it turns out that partitions of type 0 are ignored when performing this check. So, there's a partition of type 0.

Why does it cover the entire disk? Once we've booted the OS, we need to be able to read the ISO image contained on the USB stick. That means the kernel needs to be able to mount it, which means we need a partition entry that covers the entire disk. Linux is perfectly happy mounting a filesystem from a partition of type 0.

So, the next partition. This is an EFI boot partition that points at the embedded VFAT El-Torito image. Some firmware will decide that we've got an MBR and so will only boot off a partition that exists in it. So, here's an MBR partition to keep them happy.

The final partition covers the embedded HFS+ image. It's there purely for convenience - it means we can access it under Linux in order to update its contents for testing purposes.

00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00  |EFI PART....\...|
00000210  09 27 93 7e 00 00 00 00  01 00 00 00 00 00 00 00  |.'.~............|
00000220  fe 4f 14 00 00 00 00 00  30 00 00 00 00 00 00 00  |.O......0.......|
00000230  de 4f 14 00 00 00 00 00  2a 26 0f 37 23 c9 7f 4f  |.O......*&.7#..O|
00000240  90 df 5e fe 0a 60 1c b0  10 00 00 00 00 00 00 00  |..^..`..........|
00000250  80 00 00 00 80 00 00 00  ec 70 63 c1 00 00 00 00  |.........pc.....|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

And here we have the GPT header. Nothing complicated here, other than it indicates that the first partition descriptor is at LBA 0x10, ie byte 0x2000 - that's further into the disk than normal. That leaves us enough space to fit the Apple partition entries.

Why have a GPT at all? The EFI spec says that machines should boot fine from an MBR partition. Sadly, not all seem to. It also lets us represent the HFS+ partition in a way that makes the Mac firmware happy.

00000800  50 4d 00 00 00 00 00 03  00 00 00 01 00 00 00 10  |PM..............|
00000810  41 70 70 6c 65 00 00 00  00 00 00 00 00 00 00 00  |Apple...........|
00000820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000830  41 70 70 6c 65 5f 70 61  72 74 69 74 69 6f 6e 5f  |Apple_partition_|
00000840  6d 61 70 00 00 00 00 00  00 00 00 00 00 00 00 00  |map.............|
00000850  00 00 00 00 00 00 00 0a  00 00 00 03 00 00 00 00  |................|
00000860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

2K into the disk, here's an Apple partition map. It's 2K into the disk because we set the sector size to 2K in the partition map header. Why did we do that? Because the default is 512 bytes, and if we'd put the partition map at 512 bytes it would have been on top of the GPT header. Why 2K rather than 1K? A couple of reasons. First, the Apple partition map is only used when we boot off an actual CD, and the CD sector size is genuinely 2K. Secondly, because CDs have a sector size of 2K, 2K is a value that's actually been tested in the real world. It's nice to avoid tempting fate.

00001000  50 4d 00 00 00 00 00 03  00 00 00 29 00 00 04 70  |PM.........)...p|
00001010  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00  |EFI.............|
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001030  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00  |Apple_HFS.......|
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001050  00 00 00 00 00 00 04 70  00 00 00 33 00 00 00 00  |.......p...3....|
00001060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001800  50 4d 00 00 00 00 00 03  00 00 01 51 00 00 08 c0  |PM.........Q....|
00001810  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00  |EFI.............|
00001820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001830  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00  |Apple_HFS.......|
00001840  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001850  00 00 00 00 00 00 08 c0  00 00 00 33 00 00 00 00  |...........3....|
00001860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

The Apple partition map entries. One partition covers the entire disk, the other the embedded HFS+ filesystem. Why do this? Because not all Macs understand EFI El Torito, so won't EFI boot a CD unless there's an Apple partition map. Why have an HFS+ partition at all? Because the same Macs won't boot off FAT.

00002000  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7  |......3D..h..&..|
00002010  d0 84 a5 d4 aa 89 e0 40  81 f9 fc e4 04 e9 c2 99  |.......@........|
00002020  00 00 00 00 00 00 00 00  10 4b 14 00 00 00 00 00  |.........K......|
00002030  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69  |........ISOHybri|
00002040  64 20 49 53 4f 00 49 53  4f 48 79 62 72 69 64 00  |d ISO.ISOHybrid.|
00002050  41 70 70 6c 00 00 00 00  00 00 00 00 00 00 00 00  |Appl............|
00002060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002080  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7  |......3D..h..&..|
00002090  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71  |h..'r..L.....l.q|
000020a0  a4 00 00 00 00 00 00 00  13 05 00 00 00 00 00 00  |................|
000020b0  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69  |........ISOHybri|
000020c0  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00  |d.Apple.Appl....|
000020d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002100  00 53 46 48 00 00 aa 11  aa 11 00 30 65 43 ec ac  |.SFH.......0eC..|
00002110  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71  |h..'r..L.....l.q|
00002120  44 05 00 00 00 00 00 00  03 0e 00 00 00 00 00 00  |D...............|
00002130  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69  |........ISOHybri|
00002140  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00  |d.Apple.Appl....|
00002150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Three GPT entries - one covers the entire CD, one covers the embedded FAT filesystem, one covers the embedded EFI filesystem. That ensures that they're all accessible to the firmware.

00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20  |.CD001..LINUX   |
00008010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20  |                |
00008020  20 20 20 20 20 20 20 20  46 65 64 6f 72 61 2d 4c  |        Fedora-L|
00008030  69 76 65 43 44 20 20 20  20 20 20 20 20 20 20 20  |iveCD           |
00008040  20 20 20 20 20 20 20 20  00 00 00 00 00 00 00 00  |        ........|
00008050  c4 12 05 00 00 05 12 c4  00 00 00 00 00 00 00 00  |................|
00008060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00008070  00 00 00 00 00 00 00 00  01 00 00 01 01 00 00 01  |................|
00008080  00 08 08 00 40 00 00 00  00 00 00 40 15 00 00 00  |....@......@....|
00008090  00 00 00 00 00 00 00 17  00 00 00 00 22 00 1d 00  |............"...|
000080a0  00 00 00 00 00 1d 00 08  00 00 00 00 08 00 70 05  |..............p.|
000080b0  01 09 30 36 f0 02 00 00  01 00 00 01 01 00 20 20  |..06..........  |
000080c0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20  |                |

The ISO9660 superblock. This is completely standard. From here on there's nothing terribly surprising - the CD filesystem itself is entirely normal. The only slight oddity is that we have three embedded El Torito images. The first of these is a BIOS-bootable disk image. That'll be executed whenever the CD is put in a non-EFI machine. The second is a VFAT EFI boot image. That's used for the majority of EFI systems. The third is an HFS+ EFI boot image. There's no strict requirement for the HFS+ one to be an El Torito image - CD booting on Macs is already handled by the Apple partition map. Storing it as an El Torito is purely for convenience, since it guarantees correct alignment and avoids us having to parse the entire ISO9660 image to find its location when generating the partition maps.

In summary: Three partition maps, three bootable images, support for BIOS, UEFI and Mac platforms, works whether it's burned to a CD or written directly to a USB stick. I'm pretty pleased with that.

[1] Other than an awkward bug where something goes horribly wrong during Radeon graphics init, so Radeon-based Macs aren't working too well at the moment. We're working on it.

[2] The world decided that writing a real ISO9660 driver for BIOS was hard, so came up with El Torito. El Torito is a spec for embedding disk images inside ISO9660. There's a pointer to the disk image in the Cd header, so the BIOS simply reads a few blocks off CD, finds that pointer and then sets up a bunch of disk i/o interrupts to point at the embedded filesystem rather than a real disk. From that point on, the El Torito image simply behaves like a floppy or hard drive. This was a reasonable compromise given how resource limited older BIOS implementations were.

For reasons that aren't entirely clear, UEFI doesn't mandate that implementations support ISO9660. So even though our firmware is now sufficiently capable that the only thing standing between it and emacs is someone being sufficiently bored, we still use El Torito. Fortunately the spec allows multiple El Torito images to be embedded, so we can include one that's bootable by BIOS systems and another that's bootable by UEFI systems.

comments

Michael Kerrisk (manpages): man-pages-3.40 is released

Wed, 02 May 2012 05:52:00 +0000

I've released man-pages-3.40. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

The most notable changes in man-pages-3.40 are as follows:

A new process_vm_readv(2) page (created by Mike Frysinger based on material from Christopher Yeoh, with additions by me) documents the process_vm_readv() and process_vm_writev() system calls.
The prctl(2) page adds documentation of the PR_TASK_PERF_EVENTS_DISABLE and PR_TASK_PERF_EVENTS_ENABLE operations.
A new mcheck(3) page (by me) documents the mcheck(), mcheck_check_all(), mcheck_pedantic(), and mprobe() functions.
The rcmd(3) page adds documentation of the rcmd_af(), rresvport_af(), iruserok_af(), and ruserok_af() functions.
The rexec(3) page adds documentation of the rexec_af() function.

James Morris: Kernel Security Talk at LinuxCon Japan

Wed, 02 May 2012 01:57:49 +0000

Just to let folk know — I’ll be giving a talk on the state of Linux kernel security development at LinuxCon Japan in Yokohama on June 8th. From the abstract:

In this talk, we’ll examine the current state of the Linux kernel security subsystem. Starting with a brief overview of existing features, we’ll discuss recent developments, current efforts and future directions. We’ll also discuss the evolving threat landscape, and the increasing need for mobile and cloud security. This will be a high-level technical discussion aimed at IT professionals. A good general knowledge of operating system and computer security concepts will be advantageous.

I’ll also likely be in Tokyo briefly — if any kernel security development folk there want to meet up, let me know.

Andi Kleen: epoll and garbage collection

Tue, 01 May 2012 13:01:34 +0000

epoll, xor lists, pointer compression. What do they have in common? They all don’t play well with automatic garbage collection.

I recently wrote my first program using epoll. And I realized that epoll does not play well with garbage collection. It is one of the few (only?) system calls where the kernel saves a pointer for user space. It supports stuffing other data in there too, like the fd, but for anything non trivial you typically need some state per connection, thus a pointer.

A garbage collector relies on identifying all pointers to an object, so that it can decide whether the object is still used or not.

Now normally user programs should have some other reference to implement time outs and similar. But if they don’t and the only reference is stored in the kernel and the garbage collector comes in at the wrong time the connection object will be garbage collected, even though the connection is still active. This is because the garbage collector cannot see into the kernel.

Something to watch out for.

LPC 2012: Deadline for referee-track submissions extended to May 15

Mon, 30 Apr 2012 19:57:53 +0000

Due to popular demand, the Linux Plumbers Conference is extending both the deadline for submissions to the refereed track and the early registration deadline to Tuesday, May 15th. We have quite a few interesting submissions (and attendees!), but we would very much like to see your submission and also to see you in San Diego this coming August!

Instructions for submitting a refereed-track proposal are here.

Instructions for submitting a microconference proposal are here.

Dave Jones: Weekly Fedora kernel bug statistics – April 27 2012

Fri, 27 Apr 2012 18:17:54 +0000

	15	16	17	rawhide
Open:	308	562	71	147	(1088)
Opened since 2012-04-20	0	44	17	3	(64)
Closed since 2012-04-20	3	55	11	1	(70)
Changed since 2012-04-20	10	69	18	9	(106)

Weekly Fedora kernel bug statistics – April 27 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – April 13 2012 15 16 17 rawhide Open: 313 549 63...
Weekly Fedora kernel bug statistics – April 20 2012 15 16 17 rawhide Open: 308 544 64...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Pavel Machek: mbank.cz: bank that gets ssl wrong

Thu, 26 Apr 2012 09:13:16 +0000

Just try it. mbank.cz is widely advertised as mbank.cz... Actually I did not find any other URL in the documents I got from them. Now go to https://mbank.cz/, and what you get is:

This is probably not the site you are looking for!
You attempted to reach www.mbank.cz, but instead you actually reached a server identifying itself as ssl.mbank.com.pl. This may be caused by a misconfiguration on the server or by something more serious. An attacker on your network could be trying to get you to visit a fake (and potentially harmful) version of www.mbank.cz. You should not proceed.

Right url to reach mbank is probably https://cz.mbank.eu/. Oh, site certificate is on "BRE bank", but as far as I can tell, it is right bank. Trust me on it. You have no other way to verify it.

Eric Sandeen: Comparing PVWatts Estimates to Actual Solar Production

Thu, 26 Apr 2012 01:43:40 +0000

Click to enlarge

PVWatts is a website run by the National Renewable Energy Lab (NREL) which can be used to estimate how much energy a given solar array might produce, based on historical mesurements of insolation. If you ever have a site analysis done, the installer will probably use it to estimate your potential for annual production, and you might wonder how close the the mark it’ll be. Now that I’ve had solar PV for almost 2 years, it’s interesting to look at how close the estimates were for my situation.

Once you choose your location, you input a few other bits of information such as array size (in kW), orientation (tilt & azimuth), and a derate factor (more on that later). If it’ll be a tracking array rather than a fixed array, you give it that information as well. I have 11 230W panels, due south, flush mounted to a roof pitched at about 33 degrees. So, I entered:

DC Rating (kW): 2.53
DC to AC Derate Factor: 0.817
Array Type: Fixed Tilt
Array Tilt (degrees): 33
Array Azimuth (degrees): 180

There are a couple of items to note. First, the DC rating. Although the nameplate wattage of all my panels adds up to 2530W, the Enphase M190 microinverter under each one has a max output of 190W (in practice, it gets up to 199W). So even on a perfect day, the array maxes out at about 2190W:

Click to enlarge

However, this doesn’t happen very often, and when not “clipping” like this, the modules are performing as 230w panels would with any other inverter, so I left it at the nameplate value.

Second, the Derate Factor. This is essentially a measure of the aggregate efficiency of DC to AC conversion. The default in PVWatts is 0.77, but Enphase recommends a slightly higher value of 0.817 due to stated efficiencies of their system.

If you look at the graph above, some months outperform, others underperform. In the first full year, I produced 3209 kWh, and PVWatts estimated 3448 – so I produced about 93% of the estimate.

There are a couple of reasons for this, I think, which are related. PVWatts assumes no shading (although it can be incorporated into the Derate). In reality, I do have some trees and some snow. If you look at the winter months, I fairly drastically underperform the estimates, and I think snow is the likely culprit here, despite my best snow-clearing efforts.

And year round, I do have just a little shade from my neighbor’s trees – this was known at the time of install, because my installer used a Solar Pathfinder [amzn] (see here for how it works, a very cool tool) for the site survey, and it put efficacy right at about 93% after accounting for shading.

So overall, I’d say that if a site assessment is done properly, you should absolutely be able to trust the estimates, based on my experience.

Gustavo F. Padovan: BlueZ on GSoC: Accepted students announced

Wed, 25 Apr 2012 03:55:25 +0000

Yesterday Google released the accepted students for this year’s Google Summer of Code and BlueZ will be participating with 4 students:

Project: Bluetooth Replayer
Student: Anton Weber
Mentor: Anderson Lizardo

Project: OBEX Filesystem In Userspace
Student: Michał Poczwardowski
Mentor: Vinicius Gomes

Project: Implement AVRCP 1.3 Controller Role
Student: Rafael Fonseca
Mentor: Luiz Augusto von Dentz

Project: Visualization of Bluetooth traffic
Student: Thiago da Silva Arruda
Mentor: Gustavo Padovan

It is now community bonding time, where students get know their mentors and the community. We wish a great summer to all students. :-)

Eric Sandeen: Linux Filesystems LOC Update

Tue, 24 Apr 2012 04:26:04 +0000

I did a post last year about how lines of code for some of the primary Linux filesystems were evolving. It seemed to spark people’s interest, so here’s an update through v3.4-rc4:

(Click to see full size).

I added regression lines this time – btrfs should catch xfs around kernel version 3.8 or so.

Again, I draw no strong conclusions from this – I just think it’s interesting to see how things are moving as these filesystems evolve.

I used CLOC to count lines; the numbers on the graph reflect no blank lines & no comments. For ext3 & ext4 I included jbd[2] and some of the peripheral files they use; code snippet from the script:

ext3lines=`cloc fs/ext3/*.[ch] include/linux/ext3*.h  fs/jbd/*.[ch] \
           include/linux/jbd.h fs/mbcache.c include/linux/mbcache.h \
           | grep SUM | awk '{print $5}'`;
ext4lines=`cloc fs/ext4/*.[ch] include/linux/ext4*.h  fs/jbd2/*.[ch] \
           include/linux/jbd2.h fs/mbcache.c include/linux/mbcache.h \
           | grep SUM | awk '{print $5}'`;
xfslines=`cloc fs/xfs/*.[ch] fs/xfs/*/*.[ch] include/linux/dqblk_xfs.h \
           | grep SUM | awk '{print $5}'`;
btrfslines=`cloc fs/btrfs/*.[ch] | grep SUM | awk '{print $5}'`;
gfs2lines=`cloc fs/gfs2/*.[ch] include/linux/gfs2* \
           | grep SUM | awk '{print $5}'`;

(Some of the files above no longer exist, but I used the same script to iterate over all the kernel versions.)

Gustavo F. Padovan: I’m joining Collabora

Mon, 23 Apr 2012 20:14:33 +0000

After more than 3 years working at ProFUSION embedded systems I decided it was time to a move: Today is my first day (of many) at Collabora Ltd. :-)

I would like to take the opportunity to thank people at ProFUSION for the time I’ve working there. Those were good times.

And for the Collaborans: I hope we will rock a lot together. I’ll keep updating this blog with posts about my work at Collabora.

See you around. ;-)

Michael Kerrisk (manpages): man-pages-3.39 is released

Mon, 23 Apr 2012 19:33:29 +0000

I've released man-pages-3.39. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

The most notable changes in man-pages-3.39 are as follows:

A new malloc_trim(3) page (by me) documents the malloc_trim() function.
A new malloc_usable_size(3) page (by me) documents the malloc_usable_size() function.
The prctl(2) page adds documentation of the PR_SET_MM operation, thanks to Cyrill Gorcunov
Rewrites of a large part of nsswitch.conf(5) to clarify the text and bring it up to date, thanks to Mark Bannister

LPC 2012: 2012 LPC Call for Proposals – Take 2

Mon, 23 Apr 2012 18:59:45 +0000

Since posting the call for proposals last month, we’ve received several complaints about the submission process. Mostly the issues are that the submission form is too complex, and a few people don’t want to use Launchpad. We (the planning committee) have heard your concerns, and I apologize for any inconvenience.

Therefore, proposals may also be submitted via email to submissions@linuxplumbersconf.org, and there is no requirement to have or use a Launchpad account.

Include in your email:

Session title
Name of microconference (if applicable)
Abstract. Should be short (a couple of paragraphs), but clearly describe the problem, the affected areas of the Linux plumbing, and the intended audience.
(optional) Required attendees – list people critical to the discussion and who should be in the room
A one-paragraph bio, describing your experience.

Deadline for submission is 11:59 PDT on May 1, 2012. If you’ve already submitted a proposal via Launchpad, then do not fear. We have your proposal and there is no need to resubmit.

I’d also like to take this opportunity to say we’re still accepting microconference topic proposals. Submit a proposal by going to the wiki and adding a page for the new topic.

Dave Jones: Weekly Fedora kernel bug statistics – April 20 2012

Fri, 20 Apr 2012 16:41:17 +0000

	15	16	17	rawhide
Open:	308	544	64	146	(1062)
Opened since 2012-04-13	1	48	12	3	(64)
Closed since 2012-04-13	7	58	11	0	(76)
Changed since 2012-04-13	13	66	15	8	(102)

Some focus this week on closing out a bunch of old bugs, especially those suspected to have been caused by the now fixed i915 memory corruption bug.

Weekly Fedora kernel bug statistics – April 20 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – April 13 2012 15 16 17 rawhide Open: 313 549 63...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...

Related posts brought to you by Yet Another Related Posts Plugin.

Andi Kleen: The weekend error anomaly

Fri, 20 Apr 2012 03:11:02 +0000

I run mcelog.org which describes the standard way in Linux to handle machine check errors. Most of the hits of the website are just people typing a error log into a search engine mcelog.org ranks quite high on Linux machine check related terms.

The log files give me some indication how many errors are occurring on Linux systems in the field. Most of the errors are corrected memory errors on systems with ECC memory: a bit flipped, but the ECC code corrected it and and no actual data corruption occurred. (In this sense they are not actually errors, a more correct term would be “events”). Other errors like network errors or disk errors are not logged by mcelog.

I noticed is that there seem to be less memory errors on weekends. Normally the distribution of hits is fairly even over the week. But on Saturday and Sunday it drops into half.

It’s interesting to speculate why this this weekend anomaly happens.

ECC memory is normally only on server systems, which should be running 24h. In principle errors should be evenly distributed over the whole week.

Typing the error into google is no automated procedure. A human has to read the log files and do it manually.

If people are more likely to do this on work days one would expect that they would catch up on the errors from the weekend on Monday. So Monday should have more hits. But that’s not in the data: Monday is not different from other weekdays.

It’s also sticky for each system (or rather each human googling). Presumably the person will google the error only once no matter how many errors their system have and after that “cache” the knowledge what the error means. So the mcelog.org hits are more a indication of “first memory error in the career of a system administrator” (assuming the admin has perfect memory, which may be a bold assumption). But given a large enough supply of new sysadmins this should be still a reasonable indication of the true number of errors (at least on the systems likely to be handled by rookie administrators)

The hour distribution is more even, with 9-10 slightly higher. Not sure which time zone and what that means on the geographical distribution of errors and rookie admins.

One way to explain the weekend anomaly could be that that servers may be more busy on weekdays and they may have more errors when they are busy. Are these two assumptions true? I don’t know. It would be interesting to know if this shows up in other peoples large scale error collections too.

I wonder if it’s possible to detect solar flares in these logs. Need to find a good data source for them. Are there any other events generating lots of radiation that may affect servers? I hope there will never be a nuke or a super nova blast in the data.

Pete Zaitcev: Hard numbers from China

Fri, 13 Apr 2012 16:40:01 +0000

It looks like almost every cloud provider hide their numbers, which makes guidance and education unnecesserily difficult. To be fair, I think I saw AWS posting a few items for S3, but forgot to save it. So, I'm going to preserve what one Chinese gentleman posted to OpenStack list, in the context of Swift performance issues:

Our practice of  Sina Web Service Team https://launchpad.net/~sws:

total accounts:          121,961;
total containers:        160,703;
total objects:        14,291,519;
total storage usage:   1.3T

account replication time:      10 hours;
container replication time:    10 hours;
object replication time:       48 hours;
account audit time:             2 hours;
container audit time:           9 hours;
container update time:         19 hours;

Unfortunately he omitted the requests per second and gigabytes per second that the cluster is sustaining from the users, but it's very interesting anyway.

QUICKIE: Apparently I meant the official post about Amazon S3:

total objects:   905,000,000,000
total bytes:     ?
requests/s:           650,000

Dave Jones: Weekly Fedora kernel bug statistics – April 13 2012

Fri, 13 Apr 2012 16:22:56 +0000

	15	16	17	rawhide
Open:	313	549	63	141	(1066)
Opened since 2012-04-06	1	33	11	2
Closed since 2012-04-06	33	43	7	1
Changed since 2012-04-06	48	67	19	5

Slowly putting a dent in the f15/f16 backlog, now that we have 3.3.1 updates available for both with the i915 memory corruption fix. As can be seen from the ‘closed’ links above, lots of the “weird shit happened” bugs are now closed out. Still a huge number of outstanding bugs though.

Note for Fedora 15 users: Kernel updates are languishing so long in updates-testing that they keep getting obsoleted by new builds before the old one recieves enough Karma.

Weekly Fedora kernel bug statistics – April 13 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Michael Kerrisk (manpages): man-pages-3.34 is released

Thu, 12 Apr 2012 19:42:28 +0000

I've released man-pages-3.34. Again, because of the problems at kernel.org, the various pieces of the release are at different locations than usual (and possibly some of these locations may become permanent). The browsable online pages can be found on man7.org; the release tarball is available ~~here~~ on kernel.org.

A git repository for man-pages can be found on ~~github~~ kernel.org.

The most notable changes in man-pages-3.34 are as follows:

A new rt_sigqueueinfo(2) documents the rt_sigqueueinfo() and rt_tgsigqueueinfo() system calls. At the same time, the sigqueue() man page was moved from Section 2 to Section 3, since the interface it documents really is a library function (layered on top of rt_sigqueueinfo()).
New man pages for the cciss(4) and hpsa(4) drivers. Thanks to Stephen Cameron.
The fstatat(2) man page adds documentation of the AT_NO_AUTOMOUNT flag added in Linux 2.6.38.
The lseek(2) man page adds documentation of the SEEK_HOLE and SEEK_DATA commands scheduled for the upcoming Linux 3.1. (These commands allow a program to map file holes.)
The madvise(2) man page adds documentation of the MADV_HUGEPAGE and MADV_NOHUGEPAGE commands added in Linux 2.6.38.
The feature_test_macros(7) man page adds documentation of the _ISOC95_SOURCE macro added in glibc 2.12

Michael Kerrisk (manpages): man-pages-3.33 is released

Thu, 12 Apr 2012 19:42:08 +0000

After a long gap, I've released man-pages-3.33. Given the ongoing problems at kernel.org, the various pieces of the release are at different locations than usual (and possibly some of these locations may become permanent). The browsable online pages can be found on man7.org; the release tarball is available ~~here~~ on kernel.org.

A git repository for man-pages can be found on ~~github~~ kernel.org.

The most notable changes in man-pages-3.33 are as follows:

The sync(2) page adds documentation of the syncfs() system call that was added in Linux 2.6.39.
The syscalls(2) has been updated to be current as at Linux 3.1.
The syslog(2) page adds documentation of the CAP_SYSLOG capability added in Linux 2.6.37.

In addition, various fixes were made in many other pages.

Michael Kerrisk (manpages): man-pages-3.35 is released

Thu, 12 Apr 2012 19:41:14 +0000

I've released man-pages-3.35. For the moment, the various pieces of the release remain at different locations than usual. The browsable online pages can be found on man7.org; the release tarball is available ~~here~~ on kernel.org.

A git repository for man-pages can be found on ~~github~~ kernel.org.

The most notable changes in man-pages-3.35 are as follows:

A new recvmmsg(2) page (written by Andi Kleen and me) documents the recvmmsg() system call that was added to Linux back in version 2.6.32.
A new setns(2) page (written by Eric Biederman) documents the setns() system call that was added in Linux 3.0.
Various global edits to improve the consistency of terminology used in the man pages.

James Morris: 2012 Linux Security Summit (San Diego) – Call for Particpation

Thu, 12 Apr 2012 14:00:09 +0000

The 2012 Linux Security Summit (LSS) has been announced. The CFP is open from now until the 23rd of May.

This year, the summit will be a two-day event, co-located with LinuxCon, Linux Plumbers, and the Kernel Summit. We’re planning on holding developer break-out sessions for much of the second day, and extending the length of the main talks to the more traditional 45 minute + 15 minute break format. There will still be shorter 30 minute talks, and roundtable discussions.

Check out the programs from previous years to see what kind of proposals have been previously accepted:

LSS 2011, Santa Rosa
LSS 2010, Boston

Send your proposals to the program committee per the announcement.

Michael Kerrisk (manpages): man-pages-3.38 is released

Wed, 11 Apr 2012 19:32:45 +0000

I've released man-pages-3.38. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

The most notable changes in man-pages-3.38 are as follows:

A new get_nprocs_conf(3) page by Petr Beňas documents the get_nprocs_conf() and get_nprocs() functions.
A new mallopt(3) page (by me) documents the mallopt() library function and corresponding environment variables used for controlling the behavior of the malloc(3) suite of functions.
A new malloc_get_state(3) page (by me) documents the malloc_get_state() and malloc_set_state() functions.
The mtrace(3) page has been completely rewritten (by me), adding a lot of detail.
A new scandirat(3) page by Mark Bannister documents the scandirat() function.
Mark Bannister also added documentation of the qsort_r() function to the qsort(3) page.
I've added documentation of the aligned_alloc() function (specified in the recently released C11 standard) and the obsolete pvalloc() function to the posix_memalign(3)
The malloc(3) page adds a short discussion of allocation arenas.

(Updated 2012-04-03: fix some broken links, and replace erroneous mention of "mcheck()" with "mtrace()".)

Harald Welte: Name that UART: April 2012

Mon, 09 Apr 2012 02:00:00 +0000

It's sort of a cheap knock-off idea stolen from the Name that Ware on bunnies blog: I'm going to post one picture every month about a UART that I found on embedded hardware. Unfortunately I don't have much to offer in terms of a reward for whoever finds the true solution ;)

In any case, every month there are devices that I'm looking into either out of my own interest, or because the work at gpl-violations.org requires it. In most of them, you can find a UART to get to the u-boot / Linux serial console.

So here is the device that I just took apart earlier today:

The location of the UART pads was obvious, after looking at the PCB for a very short time. The entire unpopulated U1 footprint appeared suspiciously like a UART level shifter for true RS232 voltage levels:

You can see two signals going directly to a small unpopualted3-pin header
There are two other signals coming from somewhere under the main SoC
There are capacitors (C440, C441) directly connected to the U1 for the charge pump

Harald Welte: Prototype smart card chips in DIL-40 case have arrived

Mon, 09 Apr 2012 02:00:00 +0000

Finally, the first samples of the smart card chip (for the Osmocom CardOS project) have arrived. As opposed to the final smart cards, this one has been packaged in a DIL case instead of the usual thin credit-card sized plastic. The reason for this is quite simple: This way lots of I/O pins for debugging as well as JTAG can be accessible during COS development.

Here you can see the first incarnation of a veroboard connected to an adapter pcb inside an Omnikey smart card reader:

After confirming it worked, I soldered the wires directly to the adapter PCB, as can be seen here:

There is already a real PCB design that is currently manufactured, i.e. in a week or so there will be a picture of a clean, professionally-produced/etched PCB with all of the prototype pins exported.

In terms of the COS, I haven't done much more work than compared to the last posting, mainly due to a large number of other projects. But we will get there...

Eric Sandeen: Opower goes social

Fri, 06 Apr 2012 17:32:35 +0000

I wrote about Opower almost 2 years ago, a company which is working to reduce energy use by simply making people more aware of what they use in comparison with others. At the time of that post, they seemed mostly focused on working directly with utilities, and sending out reports to customers showing them how their energy use compared to similar homes in their area. The idea seems to be that if you are at all conservation & efficiency minded, seeing where you are in relation to others may actually encourage you to do even better. They track the results, and apparently, it works. However, that method works only if the utility is on board to provide the data.

It took a while, but Opower has now launched their “social” site at http://social.opower.com, in conjunction with Facebook and the NRDC. This site encourages people who know each other to engage in a little friendly utility-bill competition. If your utility is one (of the few) which is already engaged with Opower, it’ll pull in data automatically. If not, you can enter data yourself. (So far, it only tracks electricity use, but natural gas is planned). You can also create groups of friends such as college alums, coworkers, neighborhoods, etc. The site is still labeled “Beta” but it seems like a good idea to me. My journey to a lower power bill started with actually paying attention to what I was currently using, and this looks like a fun way to encourage more of that!

For the friend comparison, it requires a Facebook account, but even without that you can compare your home’s use to aggregate data. If you decide to give it a shot, send me a request and we can go head to head next month!

Dave Jones: Weekly Fedora kernel bug statistics – March 30 2012

Fri, 30 Mar 2012 18:09:02 +0000

	15	16	17	rawhide
Open:	344	526	53	142	(1065)
Opened since 2012-03-23	15	149	21	5
Closed since 2012-03-23	4	76	8	5
Changed since 2012-03-23	20	172	27	11

Weekly Fedora kernel bug statistics – March 30 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 23 2012 15 16 17 rawhide Open: 346 505 49...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...

Related posts brought to you by Yet Another Related Posts Plugin.

Dave Jones: resolution on the i915/hibernate memory corruption bug

Fri, 30 Mar 2012 17:03:48 +0000

It seems Dave Airlie managed to track down and fix the bug that has been plagueing hibernate/i915 users for over a year.
My recent posts here showed some reports of memory corruption where we occasionally saw a strip of eight pixels. Sometimes 0×00000000, and sometimes 0x00aaaaaa.
In hindsight, it seems obvious. It’s a framebuffer cursor.

https://bugzilla.kernel.org/attachment.cgi?id=72739 should fix it.

There are still a whole bunch of other hibernate bugs that need fixing, but this at least is a huge step forward.

resolution on the i915/hibernate memory corruption bug is a post from: codemonkey.org.uk

i915, hibernate, and memory corruption. While going through some of the stranger unexplained bugs last...
Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
linked list debugging After looking through all the general protection bugs yesterday, today...

Related posts brought to you by Yet Another Related Posts Plugin.

Matt Domsch: Dell Linux Engineers work over 5000 bugs with Red Hat

Fri, 30 Mar 2012 14:05:54 +0000

A post today by Dell’s Linux Engineering team announcing support for RHEL 5.8 on PowerEdge 12G servers made me stop and think. In the post, they included a link to a list of fixes and enhancements worked in preparing RHEL 5.8 for our new servers. The list was pretty short. But that list doesn’t tell the whole story.

A quick search in Bugzilla for issues which Dell has been involved in since 1999 yields 5420 bugs, 4959 of which are CLOSED, and only 380 of which are still in NEW or ASSIGNED state, many of which look like they’re pretty close to being closed as well. This is a testament to the hard work Dell puts into ensuring Linux “Just Works” on our servers, straight out of the box, with few to no extra driver disks or post-install updates needed to make your server fully functional. You want a working new 12G server? Simply grab the latest RHEL or SLES DVD image and go. Want a different flavor of Linux? Just be sure you’re running a recent upstream kernel – we push updates and fixes there regularly too.

Sure, we could make it harder for you, but why?

Congratulations to the Linux Engineering team for launching 12G PowerEdge with full support baked into Linux! Keep up the good work!

Rusty Russell: 1 Week to Go, and Rusty Goes Offline

Thu, 29 Mar 2012 06:03:26 +0000

Just as the Linux kernel merge window closes, I’m going offline. My wedding is exactly a week away, but I’ll be entertaining guests and doing final preparation. I’ll be back from our honeymoon and wading through mail on the 7 May.

Alex’s “A Bald Target” campaign to raise awareness for TimeForKids has been a huge success, even though we’re currently far short of the hair-shaving goal. She’s been on one of the local radio stations, with newspaper coverage expected this weekend; two local TV stations want to cover the actual shave if it happens. The charity is delighted with the amount of publicity they have received; given that they need local people to volunteer to mentor the disadvantaged children, that’s worth at least as much as the money.

Special thanks to a couple of people who donated direct to the charity, to avoid causing baldness! And yes, if we were starting again, having competing “shave” vs “save” campaigns would have been awesome…

Rusty Russell: Sources of Randomness for Userspace

Thu, 29 Mar 2012 05:52:43 +0000

I’ve been thinking about a new CCAN module for getting a random seed. Clearly, /dev/urandom is your friend here: on Ubuntu and other distributions it’s saved and restored across reboots, but traditionally server systems have lacked sources of entropy, so it’s worth thinking about other sources of randomness. Assume for a moment that we mix them well, so any non-randomness is irrelevant.

There are three obvious classes of randomness: things about the particular machine we’re on, things about the particular boot of the machine we’re on, and things which will vary every time we ask.

The Machine We’re On

Of course, much of this is guessable if someone has physical access to the box or knows something about the vendor or the owner, but it might be worth seeding this into /dev/urandom at install time.

On Linux, we can look in /proc/cpuinfo for some sources of machine info: for the 13 x86 machines my friends on IRC had in easy reach, we get three distinct values for cpu cores, three for siblings, two for cpu family, eight for model, six for cache size, and twelve for cpu MHz. These values are obviously somewhat correlated, but it’s a fair guess that we can get 8 bits here.

Ethernet addresses are unique, so I think it’s fair to say there’s at least another 8 bits of entropy there, though often devices have consecutive numbers if they’re from the same vendor, so this doesn’t just multiply by number of NICs.

The amount of RAM in the machine is worth another two bits, and the other kinds of devices eg. trolling /sys/devices, which can be expected to give another few bits, even in machines which have fairly standard hardware settings like laptops. Alternately, we could get this information indirectly by looking at /proc/modules.

Installed software gives a maximum three bits, since we can assume a recent version of a mainstream distribution. Package listings can also be fairly standard, but most people install some extra things so we might assume a few more bits here. Ubuntu systems ask for your name to base the system name on, so there might be a few bits there (though my laptop is predictably “rusty-x201″).

So, let’s have a guess at 8 + 7 + 2 + 3 + 3 + 2 + 2, ie. 27 bits from the machine configuration itself.

Information About This Boot

I created an upstart script to reboot (and had to hack grub.conf so it wouldn’t set the timeout to -1 for next boot), and let it loop for a day: just under 2000 times in all. I eyeballed the graphs of each stat I gathered against each other, and there didn’t seem to be any surprising correlations. /proc/uptime gives a fairly uniform range of uptime values within a range of 1 second, at least 6 bits there (every few dozen boots we get an fsck, which gives a different range of values, but the same amount of noise). /proc/loadavg is pretty constant, unfortunately. bogomips on CPU1 was fairly constant, but for the boot CPU it looks like a standard distribution within 1 bogomip, in increments of 0.01: say another 7 bits there.

So for each boot we can extract 13 bits from uptime and /proc/cpuinfo.

Things Which Change Every Time We Run

The pid of our process will change every time we’re run, even when started at boot. My pid was fairly evenly divided on every value between 1220 and 1260, so there’s five bits there. Unfortunately on both 64 and 32-bit Ubuntu, pids are restricted to 32768 by default.

We can get several more bits from simply timing the other randomness operations. Modern machines have so much going on that you can probably count on four or five bits of unpredictability over the time you gather these stats.

So another 9 bits every time our process runs, even if it’s run from a boot script or cron.

Conclusion

We can get about 50 bits of randomness without really trying too hard, which is fine for a random server on the internet facing a remote attacker without any inside knowledge, but only about five of these bits (from the process’ own timing) would be unknown to an attacker who has access to the box itself. So /dev/urandom is still very useful.

On a related note, Paul McKenney pointed me to a paper (abstract, presentation, paper) indicating that even disabling interrupts and running a few instructions gives an unpredictable value in the TSC, and inserting a usleep can make quite a good random number generator. So if you have access to a high-speed, high-precision timing method, this may itself be sufficient.

LPC 2012: Discounted Room Rate and Hotel Information for LPC 2012

Tue, 27 Mar 2012 15:38:21 +0000

We’re pleased to announce that Linux Plumbers attendees can avail of a discounted set of rooms at $170/night, plus taxes, and including free room Internet, at the conference hotel – the Sheraton San Diego Hotel & Marina. For more information, please see our ATTEND page. This discount is available via the Linux Foundation for both, Linux Plumbers and LinuxCon conferences. Note that there is a limited block of rooms, and they might fill up fast.

Greg Kroah-Hartman: openSUSE Tumbleweed status for the week of March 26, 2012

Mon, 26 Mar 2012 20:44:00 +0000

It's been about a year since I did a status report of what's going on in the openSUSE:Tumbleweed repo, let me know if you find this actually useful or not so that I can determine if I should keep it up.

As everyone knows, Tumbleweed is running on top of openSUSE:12.1, the transition to 12.1 was rocky for some people who thought that Tumbleweed was somehow a "full" distro, and not just an add-on on top of a stable openSUSE release. To make things easier for future updates of the base openSUSE release, please point to the "current" repo, not the explicitly numbers repo. For more details how to do this, see the Tumbleweed wiki page.
kernel 3.3.0 is in Tumbleweed, and seems to be working well so far.
KDE 4.8 is now in Tumbleweed, be careful if you previously had added the KDE repo manually to your system, you should now remove it as I have no idea how well it will interact with this.
Because of the KDE 4.8 update, LibreOffice was dropped from Tumbleweed. This is due to build issues with the package, not any runtime issue that I can determine. LibreOffice fails to build on Factory at the moment as well, and a bug is open about this, hopefully it gets resolved soon.
XFCE has been updated in Tumbleweed to the latest version
vim finally showed up, after a brief breakage that I caused, sorry about that, all should be good now.
To preempt any questions about a GNOME update in Tumbleweed, I am looking into it, but it will not happen until it stabilizes in Factory first.

As always, if anyone knows of any packages they wish to see added to Tumbleweed, please let me know.

Please read the wiki page for Tumbleweed if you have any basic questions about what it is or how to use it. Any other questions, please ask them on the opensuse-factory mailing list.

Greg Kroah-Hartman: Cascade Cement, March 2012

Mon, 26 Mar 2012 20:44:00 +0000

Snowboarding in Cacsade Cement, March 2012 from Greg KH on Vimeo.

LPC 2012: 2012 LPC Call For Proposals

Mon, 26 Mar 2012 16:35:29 +0000

The best presentations are not about finished work, but rather problems, proposals, or proof-of-concept solutions that require face-to-face discussions and debate among people from different areas of the Linux plumbing. Ideally, the best presentations are also working sessions that result in patches to various portions of Linux’s plumbing that make the Linux world a better place for its developers and (most important) its users.

A proposal should be short: just a couple of paragraphs describing the topic, why it is important, and what parts of the plumbing it touches.

Proposals are due at 11:59PM Pacific Time on Tuesday, May 1st, 2012. Authors will be notified by Tuesday May 15th, 2012. We look forward to seeing your proposal, and to seeing you in San Diego!

SUBMITTING A PROPOSAL

We are using the Launchpad infrastructure this year, but it is still easy to submit a proposal. Go to this URL, which will take you to a “Register a new blueprint”. However, if Launchpad doesn’t know you, it will take you to an account-creation page, in which case please create a new Launchpad account so that it does know you. Once you have a Launchpad account, the above URL will take you to the “Register a new blueprint” page.
Once you get to the “Register a new blueprint” page:

In the “Project” field, enter “lpc”.
In the “Name” field, put a short identifier for your talk. Please do -not- put your own name, as Launchpad already knows who you are.
For refereed-track presentations:
- In the “Name” field, enter a short name for your presentation starting with “lpc2012-ref-” for refereed-track presentations (for example, “lpc2012-ref-Grand-Unified-FOSS-Project”).
For microconference presentations:
- In the “Name” field, enter the code shown on that microconference’s wiki. For a few examples:
  - Containers microconference: “lpc2012-cont-”
  - Real-time microconference: “lpc2012-rt-”
  - Scaling microconference: “lpc2012-scale-”
  - Virtualization microconference: “lpc2012-virt-”
In the “Title” field, enter your proposal’s one-line title.
The “Specification URL” field can be left blank, but feel free to enter a URL pointing to additional information.
In the “Summary” field, enter:
1. Your abstract, which should be short (a couple of paragraphs), but should clearly describe the problem, the affected areas of the Linux plumbing, and the intended audience.
2. A one-paragraph bio, describing your experience.
The “Assignee”, “Drafter”, and “Approver” fields may be left blank.
Please leave the “Definition Status” in the “New” state. (Changing this field can result in your submission being lost.)

Then click the “Register Blueprint” button to submit your proposal! You will automatically be recorded as the person submitting the proposal.
If you have any difficulty submitting your abstract, please email the details to contact@linuxplumbersconf.org. For complete information on participating in LPC 2012, please visit our Participate Page.

Paul E. Mc Kenney: 2012 LPC Call For Proposals

Mon, 26 Mar 2012 16:17:05 +0000

The Planning Committee for the Linux Plumbers Conference (LPC) is pleased to announce a call for refereed-track proposals for the 2012 edition of LPC, which will be held August 29-31 in San Diego, CA, USA. Refereed-track presentations are about 45 minutes in length, and should focus on a specific aspect of the "plumbing" in the Linux system. The Linux system's plumbing includes kernel subsystems, core libraries, windowing systems, management tools, media creation/playback, and so on. Proposals are due at 11:59PM Pacific time on May 1, 2012.

The best presentations are not about finished work, but rather problems, proposals, or proof-of-concept solutions that require face-to-face discussions and debate among people from different areas of the Linux plumbing. Ideally, the best presentations are also working sessions that result in patches to various portions of Linux's plumbing that make the Linux world a better place for its developers and (most important) its users.

A proposal should be short: just a couple of paragraphs describing the topic, why it is important, and what parts of the plumbing it touches.

Proposals are due at 11:59PM Pacific Time on Tuesday, May 1st, 2012. Authors will be notified by Tuesday May 15th, 2012. We look forward to seeing your proposal, and to seeing you in San Diego!

SUBMITTING A PROPOSAL

We are using the Launchpad infrastructure this year, but it is still easy to submit a proposal. Go to this URL, which will take you to an account-creation page. However, if Launchpad doesn't know you, in which case please create a new Launchpad account so that it does know you. Once you have a Launchpad account, the above URL will take you to the "Register a new blueprint" page.
Once you get to the "Register a new blueprint" page:

In the "Project" field, enter "lpc".
In the "Name" field, put a short identifier for your talk. Please do -not- put your own name, as Launchpad already knows who you are.
For refereed-track presentations:
- In the "Name" field, enter a short name for your presentation starting with "lpc2012-ref-" for refereed-track presentations (for example, "lpc2012-ref-Grand-Unified-FOSS-Project").
For microconference presentations:
- In the "Name" field, enter the code shown on that microconference's wiki. For a few examples:
  - Containers microconference: “lpc2012-cont-”
  - Real-time microconference: “lpc2012-rt-”
  - Scaling microconference: “lpc2012-scale-”
  - Virtualization microconference: “lpc2012-virt-”
In the "Title" field, enter your proposal's one-line title.
The "Specification URL" field can be left blank, but feel free to enter a URL pointing to additional information.
In the "Summary" field, enter:
1. Your abstract, which should be short (a couple of paragraphs), but should clearly describe the problem, the affected areas of the Linux plumbing, and the intended audience.
2. A one-paragraph bio, describing your experience.
The "Assignee", "Drafter", and "Approver" fields may be left blank.
Please leave the "Definition Status" in the "New" state. (Changing this field can result in your submission being lost.)

Then click the "Register Blueprint" button to submit your proposal! You will automatically be recorded as the person submitting the proposal.
If you have any difficulty submitting your abstract, please email the details to contact@linuxplumbersconf.org.

Harald Welte: OsmoDevCon 2012 is over...

Mon, 26 Mar 2012 02:00:00 +0000

We just finished the 4th and final day of the OsmoDevCon 2012. It contained four days of in-depth presentations and discussions related to Free Software communications systems, most notably OsmocomBB, OpenBSC, OpenBTS, OsmoNITB, SIMtrace, OsmoGMR, OsmoSDR, rtl-sdr and many more.

I think it was a great chance to make sure the key developers involved with those projects are up-to-date with what everyone else is hacking on. I was especially happy with the presentations of Holger's smalltalk implementation of certain GSM protocols/interfaces, and it seems my small informal Erlang intro has raised some interest.

If anything, the 4-day conference has shown that there is a massive amount of work going on in the various different projects, and that it has clearly grown beyond anything that a single person could still be involved in all the sub-projects.

Personally, I'm happy to see what has grown out of this "we have a BS-11, let's see what we can do with it" that Dieter and I started in 2008. Now we're no longer talking about BTS/A-bis/BSC, but about SS7, MSC, TCAP/MAP, SCCP, HLR, Erlang, smalltalk, DECT, SIM/USIM, COS, SDR, GMR/Thuraya, TETRA and more recently also femtocells as well as NodeBs.

In the spirit of that 2008 presentation Running your own GSM network using the BS-11, Dieter Spaar has now demonstrated his talk on Running your own UMTS network, using NSN or Ericsson NodeBs. I'm really excited to see where that will take us - despite the fact that due to the 5 MHz wide channels, it's pretty close to impossible to get the experimental spectrum licenses that most of us have been able to get in recent years for our work.

As an outlook, over the remaining year 2012, I see progress in the following areas:

osmo-nitb will get a VLR/HLR split (async database access)
we will build a stand-alone osmo-msc with A interface
the signerl TCAP/MAP implementations will be used in production
OsmoSDR firmware will be completed, the hardware will start shipping
a new card operating system (OsmoCOS) will emerge
a UMA gateway will be implemented
a Free Software GPRS/EDGE PCU and RLC/MAC implementation will appear
last but not least, sysmoBTS will start commercial shipment really soon now

I'd like to thank our host c-base for having us block their conference room for 4 days, as well as all attendees who have travelled from all parts of Europe, but even the United States and Russia to participate. There definitely will be another OsmoDevCon, though we don't know yet at which point in time.

Harald Welte: h-online article covering OpenBTS and OpenBSC

Mon, 26 Mar 2012 02:00:00 +0000

You can find a 3-page article about OpenBTS, OpenBSC and related projects available from the h-online web site.

Gustavo F. Padovan: BlueZ on Google Summer of Code 2012

Sun, 25 Mar 2012 15:45:20 +0000

BlueZ was accepted to take part in GSoC 2012, if you don’t know what GSoC is, please go to its page and learn about.

We have already published our list of ideas, so if you are a student take a look there and check what might interest you. Then you can talk to one of our mentors to learn more about and work on a GSoC proposal.
You can get more information about BlueZ on GsoChere. Our contact info is on the same page.

We hope to have a great summer in this year’s Google Summer of Code.

Michael Kerrisk (manpages): man-pages-3.37 is released

Sun, 25 Mar 2012 03:45:07 +0000

I've released man-pages-3.37. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org (and for the time being at least, also remains available on Github).

The most notable changes in man-pages-3.37 are as follows:

A new getent(1) page, contributed by Mark Bannister, documents the getent command provided by glibc.
A major update to the ptrace(2) page by Denys Vlasenko.
Updates to the capabilities(7) page to add various privileged operations to the lists of operations governed by each capability.

(Updated 2012-03-25 to fix a broken link)

Michael Kerrisk (manpages): man-pages-3.36 is released

Sun, 25 Mar 2012 03:44:03 +0000

I've released man-pages-3.36. The release tarballs are once more available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is once more available on kernel.org (and for the time being at least, also remains available on Github).

The most notable changes in man-pages-3.36 are as follows:

A new sendmmsg(2) man page adds documentation of the sendmmsg() system call, which was added in Linux 3.0.
The fallocate(2) man page adds documentation of the FALLOC_FL_PUNCH_HOLE operation, which was added in Linux 2.6.38.
The feature_test_macros(7) man page adds documentation of the _ISOC11_SOURCE feature test macro, which is added in the upcoming glibc 2.16.

(Updated 2012-03-25 to fix a link.)

Dave Jones: Weekly Fedora kernel bug statistics – March 23 2012

Fri, 23 Mar 2012 19:11:14 +0000

	15	16	17	rawhide
Open:	346	505	49	142	(1042)
Closed since 2012-03-16	5	68	11	5
Changed since 2012-03-16	13	407	21	4

This week saw a lot of activity in Fedora 16. We pushed out a rebase to 3.3. Yesterday I did a mass-update to bugzilla requesting people retest. (Which screwed up when I got a bugzilla proxy error, causing everything to be posted 3 times. Apologies to those who got all those mails).

The good news is we seem to be closing a lot of bugs quickly from this rebase.
Around 50 or so bugs got closed just since yesterday.
As with any rebase, some new bugs are showing up, but some of the more common ones (like the bluetooth oopses) I think we’ve got a handle on, and will get fixed in an update out next week.

Next week, 3.3.1 should also land, and that will likely coincide with Fedora 15 also getting a rebase to this kernel. It’s been a while since we’ve had three releases all on the same version. (I think last time was F12/F13/F14).

Despite all this activity, the overall bug count still remains very high. We still have no real answer for all the memory corruption problems caused by hibernate (which accounts for dozens of open bugs now, plus probably a bunch that we haven’t yet attributed to this problem).

Weekly Fedora kernel bug statistics – March 23 2012 is a post from: codemonkey.org.uk

Weekly Fedora kernel bug statistics – March 16 2012 15 16 17 rawhide Open: 349 509 31...
Weekly Fedora kernel bug statistics – March 2 2012 15 16 17 rawhide Open: 344 485 22...
Weekly Fedora kernel bug statistics. No time for a complete breakdown this week. (I’ll post...

Related posts brought to you by Yet Another Related Posts Plugin.

Pete Zaitcev: Confounding

Thu, 22 Mar 2012 03:37:14 +0000

16.1.9 JSON Format for ACLs

ACE flags and masks are members of a 32-bit quantity that is widely understood in its hexadecimal representations. The JSON data format does not support hexadecimal integers, however. For this reason, all hexadecimal integers in CDMI ACLs shall be represented as quoted strings containing a leading "0x".

"cdmi_acl" : [ { "acetype" : "0xnn", .....

If readability by humans is paramount, then why not use a bit string, like in ls(1)? If readability is not an issue, just transmit decimal.

They also have hexadecimals without "0x" prefix elsewhere in the spec.

Gustavo F. Padovan: Bluetooth Changes for 3.3

Wed, 21 Mar 2012 14:05:58 +0000

The Linux 3.3 is out and it’s time to see what changed in the Bluetooth Subsystem, we had a quite big amount of changes this time, over 200 patches were merged during this release cycle.

The most noticiable change (in part due to the bugs it caused ;) was the move of the receive path handling from tasklets to workqueues. This was a long waited feature and will help a lot Bluetooth development from now and then.

In the L2CAP front Andrei Emeltchenko did a lot of clean up and added support for Extended Flow Specification(EFS) and Exteded Window Size(EWS). Johan Hedberg did a lot of Management changes, but biggest changes are landing in the 3.4 release when the Management Interface will be enabled by default.

Apart from that we have lots of LE changes, a new priority scheme inside HCI core, new socket option for L2CAP (BT_CHANNEL_POLICY). This list goes on, we had lots of changes. The shortlog can be seen with:

git shortlog -n v3.2..v3.3 — drivers/bluetooth/ net/bluetooth/ include/net/bluetooth/

LPC 2012: Early Registration Open for LPC 2012

Tue, 20 Mar 2012 17:52:56 +0000

Early registration is open for the Linux Plumbers Conference, 2012. Early registration will be available through April 29, 2012, at which point regular registration rates will apply. All the registration-related details are now available on our ATTEND page. We hope to see you in San Diego! As always, if you have any questions regarding your registration or other aspect of Linux Plumbers Conference, please contact us.

Lucas De Marchi: ELC 2012

Tue, 20 Mar 2012 17:30:48 +0000

Hey, this is my feedback of ELC 2012. If you didn’t read the first part, about ABS 2012, you can read the previous post first.

ELC is one of my favorite conferences as I can meet several talented people and have good talks about Linux in embedded devices. This time was not an exception and I enjoyed very much. The main reason I was there was because I was going to present kmod, the new tool to manage kernel modules. But that would be only on the last day of the conference. Let’s start from the beginning.

To open the conference Jon Corbet gave his usual kernel report starting from January 2011 and going on through the events in each month: the mess in ARM, death of the big kernel lock, userspace code in kernel tree (should we put libreoffice there, too?) and so on. Following this keynote I went to see Saving the Power Consumption of the Unused Memory. Loïc Pallard from ST-Ericsson talked about how memory consumption in increasingly important in embedded devices for the total power consumption. We are going from the usual 512 MB on smartphones to 2 ~ 4 GB of DDR RAM. There are some techniques to reduce this the power drained and he presented the PASR framework, that allows the kernel to turn on/off specific banks/dies of memory since not all of them is used all the time. Later on talking to the guys from Chromium OS I realized that this is especially true when the device is sleeping. We may want to discard caches (therefore use much less memory when in sleep mode) and then turn off banks not used. In my opinion the battery consumption is one of the most important today for embedded Linux devices: I’m tired to have to charge my smartphone every day or every X hours. I hope we can improve the current state by using techniques as the one presented in this talk.

In Embedded Linux Pitfalls Sean Hudson from Mentor Graphics shared his experience while coming from closed embedded solutions to open source ones. Nice talk! I think people doing closed development should see presentations like this: one of the main reasons for failing in opensource is not being able to talk to each other: HW guys not talking to SW guys, NIH, not playing the rules of the communities and therefore having to carry a lot of patches, etc. I’ve always been involved with opensource so I don’t know very well how things work for companies doing closed development, but I do know that more often than not we see those companies trying to participate in communities/opensource and failing miserably. In my opinion one of the main reason is because they fail to talk, discuss and agree on the right solution with communities.

One of the best talks in ELC 2012 was Making RCU Safe for Battery-Powered Devices. Paul McKenney is one of the well known hackers of the Linux kernel, maintaining the RCU subsystem. Prior to this talk I had no idea RCU had anything to do with power consumption. He went through a series of slides showing how and why RCU got rewritten several times in the past years, how he solved the problems reported by community and how things get much more complicated with preemption and RT. He finished his presentation saying that the last decade was the most important of his carrier and that is because of the feedback he got from RCU being used in real life. I’d really love to see more people from Academia realizing this.

The next day Mike Anderson gave a great keynote about The Internet of things. Devices in Internet are surpassing the number of people connected and soon they will be much more important. It’s a great opportunity for embedded companies and for Linux to become the most important Operating System in the world. Recent news about this are already telling us that 51% of the Internet traffic is non-human (although we can’t classify all of that as “good traffic”). Following his keynote I went to see Thomas Petazzoni from Free Electrons talk about Buildroot. I like Buildroot’s simplicity and from what Thomas said this is one thing they care about: Buildroot is a rootfs generator and not a meta-distro like openembedded. There were at least 3 people asking if Buildroot could support binary packages and he emphasized that it was a design decision not to support them. I like this: use the right tool for the each job. I already used Buildroot before to create a rootfs using uClibc and it was great to see that it was already packaging the last version of kmod before I went to ELC.

In the end of the second day I participated in Real-Time (BoFs) with Frank Rowand. It was great to have Steven Rostedt and Paul McKenney there as they contributed a lot to the discussion, pointing out the difficulties in RT, the current status of RT_PREEMPT patches regarding mainline and forecasts of when it will be completely merged. There were some discussions about “can we really trust in RT Linux? How does it compare with having an external processor doing the RT tasks?”. In the end people seemed to agree that it all boils down about what do you have in your kernel (you probably don’t want to enable crappy drivers), how do you tolerate fails (hard-RT vs soft-RT) and that RT is not a magic flag that you turn on and it’s done: it demands profiling, kernel and application tuning and expertise in the field. People gave several examples of devices using the RT_PREEMPT patches: from robots and aircrafts in the space to cameras (the Sony cameras given away on the last day were 1 of the examples).

On Friday, the last day of the conference, I was much more worried about my presentation in the end of the day than with other talks. Nonetheless I couldn’t miss Koen Kooi from Texas Instruments talking about Beaglebone. It’s a very interesting device for those who like to DIY: it’s much smaller than its brothers like Beagleboard and Pandaboard and still has enough processing power for lots of applications. Koen was displaying his slides using node.js running on a Beaglebone. What I do like to see though is barebox replacing u-boot as the bootloader. If you attended Koen’s talk on ELCE last year, you know u-boot is one of the culprits for a longer boot. Jason from TI kindly gave me a Beaglebone so I can use it for testing kmod; when I have some spare time I’ll take a look on the things missing for using barebox on it, too.

The last talk of the conference was mine: Managing Kernel Modules With kmod. I received a good feedback from people there: they liked the idea behind kmod – designing a library and then the tools on top of that. I had some issues with my laptop in the middle of my presentation, but it all went well. I could show how kmod works, the details behind the scenes, the short history of the projects and how it’s replacing a well known piece of userspace tools of Linux in all major desktop and embedded distros. When I was showing the timeline of the project I remember Mike Anderson saying: “tell us when it will be done”. I can’t really say it’s done now, but after the conference we already had versions 6 and 7 and contrary to other releases in the latest versions the number of commits is very small. After 3~4 months the project is reaching a maintenance phase as I said it would. If you would like to see my slides, download it here or see it online below. You can also watch the video of my talk as well as all the others in LF’s video website.

Matt Domsch: s3cmd sync enhancements and call for help

Tue, 20 Mar 2012 05:29:28 +0000

Coming soon, Fedora and EPEL users with virtual machines in Amazon (US East for starters) will have super-fast updates. I’ve been hacking away in Fedora Infrastructure and the Fedora Cloud SIG to place a mirror in Amazon S3. A little more testing, and I’ll flip the switch in MirrorManager, and all Amazon EC2 US East users will be automatically directed to the S3 mirror first. Yea! Once that looks good, if there’s enough demand, we can put mirrors in other regions too.

I hadn’t done a lot of uploading into S3 before. It seems the common tool people use is s3cmd. I like to think of ‘s3cmd sync’ as a replacement for rsync. It’s not – but with a few patches, and your help, I think it can be made more usable. So far I’ve patched in –exclude-from so that it doesn’t walk the entire local file system only to later prune and exclude files – a speedup of over 20x in the Fedora case. I added a –delete-after option, because there’s no reason to delete files early in the case of S3 – you’ve got virtually unlimited storage. And I added a –delay-updates option, to minimize the amount of time the S3 mirror yum repositories are in an inconsistent state (now down to a few seconds, and could be even better). I’m waiting on upstream to accept/reject/modify my patches, but Fedora Infrastructure is using my enhancements in the meantime.

One feature I’d really like to see added is to honor hardlinks. Fedora extensively uses hardlinks to cut down on the number of files, amount of storage, and time needed to upload content. Some files in the Fedora tree have 6 hardlinks, and over three quarters of the files have at least one hardlink sibling. Unfortunately, S3 doesn’t natively understand anything about hardlinks. Lacking that support, I expect that S3 COPY commands would be the best way to go about duplicating the effect of hardlinks (reduced file upload time), even if we don’t get all the benefits. However, I don’t have a lot more time available in the next few weeks to create such a patch myself – hence my lazyweb plea for help. If this sounds like something you’d like to take on, please do!

Pete Zaitcev: Bugception

Mon, 19 Mar 2012 20:00:24 +0000

[root@kvm-rei zaitcev]# rpm --rebuilddb error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db5 - (-30969)

I have a vague feeling that I encountered this before, but I do not remember how I dealt with it.

UPDATE: Needs rm /var/lib/rpm/__db* .

Harald Welte: Using a cheap (USD 20) DVB-T USB stick as SDR receiver for (not only) gnuradio

Sun, 18 Mar 2012 01:00:00 +0000

Fellow Osmocom hacker Steve Markgraf has been working on what now seems to be the cheapest way to receive real-world radio signals for PC-based SDR like gnuradio: rtl-sdr. RTL refers to the RTL2832U chipset frequently used in such devices. It can be used to obtain 2.8 Ms/s of 8-bit I+Q samples.

Below is a picture (courtesy of Steve) how the hardware looks like:

Matthew Garrett: More ways for firmware to screw you

Sat, 17 Mar 2012 23:07:32 +0000

Some of my recent time has been devoted to making our boot media more Mac friendly, which has entailed rather a lot of rebooting. This would have been fine, if tedious, except that some number of boots would fall over with either a clearly impossible kernel panic or userspace segfaulting in places that made no sense. Something was clearly wrong. Crashes that shouldn't happen are generally an indication of memory corruption. The question is how that corruption is being triggered. Hunting that down wasn't terribly easy.

My first thought was that we were possibly managing to load the kernel over a region used by UEFI code. UEFI defines two types of code - boot services and runtime services. While runtime services code and data must be preserved by the OS, in theory boot services code and data is available to the OS once the firmware has exited. In practice, that's not true. It seemed entirely possible that the kernel might be ending up on top of some of that boot services code or data and getting trodden on. Grub now has code to avoid putting the kernel on boot services, so testing the latest code seemed like a good plan. But no, crashes still happened.

That pretty much ruled out the bootloader. My next thought was that executing some of the firmware code was triggering a write to some other memory that contained the kernel. Josh Boyer suggested the next trick, which was to try marking the kernel read-only to see whether anything was hitting it. x86 lets you mark pages as read-only - any attempt to write to them should take a fault. UEFI functions are executed in the context of the kernel, so share the same page tables. That let me rule this out, since everything still went just as wrong and I wasn't taking an extra fault first.

However, at this point I was reasonably happy that it wasn't the kernel itself being overwritten - faults were occurring in userspace code as well. That was a pretty strong indication that what was happening was continuing to happen once userspace had started, so it wasn't a direct response to a firmware call. I made sure of that by stubbing out all the calls that could be triggered after kernel initialisation, and saw the same failures. Once all attempts to be clever have failed, it's time to just start using brute force. The kernel lets you reserve areas of RAM by passing arguments like memmap=0xlength$0xstart to block length bytes starting at start from being used. It took a while, but I finally found a 256MB range that made a difference - reserving it resulted in the machine booting reliably, letting the OS use it resulted in occasional crashes.

Definite progress. Comparing that memory range to the EFI memory map was helpful. There were several blocks of UEFI boot services data present there, which really seemed like too much of a coincidence. By reserving each of them in turn, I'd traced it down to a single 31MB region of boot service data - that is, memory reserved by the firmware for use by the UEFI boot services. Per spec, this is available to the OS once the boot environment has been exited. Nothing other than the OS should be touching this after boot, but something clearly was. Tracking down what was far easier than I expected, although the first attempt was a failure. Setting it read-only should have triggered a fault, but didn't. That was rather confusing. But, rather than give up, I patched the kernel to fill the region with 0xff at kernel init. Then I booted the system, read it back and looked for values that weren't 0xff. I got this:

00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
001568a0  ff ff ff ff ff ff ff ff  ff ff ff ff 84 00 00 00  |................|
001568b0  00 20 a7 ac 46 00 00 00  00 00 00 00 00 00 06 01  |. ..F...........|
001568c0  c2 0b 0c 00 ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
001568d0  ff ff 0a 04 f0 03 82 0d  40 00 00 00 ff ff ff ff  |........@.......|
001568e0  ff ff 00 21 00 36 9a 80  ff ff ff ff ff ff 00 7e  |...!.6.........~|
001568f0  00 09 43 48 41 2d 47 75  65 73 74 01 04 02 04 0b  |..CHA-Guest.....|
00156900  16 32 08 0c 12 18 24 30  48 60 6c 2d 1a 0e 18 1a  |.2....$0H`l-....|
00156910  ff ff 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00156920  00 00 00 00 00 00 00 dd  09 00 10 18 02 00 10 01  |................|
00156930  00 00 dd 1e 00 90 4c 33  0e 18 1a ff ff 00 00 00  |......L3........|
00156940  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00156950  00 00 bd ea f8 b3 ff ff  ff ff ff ff ff ff ff ff  |................|
00156960  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
022fb000

That's a lot of 0xffs (around 31MB of them) with one small section that contains an 802.11 probe packet with the SSID of the hospital across the road from my house. Apple support network booting off wireless networks. It seems that the firmware brought up the wireless card, associated with this network (it's the only public one nearby) and then left the card DMAing packets into RAM. The read-only page attribute only applies to CPU-initiated accesses, so it could do this without triggering a page fault. It also explained why it was so random - whether memory corruption occurred would depend on whether a packet appeared between that memory being used by the OS and the kernel reinitialising the wireless card. It certainly explains why I couldn't reproduce it when I left the machine repeatedly rebooting on the bus home.

How do we fix this? Unsure. With luck disconnecting the UEFI driver in the bootloader should quiesce the hardware, but without testing I'm not sure of that yet. For now it's just another example of firmware managing to break expectations in deeply strange ways.

comments

Lucas De Marchi: Android Builders Summit 2012

Sat, 17 Mar 2012 12:25:59 +0000

Four weeks ago, from Februart 13rd to Februrary 17th I was at Android Builders Summit and Embedded Linux Conference in San Francisco. I was a bit busy these last weeks so I didn’t have an opportunity to write here about the conferences as I usually do. I was going to do a post about both the conferences, but after writing a little bit I realized it would be very big. So I split in 2 and here is the one for ABS 2012; the other is coming soon.

This was my first time at Android Builders Summit. Since in the end of last year I participated in a project modifying Android internals, I felt it would be really good to be in touch with people doing the same things and learn with them. Before going there I was already surprised that Google was not sponsoring the conference, but there I was astonished that there was nobody from Android’s team and I don’t remember talking to Googlers, too. I don’t know what happened but it would be really good for the next conference if Google could be part of the conference since for the very nature of how they manage the project they are the people pushing the platform forward.

In the first day of the conference Greg Kroah-Hartman, Tim Bird and Zach Pfeffer answered the question “Android and the Linux Kernel Mainline: Where Are We“: it’s done. Well, not totally done, but most of the code needed in kernel is already in mainline: except for some pieces that render your device useful it’s already possible to boot Android userspace with a mainline kernel. I think the main point of this effort is to allow companies and enthusiasts to use features from the mainline kernel and newer versions than the ones available on AOSP. As the diff between mainline and Android’s kernel decreases it’s much easier to deploy an Android device with a different kernel. More details can be found in http://lwn.net/Articles/481661/.

From the other talks I attended on the first day, the one that caught my eyes was USB Device Support Workshop. Bernie Thompson from Plugable talked a bit about the lack of proper support in Android to deal with kernel modules: it’s really hard for device maker companies like his own to have products working on Android. And it’s not because they aren’t committed to developing Linux device drivers but because of the lack of support in Android to easily deal with kernel drivers: either the external device is supported by the company shipping the Android product or there’s no way for example to plug in an external camera and get it to work. Audience was a bit confused saying that that was a Linux problem and some voices telling that in Windows lands that doesn’t happen. Not true. Linux supports more devices that any other operating system in the world, however Android is currently missing some tools to profit from it. After some discussion Bernie prepared some tables with USB devices that people could hack on, get it supported in Linux/Android, etc.

On the second day I attended Real-Time Android, particularly because of my involvement with real-time since I graduated at university and because I was curious about applying it to Android. As I said one of the benefits of having Android kernel closer to mainline is that it’s easier to do things like this. Wolfgang Mauerer from Siemens applied the RT_PREEMPT patches to Android’s kernel so you could have a real-time embedded system and still use Android’s app. As I was expecting RT would be applied for native applications, not java based ones.

Topics in Designing An Android Sensor Subsystem: Pitfalls and Considerations was advanced talk about Sensors in Android and how one would choose one strategy over another and the tradeoffs between battery life, sample rate, external co-processor, DIY or license the algorithms used, etc. It was not a talk for the regular Joe doing an app that uses the Android’s Sensors API (that was what I knew about it) but more for people creating devices that would like to use sensors.

It was a conference different from the conferences I’m used to attend like ELC/LinuxCon: there was very few people who I already knew and I had the feeling that we were talking about a product from someone else, not a product we were helping to develop – instead we were having talks about how to hack a platform we do not own. In general I liked the talks I could attend and talking to people at the corridors. They even gave me some insights for my talk about kmod, later on Friday at ELC. I’ll talk more about it on the next post.

For those wanting to see the slides/videos, Linux Foundation made them available at their site – go on and see for yourselves.