Dedication

This LWN.net Weekly Edition is dedicated to the people of Japan as they work to recover from the terrible events which have overtaken them. LWN has a lot of friends in Japan; you are all constantly in our thoughts.

Comments (1 posted)

The world of Chromium

Never let it be said that all LWN editors are on the cutting edge (no pun intended)—or even near it. While some grumpy editors may run their desktops on rather frequently updated distributions like Rawhide, others like a more sane day-to-day desktop, reserving Rawhide for the laptop. That particular configuration can be hazardous without proper vetting before setting out across the Atlantic for a conference, which is a problem I have smacked into one too many times. So both of my main systems are running Fedora 14 these days. But there is this "new" web browser—perhaps you've heard of it—called Chromium, that all the cool kids are running; the release of Chrome 10 seemed like a good time to dig in and give it a try.

I have dipped my toes in the Chromium waters previously, of course, but not really used it seriously. I have been using Firefox for approximately forever, am used to its quirks, and know how to get the most out of it, at least for my needs. When playing with Chromium in the past, I have grabbed it from Tom "spot" Callaway's Fedora repository, but being a bit confused about Chromium development, I assumed that he hadn't yet updated to the Chrome 10 sources since his most recent build was from the end of February. That led to a plan to build Chromium from source.

In retrospect, it should have been obvious that by building Chromium from source, one gets a version that is quite a ways ahead of the most recently released proprietary Chrome browser. Google releases binary versions of its Chrome browser quite frequently, and has versions available for Linux as well. Its most recent release is version 10, but Chromium builds are already based on version 11. Chrome is based on the same codebase as Chromium (or vice versa), but bundles Flash support along with support for patent-encumbered video codecs like H.264 (though that support is slated to be dropped). The Chrome/Chromium naming is somewhat confusing, unfortunately, and is only made worse by the OS versions (i.e. ChromeOS and ChromiumOS).

Building Chromium

To start with, building Chromium is not for anyone with a lack of extra disk space. The bootstrap tarball is nearly a gigabyte in size. Then you need to install the depot_tools which will sync the repositories in the tarball to the latest versions. Make sure that the system has all of the prerequisites (for which the project helpfully has distribution-specific instructions) and you are ready to build.

The sync process, which uses something called gclient, completes by running a configure-like step to generate makefiles. After that, a simple "make" will build the browser—or not. As it turns out, by default all compiler warnings are turned into errors, so the build failed almost immediately for me. I like the idea behind the -Werror option for gcc, but it does seem like you would only make it the default if you were pretty darn sure that the compiler would make no complaints. Perhaps for Google that's true.

Fixing that problem was straightforward using the supplied gyp_chromium configuration tool, but that seemed to break a simple "make"—a "make chrome" was now required. That and many hours of compilation (I'm not sure how long as I let it run overnight) along with a truly astonishing 7G of disk space gives one a chrome executable to run. There is no installation step, so I have just been running it from the out/Debug directory.

Running Chromium

I had some lingering configuration files from running Callaway's versions of Chromium, which immediately caused my version to crash. Moving the ~/.config/chromium directory aside made that problem go away. When it first starts up, Chromium, interestingly, asks which search engine you want to use by default (either Google, Yahoo, or Bing) in a dialog box that is only half-visible because it is located too low on the desktop. I am a bit surprised that the choice is even that prominent though; one would have to guess that Google is trying to head off claims of anti-competitiveness here.

Once the browser itself comes up, it ... well ... acts pretty much like a browser. There are some nice features, certainly, like typing searches directly into the address bar, the better use of vertical screen real estate by not cluttering it with a bunch of extra gunk, the isolation of a different process for each tab, and so on. Most of the keystroke combinations for opening tabs, highlighting the contents of the address bar, and the like all work much as they do in Firefox.

While I have heard raves about how fast Chromium is, it didn't really seem that way to me. I didn't find it noticeably faster or slower than Firefox. Whether that's because of the voluminous debugging output, the build configuration, or my sometimes overly pokey fixed wireless internet is unclear. Certainly Firefox has its warts, but I haven't really found the speed of its rendering or JavaScript execution to be a serious problem; others who use their browsers in their own way may see things differently.

One of the first things that the start page for Chromium suggests is importing settings, bookmarks, and passwords from another browser. That necessitates shutting down said other browser—not unreasonably—but importing would then reliably crash Chromium with a relatively unhelpful:

    [29941:29941:1904760218149:FATAL:thread_restrictions.cc(36)] Function
    marked as IO-only was called from a thread that disallows IO!  If this
    thread really should be allowed to make IO calls, adjust the call to
    base::ThreadRestrictions::SetIOAllowed() in this thread's startup.

Earlier messages in the trace indicate that it may be a problem trying to talk to KWallet, but neither Callaway's version nor Google's Chrome binary had any problems of that sort.

Trying to fire up Callaway's version for comparison allowed me to see another oddity. Even though the version I built is only a few weeks more recent than Callaway's (11.0.698 vs. 11.0.679), the older one would not run using the configuration of the newer. Running chromium-browser --help (which is the name of Callaway's executable) was not particularly helpful, but there is a man page in the RPM, which leads one to the --user-data-dir option.

A note of warning to those who use that option, however, no shell expansion is done on the argument, so an invocation like:

    $ chromium-browser --user-data-dir=~/.config/fedora-chromium

will not do what one might expect. Instead of tossing its configuration directory below your home directory, it will create a "~" directory in your current working directory—with all of the amusing chaos that can result.

Fuzzy fonts

The reason a comparison was needed at all was due to a, seemingly, very basic problem: the fonts displayed by Chromium were lacking sharpness and were in fact fuzzy and hard to read (see example at right). Based on using one of Google's other products, it would seem that this is a common problem on Linux. None of the solutions that were found in an admittedly fairly short search fixed the problem, though there were hints about sub-pixel rendering causing it and possibly a Microsoft-held patent that made doing the right thing difficult for Linux.

Neither Callaway's version nor Google's Chrome showed any improvement, and even a switch to GNOME (from KDE) didn't fix it. The Microsoft mention while searching, though, reminded me that I had installed msttcorefonts (Microsoft's TrueType core fonts) as directed by the prerequisites page. Installing those fonts had noticeably changed the appearance of many web pages in Firefox as it will evidently pick up those fonts if they are installed. But Firefox was able to do whatever font-rendering magic was required so that the fonts appeared clean and sharp (example at right).

Undoubtedly there are ways to fix the problem, but why use finesse when a big hammer is at hand? Removing msttcorefonts from the system made the fonts much clearer, though still not quite up to the standard set by Firefox. It should be noted that Firefox itself is rather unamused by having those fonts pulled out from under a running instance of the program, but reloading the affected pages cleared that up. Fonts on Linux are a longstanding problem, overall, but one that seemed to have been largely fixed by modern distributions. Chromium and Chrome's failure to provide a good viewing experience is pretty clearly a consequence of Google's lack of interest in the Linux desktop platform.

Other annoyances

One of the first irritating things to confront me when using Chromium was the "warning" that LWN.net was not actually serving all of its content via HTTPS. The warning manifests itself as a red slash through the (now red) "https" in the address bar along with a red "X" on the padlock (seen at right). This problem isn't confined to LWN, of course, any site that tries to serve its content over TLS/SSL, while simultaneously trying to generate a little income from Google ads will suffer the same fate. Google's sites, like Gmail, are not so afflicted, of course, and get a lovely green padlock and "https" text. While it's not exactly the fault of Chromium, it's just rude enough to make one consider turning off the Google ads on the site entirely.

There is also a bright green flash that shows up every time you switch tabs, expose part of the window, or load a new page. Either the whole window or just the newly exposed part flash green for a split second. It's not clear whether that's intentional or just some artifact of how Chromium does its rendering to the display. It is kind of annoying after only a short time using the browser, however.

The recommendation that JavaScript be turned on for all sites, as found in the preferences screen, is a little puzzling. Chromium certainly has a focus on security, so maybe the project is comfortable with that recommendation. Were I to switch over to Chromium permanently, investigating an extension like the NoScript-inspired NotScripts would be high on the list.

In using Chromium for several days, it pretty much just worked for most anything I tried to do—at least once the font problem was hacked around. The only exception I found was in using the WordPress editor for a news site that I do some volunteer editing for. The cut-and-paste functionality that I was used to using with Firefox did not work the same way in Chromium. That's another problem that could presumably be fixed, or worked around, but it seemed simpler to just switch over to Firefox to complete that task.

Chromium development

Chromium is free software, with a BSD license, but it also requires a Contributor License Agreement (CLA). The CLA gives Google an irrevocable copyright license to any contributions without making any promises about what it might do with the code. Obviously that code will end up in Chrome, which has a proprietary license. Where else it might end up is unclear.

There is, of course, a Chromium project, with all of the requisite tools for managing and developing the browser, including documentation, mailing lists (in the Google Groups sense anyway), bug tracking, source code browsing, and so on. It is a bit difficult to get a sense for how many folks outside of Google are participating in Chromium development though. All committers get @chromium.org email addresses, which unfortunately masks the contributors to some extent.

The chromium-discuss group is fairly active, but doesn't seem directly focused on development topics. There are more questions about how things work and requests for features to be added than there are design or development discussions at least over the last several days. One gets the sense that most of the work is done behind closed doors at the Googleplex, but that may just be jumping to conclusions.

Conclusion

Chromium is a perfectly usable browser, with a bunch of interesting features, and is certainly well worth a look for all six readers who haven't yet done so. I hadn't really put it through its paces, largely due to inertia, and I didn't find anything in the experience that would make me switch permanently to Chromium. There are things to like and dislike about any tool, but a browser is not typically a tool for which application-hopping makes sense. Once you find one that suits your needs, it would take something very compelling to cause a switch. Obviously lots of folks have found that feature or combination of features in Chromium and made the leap, but that wasn't my experience.

The fact that the open source Chromium browser exists is a real boon for Linux and its users. More competition in the browser space is welcome and many of the innovations that Chromium brings will find their way into Firefox and other browsers down the road. People use browsers in different ways, and have different needs, so more choices of well-polished, robust browsers can only be a good thing in the long run.

Comments (72 posted)

Turning VistA into a "real" open source project

The US Department of Veterans Affairs (VA) is exploring the creation of a "custodial agent" to govern the development of VistA, its popular open source electronic health record (EHR) platform. VistA exists as open source code because it is in the public domain, and a successful ecosystem of VistA-based projects and companies has grown up around it offering EHR services, but without the governance and infrastructure of a "real" open source project.

High-angle view on VistA

VistA was developed slowly by the VA, beginning around 1977. In its current form, it consists of more than 100 individual applications handling discrete EHR tasks, including prescriptions, medication dispensing, clinical orders, doctors notes, tracking test results, and many more. It is used by the VA's network of more than 1000 hospitals, clinics, and medical facilities, and is regarded as one of the key factors behind the Veterans Health Administration's top ratings on patient satisfaction and quality of care — regularly beating out Medicare, Medicaid, and private health care services.

In addition to its use by the VA, VistA was adopted as the core of the US Department of Defense's Composite Health Care System (CHCS), the Indian Health System's Resource and Patient Management System (RPMS), Finland's MUSTI Consortium, and a number of non-governmental institutions. Several open source projects packaging VistA for deployment have developed over the years as well, most designed as commercial products created by system integrators that offer support services and add-ons, such as OpenVistA by Medsphere and vxVistA by DSS. The WorldVistA non-profit organization was established in 2002, and packages its own version, called WorldVistA EHR, in addition to providing discussion forums, documentation, and resources for adopters.

In spite of its popularity, VistA deployment is not simple. Modules are written in the healthcare-tailored language MUMPS, which offers its own database functionality that is not compatible with general-purpose relational databases. This is not a relic inherited from VistA's age, either: as VistA architect Tom Munnecke explains, relational databases' row-and-column table requirements simply are not compatible with the large structures needed by EHR, starting with the fact that EHR records are extremely sparse (consider, for example, that the non-null entries in a single patient record constitute only a tiny fraction of the set of all possible medical diagnoses).

In addition, the VistA code is in the public domain because it is taxpayer-funded — available courtesy of the Freedom of Information Act (FOIA) — but this is a long way from a truly open source project: APIs and ABIs are not formally specified, the documentation is slim and infrequently updated, and there is no mechanism for downstream VistA users to request enhancements. While the WorldVistA group helps fill in some of these gaps, the VistA community has long lobbied for a more formal solution.

Requesting information

In recent years voices within the VA itself began to voice frustration at the slowing pace of VistA development. VA CIO Roger Baker told the FierceGovernmentIT blog that the EHR marketplace had changed considerably since VistA's early days, and that private sector players are creating features that the VA would like to incorporate into its own system — and producing them far more rapidly than the VA.

Baker's proposed solution was to transform VistA into a publicly-run open source project in the traditional sense, incorporating non-VA VistA users into the design and development process. "One of the things that has slowed down innovation is that about 10 years ago, the VA centralized development of the VistA applications in a software development group", he said. In August of 2010, the VA issued a formal request for information (RFI) to solicit input from the public in "evaluating the viability of including Open Source (OS) software as a component of IT development within the VistA Electronic Health Record (EHR) Architecture".

The initial results of that RFI were released in February, outlining a plan to move forward with VistA as an open source project. This second document is also an RFI, since it solicits feedback from the community on the plan. The primary headline is that the VA would create an open source "Custodial Agent" (CA) to manage the VistA source code repository, certify proprietary applications for compliance, and accept patches and code contributions. The CA would be governed by a board that would set bylaws, define membership and intellectual property guidelines, and determine the appropriate license for the VistA codebase.

Regarding the license selection question, the RFI says it "expects that the CA will use an Open Source Initiative (OSI) Approved license but is open to alternative frameworks that may accomplish its objectives." The objectives listed include royalty-free source code, open access to all code in the repositories, non-discriminatory access, the right to make and distribute derivative works, and the right to package, redistribute, and support without requiring attribution.

The RFI also lays out a plan for modernizing the VistA codebase, refactoring the project, publishing the refactoring design, and committing the work in the open — including all architecture documentation and API specifications. On the other hand, it does not get into many specifics, such as the size, election, and makeup of the CA board, nor does it attempt to spell out any of the technical decisions (which it says will be among the board's first tasks).

The full document is 21 pages long, although the last four or so are devoted to a detailed questionnaire asking for feedback and demographics about personal or organizational experience and background. Each section of the RFI ends with a bullet list of specific feedback questions, such as "What methods of generating, sharing, and maintaining documentation provide the most value to the open source community?" and "How important is a common development environment to the developer community?"

Reaction time

The response from the open source VistA community has been mixed, but generally positive. Open source healthcare "hacktivist" Fred Trotter said the VistA community viewed the RFI "with both hope and a little fear. We have been waiting for something like this for years, but a lot could go wrong here." Postings on WorldVistA's Hardhats mailing list typified that reaction — several emphasized the importance of the API and architecture documentation, while expressing hesitance to embrace the plan outright. In that thread, Gregory Woodhouse said "it's high time we stop complaining and get about the business of developing a set of open standards people can rely upon."

Trotter had called for a VistA "council" similar to the proposed CA as far back as 2008. His proposal hinges on giving stewardship of the VistA code to an organization not controlled by the VA, but also not beholden to the commercial VistA integrators. The latter requirement depended on electing a fixed number of council representatives from VistA-using hospitals, rather than allowing all seats to be selected by the development community.

Some of what Trotter describes dips into the details that the RFI defers to the eventual CA board. What is more interesting in the long run is the closing thought, "anyone should see that the council that I am proposing has parallels with WorldVistA." WorldVistA is currently the closest thing to the proposed CA, because it provides a voice for non-commercial VistA users and developers, but it also lacks a definite governance structure, functioning as a free-form entity instead.

If the VA adopts the CA model described in the RFI, then the need for WorldVistA may eventually fade, or else the group might simply evolve into a different role. For the foreseeable future, however, it serves as the central meeting place and voice for VistA users, so its support for the VA open source strategy is essential. There is clearly trepidation among WorldVistA members about whether the VA will deliver on the principles and promises outlined in the RFI.

I have only an outsider's perspective on VistA, but it appears that this trepidation stems from years of dealing with a VA that threw the code over the FOIA-wall and did little to directly address outside users' needs. In light of that history, trepidation seems normal. But the plan outlined in the RFI sounds solid, and hits the right notes: it does not simply describe an open source foundation, but it details the reasons why it must function independently of the VA to succeed, and addresses the concerns of developers in a way that indicates that the VA understands why open source actually works.

The VA says it will complete its evaluation of submitted responses by March 25, and presumably will issue at least a summary of them several weeks after that. Still it may be many months before the first concrete steps of the plan begin — so far, not even a rough timeline has been discussed. For his part, Trotter is cautiously optimistic because of the slow pace the VA is taking. Putting out an RFI on the plan, he said, shows balance, and will allow the agency to gather input and avoid the big pitfalls. So it may be quite a while before we see VistA complete any transformation into a real, vibrant open source project, but if the success story of the VistA ecosystem is anything to judge by, it will be a huge win for open source — and for health care — when it finally arrives.

Comments (22 posted)

The Grumpy Editor's GNOME 3 experience

There are advantages and disadvantages to running a development distribution like Rawhide. One of those is that users often get to experience new software well ahead of all but the most dedicated developers and testers. Whether this feature qualifies as an "advantage" or not will be left for the reader to determine. While (sometimes unwelcome) bits of GNOME 3 have been slipping onto your editor's desktop for a while, he has, thus far, avoided engaging with the full GNOME 3 experience. Nothing lasts forever, though, especially when it comes to development distributions. As the features slowly drained out of the GNOME "fallback" environment, it seemed to be the right time to jump in with both feet. What follows are some impressions of where GNOME is going.

The early days of GNOME 2 were characterized by a relentless campaign to hunt down and eliminate any configuration options that seemed unnecessary, for a large and inclusive value of "unnecessary." Strangely enough, this move proved to be unpopular with users, with the result that, over time, many of those options were added back. GNOME 3 shows signs of wanting to repeat this history; the end result may well be about the same.

Configuration options

This all came forcibly to your editor's attention when the font used for various desktop elements (application menu and tool bars, for example) suddenly changed and became larger. That created havoc on a carefully laid-out desktop, creating scrollbars and "more bookmarks" menus where none had existed previously. Anybody who follows GNOME releases knows that there are few things this project likes better than forgetting its users' configuration selections; it was logical to assume that this had happened again, sigh heavily, and go off to fix things up. The assumption seemed valid - blinking cursors in text areas made an unwelcome reappearance at about the same time - but things turned out to be different this time.

One of the more visible changes in GNOME 3 is the new "system settings" window. This window now lacks any sort of font selection utility. Some searching turned up others who were asking how to change their fonts in this new world; it turns out that there is an answer: go to the "universal access" section. Sure enough, "universal access" has a menu for "text size" with all of three options: "normal," "large," and "larger." Not much help for somebody who wants his old ten-point fonts back. But it seems that anybody who wants to change his fonts (beyond "larger") will have to delve into the GNOME settings registry; there is no plan to restore font selection to the interface. Why is that? Here's what your editor was told:

Having wandered out of the area of "settings that users *should* want to change," your editor is out of luck. His complaints have led to a suggestion that a "smaller" option might be added, but it has not yet made an appearance as of this writing.

This sort of thinking appears throughout GNOME 3. It will no longer be possible to configure what happens when a laptop's lid is closed. The nice dialog which made it possible to restore the control key to its $DEITY-intended position is long gone; it's a good thing that xmodmap still works. Screen saver configuration seems to have gone entirely by the wayside; it's fade-to-black for everybody now. User-supplied background wallpaper can only come from the "pictures folder," the location of which evidently cannot be queried or changed; in this case it turned out to be the home directory. Evidently nobody wants to enable mouse clicking with touchpad taps anymore. And so on. That is the world GNOME is (once again) aiming for. It is, evidently, seen as an increase in usability that will bring large numbers of new users to the platform. (Update: it turns out that some of these options still exist; see the comments for their new locations.)

gnome-shell

It is in this context that your editor, with some trepidation, enabled gnome-shell on his desktop. Given the changes in the platform, the complaints about gnome-shell seen elsewhere, and some early experiences, he thoroughly expected to hate the whole thing. After two weeks of usage in the doing of real work, things have not turned out quite that way; gnome-shell is not that bad, and there are even some things to like about it. That said, there are a lot of things that could be better, and a return to the "fallback" environment is likely, at least for a while.

The initial gnome-shell experience was harsh indeed; it ran so slowly that the desktop was essentially frozen. Your editor is a fast typist, but, unreasonable person that he is, he still believes that a terminal emulator should be able to keep up with him; that was not the case when running gnome-shell. The problem, as it turns out, is the result of limitations in some Intel graphics chipsets which cannot do 3D rendering if the display width or height exceeds 2048 pixels. The system in question, running with two monitors, had such a display. Disabling the second monitor makes things work, but with an obvious cost; one might describe it as a new form of the classic time/space tradeoff.

One can argue that this particular failure is not gnome-shell's fault, but it is a consequence of the decision to require working 3D support to run gnome-shell at all. It seems likely that, as more users try GNOME 3, the number of people encountering this kind of problem will grow. Systems which work perfectly under GNOME 2 can collapse under GNOME 3. One can only hope that the detection of such systems improves in the near future; gnome-shell should simply refuse to run in such situations. The alternative - a nearly-frozen desktop - is just not the sort of Amazingly Improved Usability experience that users might have been hoping for.

Beyond that, the desktop is not quite as responsive as it was under GNOME 2; there is a perceptible delay when scrolling within windows, for example. Emacs windows get corrupted when compositing is in use; this bug has been reported for a while, but no solutions are in sight. That said, gnome-shell does offer a desktop which is visually pleasing, with nice drop shadows, zooming effects, and such. One can argue about whether it's all worth the cost, but 3D rendering is something that should work well on most Linux systems at this point. The decision to make use of hardware-accelerated 3D rendering in gnome-shell is defensible - as long as the alternatives work properly.

The GNOME 2 panels, which could be configured to appear almost anywhere on the display (your editor's laptop has a single panel running vertically on the left edge) has been replaced by a mostly featureless black bar wired to the top of the screen. Launchers as such are a thing of the past. Instead, one clicks on the "Activities" button (or simply slams the pointer into the upper left corner of the screen); in response, the desktop will zoom back yielding a display of application windows (no longer overlapping), a "dash" on the left side of the display, and a strip representing workspaces down the right side.

The "dash" looks like an application launcher panel, but it is a bit different. Its contents are the union of the applications which are currently running and the user's "favorites." Clicking on an icon will either (1) launch the application if an instance is not already running, or (2) take the user to a running instance if it exists. Getting a second browser window (say) requires right-clicking on the icon and selecting "new window." So, an operation which was previously accomplished with a single click on a panel icon now requires (1) moving to the upper right corner of the display, (2) right-clicking on an icon, and (3) selecting a menu entry - a rather longer and slower sequence of events.

It is not immediately clear this behavior is an increase in usability. The reasoning behind this change is described this way:

Your editor, having been blissfully unaware of the scourge of unnecessary calculators just waiting for their opportunity to overwhelm his desktop, has not yet come to love the new way of doing things.

Happily, gnome-shell has not done away with the concept of workspaces, despite some claims that workspaces confuse inexpert users more than almost any other feature. They have become more dynamic, though, and do not really exist until windows are dragged into them. Anybody who uses workspaces heavily may come to miss the old scheme where workspaces were fixed. In your editor's world, workspace 2 is where LWN writing gets done, workspace 3 is for programming, and email is hidden in workspace 1 where it can be ignored for extended periods. Photo editing tends to happen in workspace 4, and so on. Gnome-shell makes it harder to know where everything is without having to go looking for it; one learns to be careful in the initial population of the workspaces.

Switching between workspaces involves moving the pointer to the upper left corner to zoom back, then going all the way to the right side of the display to select the workspace of interest, then clicking on a window (or hitting escape) to zoom back in - a lengthy ritual. The good news is that it is still possible to designate keystrokes for moving quickly between workspaces, so it's not necessary to do the zoom-back-and-click dance every time.

As has been discussed elsewhere, gnome-shell has done away with the "minimize" and "maximize" buttons on window titlebars. Your editor never found any use for either and, thus, does not miss them at all. On the other hand, the maximization of windows by dragging them to the top of the screen is unintuitive, surprising, and unwelcome. It's easily undone, but it's annoying.

In summary

There are a number of other weirdnesses perpetrated in the name of usability. For example, there's no "power the system off" option in sight. To turn off the system, the user must click on one's own name at the upper right and hold down the "Alt" key while the menu is visible. Clicking on one's name may be more intuitive than stopping the system with the "Start" button, but it seems a little strange and possibly narcissistic as well. Hiding a useful menu item (we all turn off our systems sometimes) behind the "Alt" key, instead, seems intended to confuse.

All of these complaints notwithstanding, it must be said that gnome-shell is not an unpleasant environment to work in most of the time. It looks nice, and most of the needed functionality is not that far away. It is also said to be designed for easy enhancement via extensions written in JavaScript; very few of these extensions exist now, but one can imagine that developers will use this capability to improve the gnome-shell experience considerably in the future.

For now, your editor will be switching back to the fallback environment so he can have his second monitor and Emacs back. But, if a properly functioning gnome-shell were the only alternative available, it would not be that bad a fate. Around GNOME 3.4 or so, when the important configuration options come back, GNOME 3 has a good chance of being a pleasant, flexible, and powerful desktop. But the road between here and there may be a little rough in spots.

(Finally, your editor would like to thank the numerous GNOME developers who have responded to his questions. We did not always agree with each other, but they were always more than willing to answer questions and to help get things working.)

Comments (344 posted)

Postfix TLS plaintext injection

Transitioning between states, or contexts—unencrypted to encrypted for example—is one place where security flaws can sometimes hide. We have seen one example of that in the TLS renegotiation vulnerability that cropped up in late 2009. More recently, a somewhat similar problem was discovered in the Postfix mail transfer agent (MTA) (as well as other mail server software and MTAs). The problem lies in improperly handling the transition between states such that the boundaries that should exist between them are not enforced.

The problem came to light in a lengthy post on March 7 to the postfix-users mailing list by Postfix creator Wietse Venema, but he had discovered it back in January. Venema silently fixed the problem in Postfix at that time, and then worked with CERT to coordinate fixes for other affected projects and vendors. That work has been completed, so the details are now being made public.

The problem occurs when Postfix changes from unencrypted to encrypted mode via the STARTTLS SMTP command. Venema goes into some detail about how Postfix makes that switch (at least at a high level), but the basic flaw is that it doesn't flush its buffers after it switches over to the TLS encrypted mode. That allows a man-in-the-middle attacker to inject some plaintext commands into the SMTP data stream between the STARTTLS and the client's first TLS-encrypted SMTP commands.

Venema demonstrates the problem using the OpenSSL s_client command with a minor modification. Using that program, one can easily test for the problem. In the example, the STARTTLS command is followed directly by the RSET command, which just resets the state (sender and receiver addresses for instance) of any in-progress mail transaction. In an affected MTA, the TLS negotiation will take place, so that subsequent traffic is encrypted, but the server will read the buffered RSET command even though it was sent prior to the establishment of the TLS session. But, since the server believes it is in encrypted mode, it treats the RSET as being in that context.

Obviously, a RSET is not particularly harmful. There are other things an attacker could do, as Venema mentions:

The exploits would look similar to those described for the TLS renegotiation flaw. If the attacker can predict what commands a client will send (which isn't terribly difficult at least for SMTP), they can prefix their own set of commands and relay the server responses to the victim. Typically, the attacker commands will leave the server in a kind of dangling state waiting for the client to send data that will complete the commands. The classic example is for the attacker to send the SMTP DATA command after setting the from and to addresses appropriately; all of the commands the client sends are then included into the email that gets sent to the attacker.

Clients that don't check the TLS certificate are, in some sense, unaffected by this problem. They are always vulnerable to man-in-the-middle attacks that don't need to rely upon tricks like this plaintext injection. But clients that do check those certificates were at risk. Given that even security-conscious users are much more inclined to use a random open WiFi access point because they are using encrypted communications, this vulnerability could be used to capture a lot of outgoing mail—or worse.

Venema also points out that part of the problem in Postfix was that it was written to adhere to the "robustness principle" (aka Postel's Law): "Be conservative in what you send; be liberal in what you accept". The idea is that protocol implementations should strive to only send compliant messages, but to accept non-compliant messages from others when the intent is clear. Venema puts it this way:

Sadly in some ways, the robustness principle has been generally deprecated over the years as attackers (and other malicious entities such as spammers) have exploited the liberal acceptance of messages to further their aims. It has also allowed ill-behaved programs to continue to exist well past the time they should have been fixed. Strict protocol compliance in both directions is now the norm.

The full message from Venema is well worth reading as it provides many more details than other advisories generally do. It should also be something of a wakeup call to other developers of servers that switch between contexts (either switching from plaintext to encrypted or by encryption renegotiation). Looking closely at those transitions might just turn up a hole or two. Hopefully if bugs like that do get discovered, the developers will put out an advisory as informative as Venema's.

[ We would like to thank Brad Hards for giving us a heads-up about this issue. ]

Comments (none posted)

Security quotes of the week

-- ITworld (seen at Boing Boing)

-- Brad Spengler

-- Nmap developer Fyodor reads some HB Gary emails (the whole post is worth reading for its amusement value)

Comments (none posted)

Your passwords have been hashed

In a long-overdue upgrade, we have recently switched over to storing hashed passwords in our database. We have occasionally been taken to task (and deservedly so) for not doing that, and have finally gotten around to implementing a bcrypt-based hash for passwords. When the LWN site code was first implemented, passwords seemed like a pretty low-security item—there just wasn't much that an attacker could do if they got access to one—and the ability to remind users of their passwords seemed useful. Over time, though, it has become clear that password reuse can make the compromise of even "low security" passwords into a serious problem. Sites like ours clearly should not store passwords in plain text; we are now happy to say that we do not.

The only user-visible side of the change is in the username/password recovery process as we can no longer send you your password in email (at least we hope that's the only user-visible part, the rest should just be working invisibly in the background). Username and password recovery have been added to the Login page in case you ever need them. One thing to note, however, is that none of it will work unless we have an up-to-date email address for you in our database. We don't send very much email that you haven't requested (essentially just subscription reminders) and we definitely will not share your email address with anyone else, so please check your address via the My Account page, and update it if necessary.

Comments (3 posted)

aaa_base: arbitrary file corruption

Package(s):	aaa_base	CVE #(s):	CVE-2011-0461
Created:	March 14, 2011	Updated:	May 17, 2011
Description:	From the openSUSE advisory: The boot.localfs init script wrote a file to /dev/shm during shut-down. Since local users may create symlinks there a malicious user could cause corruption of arbitrary files.
Alerts:	SUSE SUSE-SR:2011:005 hplip, perl, subversion, t1lib, bind, tomcat5, tomcat6, avahi, gimp, aaa_base, build, libtiff, krb5, nbd, clamav, aaa_base, flash-player, pango, openssl, subversion, postgresql, logwatch, libxml2, quagga, fuse, util-linux 2011-04-01 SUSE SUSE-SR:2011:009 mailman, openssl, tgt, rsync, vsftpd, libzip1/libzip-devel, otrs, libtiff, kdelibs4, libwebkit, libpython2_6-1_0, perl, pure-ftpd, collectd, vino, aaa_base, exim 2011-05-17 openSUSE openSUSE-SU-2011:0171-1 aaa_base 2011-03-14

Comments (none posted)

asterisk: multiple vulnerabilities

Package(s):	asterisk	CVE #(s):
Created:	March 10, 2011	Updated:	March 16, 2011
Description:	From the Fedora advisory: The release of Asterisk 1.6.2.17 resolves several issues reported by the community and would have not been possible without your participation. Thank you!
Alerts:	Fedora FEDORA-2011-2558 asterisk 2011-03-03 Fedora FEDORA-2011-2438 asterisk 2011-03-02

Comments (none posted)

build: unsafe use of cpio

Package(s):	build	CVE #(s):	CVE-2010-4226
Created:	March 15, 2011	Updated:	April 18, 2011
Description:	From the openSUSE advisory: The build script uses cpio to extract untrusted rpm packages for bootstrapping virtual machines. cpio is not safe to use for this task, therefore the build script now uses bsdtar instead
Alerts:	SUSE SUSE-SR:2011:005 hplip, perl, subversion, t1lib, bind, tomcat5, tomcat6, avahi, gimp, aaa_base, build, libtiff, krb5, nbd, clamav, aaa_base, flash-player, pango, openssl, subversion, postgresql, logwatch, libxml2, quagga, fuse, util-linux 2011-04-01 openSUSE openSUSE-SU-2011:0174-1 build 2011-03-15

Comments (2 posted)

cgit: denial of service

Package(s):	cgit	CVE #(s):	CVE-2011-1027
Created:	March 16, 2011	Updated:	March 16, 2011
Description:	The cgit server can be thrown into an infinite loop by a remote attacker using a specially crafted URL.
Alerts:	Fedora FEDORA-2011-2803 cgit 2011-03-07 Fedora FEDORA-2011-2815 cgit 2011-03-07

Comments (none posted)

chromium-browser: multiple vulnerabilities

Package(s):	chromium-browser	CVE #(s):	CVE-2011-1108 CVE-2011-1109 CVE-2011-1113 CVE-2011-1114 CVE-2011-1115 CVE-2011-1121 CVE-2011-1122
Created:	March 10, 2011	Updated:	March 16, 2011
Description:	From the Debian advisory: CVE-2011-1108: Google Chrome before 9.0.597.107 does not properly implement JavaScript dialogs, which allows remote attackers to cause a denial of service (application crash) or possibly have unspecified other impact via a crafted HTML document. CVE-2011-1109: Google Chrome before 9.0.597.107 does not properly process nodes in Cascading Style Sheets (CSS) stylesheets, which allows remote attackers to cause a denial of service or possibly have unspecified other impact via unknown vectors that lead to a "stale pointer." CVE-2011-1113: Google Chrome before 9.0.597.107 on 64-bit Linux platforms does not properly perform pickle deserialization, which allows remote attackers to cause a denial of service (out-of-bounds read) via unspecified vectors. CVE-2011-1114: Google Chrome before 9.0.597.107 does not properly handle tables, which allows remote attackers to cause a denial of service or possibly have unspecified other impact via unknown vectors that lead to a "stale node." CVE-2011-1115: Google Chrome before 9.0.597.107 does not properly render tables, which allows remote attackers to cause a denial of service or possibly have unspecified other impact via unknown vectors that lead to a "stale pointer." CVE-2011-1121: Integer overflow in Google Chrome before 9.0.597.107 allows remote attackers to cause a denial of service or possibly have unspecified other impact via vectors involving a TEXTAREA element. CVE-2011-1122: The WebGL implementation in Google Chrome before 9.0.597.107 allows remote attackers to cause a denial of service (out-of-bounds read) via unspecified vectors, aka Issue 71960.
Alerts:	Debian DSA-2189-1 chromium-browser 2011-03-10

Comments (none posted)

chromium-browser: multiple vulnerabilities

Package(s):	chromium-browser	CVE #(s):	CVE-2011-0779 CVE-2011-1290
Created:	March 15, 2011	Updated:	March 16, 2011
Description:	From the Debian advisory: CVE-2011-0779: Google Chrome before 9.0.597.84 does not properly handle a missing key in an extension, which allows remote attackers to cause a denial of service (application crash) via a crafted extension. CVE-2011-1290: Integer overflow in WebKit allows remote attackers to execute arbitrary code via unknown vectors, as demonstrated by Vincenzo Iozzo, Willem Pinckaers, and Ralf-Philipp Weinmann during a Pwn2Own competition at CanSecWest 2011.
Alerts:	Debian DSA-2192-1 chromium-browser 2011-03-15

Comments (none posted)

kernel-rt: multiple vulnerabilities

Package(s):	kernel-rt	CVE #(s):	CVE-2010-4250 CVE-2010-4648
Created:	March 11, 2011	Updated:	September 13, 2011
Description:	From the Red Hat advisory: Memory leak in the inotify_init() system call. In some cases, it could leak a group, which could allow a local, unprivileged user to eventually cause a denial of service. (CVE-2010-4250) A logic error in orinoco_ioctl_set_auth() in the Linux kernel's ORiNOCO wireless extensions support implementation could render TKIP countermeasures ineffective when it is enabled, as it enabled the card instead of shutting it down. (CVE-2010-4648)
Alerts:	Oracle ELSA-2013-1645 kernel 2013-11-26 Ubuntu USN-1394-1 Linux kernel (OMAP4) 2012-03-07 Ubuntu USN-1256-1 linux-lts-backport-natty 2011-11-09 Ubuntu USN-1243-1 linux 2011-10-25 Ubuntu USN-1242-1 linux-lts-backport-maverick 2011-10-25 Red Hat RHSA-2011:1253-01 kernel-rt 2011-09-12 SUSE SUSE-SA:2011:017 kernel 2011-04-18 openSUSE openSUSE-SU-2011:0346-1 kernel 2011-04-18 Red Hat RHSA-2011:0421-01 kernel 2011-04-07 Red Hat RHSA-2011:0330-01 kernel-rt 2011-03-10 Red Hat RHSA-2011:0498-01 kernel 2011-05-10 openSUSE openSUSE-SU-2011:0399-1 kernel 2011-04-28

Comments (none posted)

krb5: denial of service

Package(s):	krb5	CVE #(s):	CVE-2011-0284
Created:	March 16, 2011	Updated:	April 1, 2011
Description:	The krb5 key distribution center suffers from a double-free flaw which can be exploited by a remote attacker to cause the daemon to abort.
Alerts:	Gentoo 201201-13 mit-krb5 2012-01-23 SUSE SUSE-SR:2011:005 hplip, perl, subversion, t1lib, bind, tomcat5, tomcat6, avahi, gimp, aaa_base, build, libtiff, krb5, nbd, clamav, aaa_base, flash-player, pango, openssl, subversion, postgresql, logwatch, libxml2, quagga, fuse, util-linux 2011-04-01 Fedora FEDORA-2011-3464 krb5 2011-03-16 Fedora FEDORA-2011-3462 krb5 2011-03-16 Ubuntu USN-1088-1 krb5 2011-03-15 Red Hat RHSA-2011:0356-01 krb5 2011-03-16

Comments (none posted)

libvpx: denial of service

Package(s):	libvpx	CVE #(s):	CVE-2010-4489
Created:	March 14, 2011	Updated:	June 19, 2012
Description:	From the Ubuntu advisory: Chris Evans discovered that libvpx did not properly perform bounds checking. If an application using libvpx opened a specially crafted WebM file, an attacker could cause a denial of service.
Alerts:	openSUSE openSUSE-SU-2012:0759-1 libvpx 2012-06-19 Ubuntu USN-1087-1 libvpx 2011-03-11

Comments (none posted)

openldap: multiple vulnerabilities

Package(s):	openldap	CVE #(s):	CVE-2011-1024 CVE-2011-1025 CVE-2011-1081
Created:	March 11, 2011	Updated:	September 26, 2011
Description:	From the Red Hat advisory: A flaw was found in the way OpenLDAP handled authentication failures being passed from an OpenLDAP slave to the master. If OpenLDAP was configured with a chain overlay and it forwarded authentication failures, OpenLDAP would bind to the directory as an anonymous user and return success, rather than return failure on the authenticated bind. This could allow a user on a system that uses LDAP for authentication to log into a directory-based account without knowing the password. (CVE-2011-1024) It was found that the OpenLDAP back-ndb back end allowed successful authentication to the root distinguished name (DN) when any string was provided as a password. A remote user could use this flaw to access an OpenLDAP directory if they knew the value of the root DN. Note: This issue only affected OpenLDAP installations using the NDB back-end, which is only available for Red Hat Enterprise Linux 6 via third-party software. (CVE-2011-1025) A flaw was found in the way OpenLDAP handled modify relative distinguished name (modrdn) requests. A remote, unauthenticated user could use this flaw to crash an OpenLDAP server via a modrdn request containing an empty old RDN value. (CVE-2011-1081)
Alerts:	Gentoo 201406-36 openldap 2014-06-30 Fedora FEDORA-2011-3627 openldap 2011-03-19 Pardus 2011-76 openldap-server 2011-05-26 SUSE SUSE-SR:2011:007 NetworkManager, OpenOffice_org, apache2-slms, dbus-1-glib, dhcp/dhcpcd/dhcp6, freetype2, kbd, krb5, libcgroup, libmodplug, libvirt, mailman, moonlight-plugin, nbd, openldap2, pure-ftpd, python-feedparser, rsyslog, telepathy-gabble, wireshark 2011-04-19 openSUSE openSUSE-SU-2011:0363-1 openldap2 2011-04-18 openSUSE openSUSE-SU-2011:0359-1 openldap2 2011-04-18 openSUSE openSUSE-SU-2011:0356-1 openldap2 2011-04-18 CentOS CESA-2011:0346 openldap 2011-04-14 Ubuntu USN-1100-1 openldap, openldap2.3 2011-03-31 Mandriva MDVSA-2011:055 openldap 2011-03-30 Mandriva MDVSA-2011:056 openldap 2011-03-30 Red Hat RHSA-2011:0346-01 openldap 2011-03-10 Red Hat RHSA-2011:0347-01 openldap 2011-03-10

Comments (none posted)

perl-mail-box: boundary guessing

Package(s):	perl-Mail-Box	CVE #(s):
Created:	March 11, 2011	Updated:	March 16, 2011
Description:	From the Fedora advisory: 2.097 now randomizes boundary string to avoid (undemonstrated) security risks with boundary guessing.
Alerts:	Fedora FEDORA-2011-2531 perl-File-FcntlLock 2011-03-03 Fedora FEDORA-2011-2531 perl-Mail-Box 2011-03-03 Fedora FEDORA-2011-2541 perl-Mail-Box 2011-03-03

Comments (none posted)

php-zendframework: cross-site scripting

Package(s):	php-ZendFramework	CVE #(s):
Created:	March 14, 2011	Updated:	March 16, 2011
Description:	From the Zend Framework advisory: The default error handling view script generated using Zend_Tool failed to escape request parameters when run in the "development" configuration environment, providing a potential XSS attack vector.
Alerts:	Fedora FEDORA-2011-2678 php-ZendFramework 2011-03-05 Fedora FEDORA-2011-2689 php-ZendFramework 2011-03-05

Comments (none posted)

pidgin: denial of service

Package(s):	pidgin	CVE #(s):	CVE-2011-1091
Created:	March 14, 2011	Updated:	November 10, 2011
Description:	From the Red Hat bugzilla: Multiple NULL pointer dereference flaws were found in the way Yahoo protocol plug-in of the Pidgin instant messaging client handled malformed YMSG packets (SMS messages and notification packets). A remote, authenticated user could use this flaw to cause denial of service (Pidgin crash) via specially-crafted notification message. The SMS messages handling issue is exploitable only via specially-crafted SMS message, sent from remote, malicious Yahoo server.
Alerts:	openSUSE openSUSE-SU-2012:0066-1 pidgin 2012-01-09 Ubuntu USN-1273-1 pidgin 2011-11-21 CentOS CESA-2011:1371 pidgin 2011-11-09 CentOS CESA-2011:1371 pidgin 2011-10-14 Scientific Linux SL-pidg-20111013 pidgin 2011-10-13 Red Hat RHSA-2011:1371-01 pidgin 2011-10-13 Pardus 2011-59 pidgin 2011-03-22 Mandriva MDVSA-2011:050 pidgin 2011-03-21 Fedora FEDORA-2011-3132 pidgin 2011-03-11 Fedora FEDORA-2011-3113 pidgin 2011-03-11 Red Hat RHSA-2011:0616-01 pidgin 2011-05-19

Comments (none posted)

vsftpd: denial of service

Package(s):	vsftpd	CVE #(s):	CVE-2011-0762
Created:	March 10, 2011	Updated:	October 11, 2011
Description:	From the CVE entry: The vsf_filename_passes_filter function in ls.c in vsftpd before 2.3.3 allows remote authenticated users to cause a denial of service (CPU consumption and process slot exhaustion) via crafted glob expressions in STAT commands in multiple FTP sessions, a different vulnerability than CVE-2010-2632.
Alerts:	Gentoo 201110-07 vsftpd 2011-10-10 Debian DSA-2305-1 vsftpd 2011-09-19 CentOS CESA-2011:0337 vsftpd 2011-04-14 Ubuntu USN-1098-1 vsftpd 2011-03-29 SUSE SUSE-SR:2011:009 mailman, openssl, tgt, rsync, vsftpd, libzip1/libzip-devel, otrs, libtiff, kdelibs4, libwebkit, libpython2_6-1_0, perl, pure-ftpd, collectd, vino, aaa_base, exim 2011-05-17 openSUSE openSUSE-SU-2011:0435-1 vsftp 2011-05-06 Mandriva MDVSA-2011:049 vsftpd 2011-03-21 Fedora FEDORA-2011-2615 vsftpd 2011-03-04 Fedora FEDORA-2011-2590 vsftpd 2011-03-04 CentOS CESA-2011:0337 vsftpd 2011-03-10 Red Hat RHSA-2011:0337-01 vsftpd 2011-03-09

Comments (none posted)

wireshark: denial of service

Package(s):	wireshark	CVE #(s):	CVE-2011-1143 CVE-2011-1138
Created:	March 14, 2011	Updated:	April 19, 2011
Description:	From the CVE entries: epan/dissectors/packet-ntlmssp.c in the NTLMSSP dissector in Wireshark before 1.4.4 allows remote attackers to cause a denial of service (NULL pointer dereference and application crash) via a crafted .pcap file. (CVE-2011-1143) Off-by-one error in the dissect_6lowpan_iphc function in packet-6lowpan.c in Wireshark 1.4.0 through 1.4.3 on 32-bit platforms allows remote attackers to cause a denial of service (application crash) via a malformed 6LoWPAN IPv6 packet. (CVE-2011-1138)
Alerts:	Oracle ELSA-2013-1569 wireshark 2013-11-26 CentOS CESA-2012:0509 wireshark 2012-04-24 Oracle ELSA-2012-0509 wireshark 2012-04-23 Scientific Linux SL-wire-20120423 wireshark 2012-04-23 Red Hat RHSA-2012:0509-01 wireshark 2012-04-23 Gentoo 201110-02 wireshark 2011-10-09 SUSE SUSE-SR:2011:007 NetworkManager, OpenOffice_org, apache2-slms, dbus-1-glib, dhcp/dhcpcd/dhcp6, freetype2, kbd, krb5, libcgroup, libmodplug, libvirt, mailman, moonlight-plugin, nbd, openldap2, pure-ftpd, python-feedparser, rsyslog, telepathy-gabble, wireshark 2011-04-19 openSUSE openSUSE-SU-2011:0347-1 wireshark 2011-04-18 CentOS CESA-2011:0370 wireshark 2011-04-14 CentOS CESA-2011:0370 wireshark 2011-03-22 Red Hat RHSA-2011:0370-01 wireshark 2011-03-21 Pardus 2011-57 wireshark 2011-03-21 Fedora FEDORA-2011-2620 wireshark 2011-03-04 Fedora FEDORA-2011-2632 wireshark 2011-03-04

Comments (none posted)

wordpress: multiple vulnerabilities

Package(s):	wordpress	CVE #(s):	CVE-2011-0700 CVE-2011-0701
Created:	March 11, 2011	Updated:	September 18, 2012
Description:	From the Debian advisory: CVE-2011-0700: Input passed via the post title when performing a "Quick Edit" or "Bulk Edit" action and via the "post_status", "comment_status", and "ping_status" parameters is not properly sanitised before being used. Certain input passed via tags in the tags meta-box is not properly sanitised before being returned to the user. CVE-2011-0701: Wordpress incorrectly enforces user access restrictions when accessing posts via the media uploader and can be exploited to disclose the contents of e.g. private or draft posts.
Alerts:	Fedora FEDORA-2011-3746 wordpress 2011-03-21 Fedora FEDORA-2011-3738 wordpress 2011-03-21 Debian DSA-2190-1 wordpress 2011-03-11

Comments (none posted)

Kernel release status

The 2.6.38 kernel is out, released by Linus on March 14. "As to the "big picture", ie all the changes since 2.6.37, my personal favorite remains the VFS name lookup changes. They did end up causing some breakage, and Al has made it clear that he wants more cleanups, but on the whole I think it was surprisingly smooth." Other significant changes in 2.6.38 include transparent hugepage support, per-session group scheduling, a number of Btrfs improvements, and more. The always excellent KernelNewbies.org page has all the details.

Stable updates: the 2.6.37.4 and 2.6.32.33 updates were released on March 14. Both contain several important fixes.

Comments (none posted)

Quotes of the week

-- Linus Torvalds

-- Steven Rostedt

-- Hugh Dickins

Comments (none posted)

Schultz: Diving into the Linux Networking Stack, Part I

Michael Schultz has posted an introductory look at the Linux networking stack, focusing on driver initialization and packet reception. It's a "how it works" discussion, rather than a look at the actual code. "In general network drivers follow a fairly typical route in processing: the kernel boots up, initializes data structures, sets up some interrupt routines, and tells the network card where to put packets when they are received. When a packet is actually received, the card signals the kernel causing it to do some processing and then cleans up some resources. I'll talk about the fairly generic routines that network devices share in common and then move to a concrete example with the igb driver."

Comments (none posted)

A group scheduling demonstration

There has been much talk of the per-session group scheduling patch which is part of the 2.6.38 kernel, but it can be hard to see that code in action if one isn't doing a 20-process kernel build at the time. Recently, your editor inadvertently got a demonstration of group scheduling thanks to some unexpected results from a Rawhide system upgrade. The way the scheduler works was clearly shown in a way that could be captured at the time.

Rawhide users know that surprises often lurk behind the harmless-looking yum upgrade command. In this particular case, something in the upgrade (related to fonts, possibly) caused every graphical process in the system to decide that it was time to do some heavy processing. The result can be seen in this output from the top command:

The per-session heuristic had put most of the offending processes into a single control group, with the effect that they were mostly competing against each other for CPU time. These processes are, in the capture above, each currently getting 5.3% of the available CPU time. Two processes which were not in that control group were left essentially competing for the second core in the system; they each got 46%. The system had a load average of almost 22, and the desktop was entirely unresponsive. But it was possible to log into the system over the net and investigate the situation without really even noticing the load.

This isolation is one of the nicest features of group scheduling; even when a large number of processes go totally insane, their ability to ruin life for other tasks on the machine is limited. That, alone, justifies the cost of this feature.

Comments (19 posted)

2.6.39 merge window part 1

Linus released the 2.6.38 kernel on March 14, and started merging patches for the 2.6.39 development cycle the following day. As of this writing, just over 1,000 patches have been merged into the mainline. Clearly the merging process has just begun for this cycle, but some interesting features have been added. User-visible changes merged so far include:

• The open by handle system calls have been added. The final form of the API is:

      int name_to_handle_at(int dfd, const char *name, struct file_handle *handle,
  			  int *mnt_id, int flag);
      int open_by_handle_at(int dirfd, struct file_handle *handle, int flags);
  

  This functionality is intended for use by user-space file servers, which can more efficiently track files using file handles.

• The open() system call has a new flag: O_PATH. A file opened with this flag will have had its path resolved by the kernel and is known to exist, but there is little else that can be done with it. System calls which operate on file descriptors directly (close() or dup(), for example) will work; these file descriptors can also be passed to another process over Unix-domain sockets using SCM_RIGHTS datagrams. The reason for the existence of O_PATH file descriptors is for use as the directory file descriptor in the various "*at()" system calls.

• Tasks in the SCHED_IDLE class are now allowed to upgrade themselves into the SCHED_BATCH or SCHED_OTHER classes if their "nice" rlimit is adequate.

• There is a new system call which allows the adjustment of POSIX clocks:

      int clock_adjtime(clock_id which_clock, struct timex *time);
  

  Time adjustments possible are the same as for adjtimex(), but specific POSIX clocks may not support all operations.

• The CLOCK_BOOTTIME POSIX clock has been added.

• The new Smack SMACK64MMAP attribute can be used to control when specific libraries can be mapped by running programs.

• New hardware support includes:
  • Systems and processors: Intel "SandyBridge" CPUs, CompuLab TrimSlice boards, and several variations of the Seaboard evaluation platform.

  • Block: ARASAN CompactFlash PATA controllers.

  • Miscellaneous: picoXcell IPSEC and Layer2 crypto engines.

Changes visible to kernel developers include:

• There is a new interrupt flag (IRQF_FORCE_RESUME) which forces the interrupt to be re-enabled at resume time regardless of whether it was disabled during suspend.

• The kernel can now force (almost) all interrupt handlers to be run in threads; this capability is controlled with the threadirqs command line option. This is a useful debugging feature, as a crashing interrupt handler will, when running in a thread, merely cause a kernel oops instead of bringing down the whole system. Interrupt handlers which should never be forced into threads can be marked with IRQF_NO_THREAD, but its use is expected to be rare.

• The object debugging infrastructure now allows the specification of a "debug hint" function; it returns an address which can be used to better identify a specific object. See this commit for details.

• The long-deprecated SPIN_LOCK_UNLOCKED and RW_LOCK_UNLOCKED lock initializers have been removed.

• The perf events subsystem has a new monitoring mode wherein it only watches processes belonging to a specific control group. The new -G option to perf provides access to this functionality.

• The directed yield feature has been added to the fair scheduler; this feature should improve performance for guests virtualized with KVM.

• There is a new mechanism for the dynamic addition of POSIX clocks; see <linux/posix_clock.h> for the details of the interface.

• The x86 architecture has gained minimal device tree support.

• There is a new global workqueue called system_freezable_wq; it differs from the others in that it can be frozen at suspend time.

• Core subsystems can make use of the new syscore_ops mechanism to register power management callbacks without the need to create otherwise useless system devices.

If the usual rules apply, the 2.6.39 merge window can be expected to close around March 29, and the 2.6.39 release should happen around the first week of June.

Comments (5 posted)

Uprobes: 11th time is the charm?

Last week's Kernel Page included an article about improving the ptrace() interface; the author of that work, Tejun Heo, was quoted as saying that part of the problem with ptrace() is that it has been starved of developer attention in favor of efforts to replace it entirely. One of those efforts is uprobes, which has also been featured on this page a few times. A new uprobes patch was posted on March 14; so this seems like a good time to have a look at it and further deprive ptrace() of attention. Uprobes looks like it is getting closer to acceptance, but it seems unlikely that the 11th revision will be the last.

The purpose of the uprobes subsystem is what one might expect: to enable the placement of probes into user-space executable process memory. These probes might be used to support a debugger like gdb (though uprobes is said to be unsuitable for use by gdb in its current form) or to support user-space tracing. This feature does thus duplicate some of the functionality provided by ptrace(), which will make its acceptance harder, especially since ptrace() is (more or less) a standardized interface. To succeed, uprobes will clearly have to do things better than ptrace() does.

The ptrace() interface is tied to processes; uprobes, instead, works with files. A probe is placed at a certain offset within a specific file; it will then trigger for every process which executes through the probe's location. If the code placing the probe is only interested in specific processes, it will need to filter the events itself. The interface may seem a little strange - users will probably almost always be interested in specific processes - but there are some advantages to doing things this way.

Underneath the hood, uprobes works by faulting in the page which will contain the probe. The instruction at the probe location is copied aside and replaced by a breakpoint. Every process which has that file mapped then gets a pointer in its mm structure pointing to the data describing the probe(s) for that file. When a process executes the breakpoint, the probe's handler function will be called; on that handler's return, the kernel will single-step the displaced instruction, then return to the location following the probe.

This "execute out of line" (XOL) mechanism has been controversial in the past because it requires the injection of a new virtual memory area (VMA) into every process which encounters probes. That VMA is seen as a distortion of the process's behavior which could have strange effects. The alternatives, though, are not entirely appealing either. The ptrace() approach is to put the original instruction back into its original location, execute it, then replace the breakpoint; that only works if every process which has the file mapped is stopped for the duration of the operation (otherwise they might execute the affected code while the breakpoint is missing). Uprobes, instead, is able to handle breakpoint hits without perturbing other processes. Another alternative discussed in the past is emulating the displaced instruction in the kernel; that requires having a full x86 emulator in kernel space, which is not entirely appealing either. So the current plan appears to be to stick with XOL.

Not having to stop the world when a breakpoint is hit is one of the advantages of uprobes, but there are others. It dispenses with the whole ptrace() mechanism involving signals, reparenting processes, and so on. Handling a probe hit does not require a context switch unless the probe itself does; many types of tracing tasks, for example, would never have to switch to another process. Uprobes also allows multiple applications to be tracing the same set of processes at the same time. All of these make the interface appealing to some users.

Who those users are is not clear to everybody, though. There is clearly some interest in the SystemTap camp, but the needs of SystemTap do not necessarily carry a lot of weight on linux-kernel. Thomas Gleixner put it this way:

At times, gdb developers have indicated that they might be open to using a Linux-specific interface if there were advantages to doing so. Such use seems distant at the moment, though. More immediate users are likely to be found in the tracing community; uprobes opens the possibility of getting single stream of trace data covering both user and kernel space. ptrace() is not a useful interface for tracing, so something needs to be done (though there is still some disagreement over whether user-space tracing needs to involve the kernel at all). Uprobes might be that something.

In fact, this version of the uprobes patch includes an ftrace-based interface. Part 20 of the patch contains the entirety of the documentation for this feature, quoted below:

    # cd /sys/kernel/debug/tracing/
    # cat /proc/`pgrep  zsh`/maps | grep /bin/zsh | grep r-xp
    00400000-0048a000 r-xp 00000000 08:03 130904 /bin/zsh
    # objdump -T /bin/zsh | grep -w zfree
    0000000000446420 g    DF .text  0000000000000012  Base        zfree
    # echo 'p /bin/zsh:0x46420 %ip %ax' > uprobe_events
    # cat uprobe_events
    p:uprobes/p_zsh_0x46420 /bin/zsh:0x0000000000046420
    # echo 1 > events/uprobes/enable
    # sleep 20
    # echo 0 > events/uprobes/enable
    # cat trace

An actual document is listed as a "TODO" item. The current interface looks a bit painful to use, and it appears to be limited to printing register contents for now. A more flexible and better documented interface could prove useful, though, especially if (as planned) it also can be made to work with the perf events subsystem.

The comments on the patch set indicate some concern about whether the kernel needs the feature or not. But even the more critical reviewers have been going over the code pointing out small things - the kind of review one does when one wants to help the author get the code into shape for merging. This code will not be merged for 2.6.39, and, for this type of code, making predictions for merging at any definite time is a hazardous affair. But, given sufficient will, it seems like uprobes could be made ready for inclusion sometime this year.

Comments (2 posted)

APIs for sensors

Environmental sensors were, once upon a time, equipment which were only found in specialized settings like industrial process control or scientific research. They were expensive and tuned to a specific task. Increasingly, though, sensors are being attached to all kinds of devices. Mobile handsets have compasses, accelerometers, and more. Sensors for temperature, pressure, etc. are becoming increasingly common as well. The implications are fun; any Linux machine can be a versatile data collection device.

The only problem with all of this is that the Linux kernel does not yet have an established API - either internal or to user space - for sensors. There are interfaces for specific types of sensors; Video4Linux2 handles cameras, for example, and the hwmon subsystem deals with the specific class of sensors aimed at monitoring the health of the computer itself. In these areas, the interfaces are well established and interoperation is possible. For sensors which fall outside of these classes, though, there are no real rules. The outcome of this kind of situation is always the same: new devices are added with inconsistent interfaces, making life hard for application developers.

This situation came to light (again) with the recent submission of a pressure sensor driver which was implemented as a misc device. It used the input subsystem to present its interface; Jonathan Cameron, who has been working on sensor interfaces, pointed out that the patch would not be accepted in that form. Input devices are meant for human input; since most humans do not communicate with their systems via large ambient pressure changes, this device did not fit. So the driver needs another home. The hwmon subsystem was suggested, but the pressure sensor is not really a hardware monitor, so the driver is not welcome there either. Arnd Bergmann also does not like the use of the misc interface:

That leaves the industrial I/O (IIO) subsystem, which is meant "for devices that in some sense are analog to digital converters." IIO tries to handle a wide variety of sensors in some sort of standard way with support for events, higher bandwidth I/O, and more. There are quite a few drivers in the IIO subsystem now; the only problem is that the whole thing lives in the staging tree and the associated "TODO" list is reasonably long. The devices which are represented there now are not all consistent in their interface use - and the form of the desired interface is not at all clear.

Still, putting together such an interface is Jonathan's goal:

He adds that the interface and support for simple devices (those with slow data rates and hwmon-style sysfs interfaces) is in reasonably good shape. The question is how to get the rest of the job done.

One alternative would be to define an essentially new IIO core which would be merged into the mainline. Individual drivers could then be worked into shape and moved over once they are ready. The problem is that this could be a long process, and that the mainline versions of the drivers might not initially have all of the functionality of their black-sheep staging cousins. That would mean more maintenance work keeping both versions of the driver working for some time.

Still, that's the approach that Arnd recommends. The move to the mainline is the last good chance to define an interface which will then need to be supported for many years. So some pain now, if used properly, may be warranted in order to make life easier in the future. Getting driver developers to buy into this idea may not be entirely easy; most of them spend the bulk of their time doing something other than writing Linux driver code and may lack the desire to move to a new interface when what they have now works. But that's almost certainly the best way forward. Now is almost certainly a good time for people with an interest in this area to help in the development of the mainline version of the IIO interface.

Comments (14 posted)

Taking openSUSE 11.4 for a spin

When I worked for Novell as the openSUSE community manager, I ran openSUSE more or less exclusively (modulo a few installs of Fedora and Ubuntu to see what other distros were up to). Since I left last year, I resumed distribution-hopping and haven't spent as much time with openSUSE. With some time away, I was looking forward to trying 11.4 and seeing what had changed. The answer? Not very much.

The openSUSE installation routine has changed very little since 11.0. In fact, very little specific to openSUSE as a distribution has changed since the last release or even 11.2. The 11.4 release carries plenty of software updates — KDE 4.6, GNOME 2.32.2.1, the 2.6.37 Linux kernel, Xfce 4.8, and so on — but very little in the way of improvements to distribution-specific features and tools. Consider, for instance, the improvements that have been made for the Ubuntu software Center or Linux Mint's menu in the last few releases — openSUSE has been fairly static. The Libzypp backend for YaST and Zypper package management has picked up a few tricks, WebYaST has matured from its status as a "technical preview" in 11.3, but openSUSE 11.4 really contains few major updates specific to the distribution itself.

That said, there's still plenty to like about openSUSE 11.4, especially for KDE users. If you want a smörgåsbord of desktops on one disk, look no further than the openSUSE 11.4 DVD. Users have the latest KDE, two GNOMEs, Xfce, and LXDE to choose from, along with a few assorted window managers as well.

The KDE 4.6 experience is well-polished, and carries the improvements that come with 4.6 like KDE's Activities, improvements in the Dolphin file manager, and so on. Given the eight month release cycle, it means that openSUSE has jumped two major KDE releases with 11.4 — the openSUSE 11.3 release shipped with KDE 4.4. Even with KDE as the default desktop and a strong affinity for the KDE Project, the openSUSE team picks and chooses its applications based on the best tool for the job rather than defaulting to KDE applications for all tasks. For example, the default browser for all desktops on openSUSE 11.4 is Firefox 4.0beta12 (which will update as it approaches release). The project chooses GIMP for photo editing, rather than Krita, and LibreOffice is the default office suite for GNOME and KDE, rather than KOffice. Speaking of LibreOffice, openSUSE is the first of the major distros to ship LibreOffice in a stable release. Not surprising, given that some of the folks behind LibreOffice are involved with openSUSE and/or SUSE Linux development.

Where KDE has made a significant leap forward in 11.4, the default GNOME desktop in openSUSE 11.4 has evolved very little since 11.3 or even 11.2. This is due in part to GNOME's focus on GNOME 3.0 and the eight month release cycle for openSUSE. Since the openSUSE folks are shipping GNOME stable, they're stuck with the 2.32 release from last September. It's still a usable desktop, but it's fairly calcified. Novell seems to have largely lost interest in the SUSE Linux Enterprise Desktop (SLED) business, or investing significant development in GNOME — and that's reflected in openSUSE 11.4. This is not to say that Novell is not investing in GNOME, but it seems the heyday of desktop innovation from Novell has passed.

Many of the openSUSE/Novell-driven packages in GNOME are showing signs of neglect as well. Take, for instance Evolution, the Slab menu, and Tasque. The Slab menu was driven by Novell and maintained for SUSE Linux Enterprise Desktop (SLED), which doesn't seem to be a major focus for the company at this point. Assuming the openSUSE folks adopt GNOME Shell with the next release, the menu has probably reached its end of life. Tasque (a simple to-do manager) was a project started during one of the Novell hackweeks, and has mostly stagnated after an initial burst of activity. Evolution, since it is an upstream part of GNOME, is no longer Novell's responsibility alone — but from the planning pages for Evolution it appears that little in Evolution is being addressed until after GNOME 3.0.

GNOME 3.0 is not entirely absent from 11.4, however. The openSUSE 11.4 DVD (and repositories) have GNOME 3.0 packages that seem relatively current. When first installed, GNOME Shell on openSUSE 11.4 did not start properly — but an update seems to have fixed the problem and GNOME Shell and the GNOME 3.0 preview seem fairly solid at this point. I'm not entirely convinced yet that GNOME 3.0 is an improvement on 2.32, but it's there for users to try if they so desire. Those interested in GNOME 3 may also want to check out the Grumpy Editor's experience from this week's edition.

Packages and repositories

One thing that users may find off-putting about openSUSE is the number, or lack thereof, of packages available for the distribution and the lengths that one has to go to get some fairly popular packages. This is in part a real problem, and in part a problem of perception and education. If one knows where to look, it's often possible to find the software you want — but it involves a level of commitment.

One example is Dropbox. This is a relatively popular third-party package for many Linux users — but Dropbox only provides packages for Fedora and Ubuntu, not openSUSE. You can find Dropbox packages in the openSUSE Build Service (OBS), but it's going to require that a user know about the build service and where to look. It's not clear from running "Install/Remove Software" (which launches YaST's Software Manager) that you have options in OBS.

But Dropbox is a (partly) proprietary third-party application that openSUSE has little control over. A more glaring omission is the lack of some fairly popular free software tools, like Bluefish, OpenShot, and Snort. You'll also not find things like WordPress or Drupal packaged for openSUSE, whereas you'll find packages for those for Debian and Ubuntu. However, I don't fault the project for not focusing on LAMP-based projects that are not well-suited to installation and setup from packages.

This is compensated for by the openSUSE Build Service for some packages, and by the Packman project, which provides a number of packages for openSUSE that aren't in the "official" repositories. You'll find a number of packages in Packman, in particular multimedia packages that are not distributed with openSUSE due to legal concerns. You can find, say, Bluefish in the openSUSE Build Service, but it's less than clear which iteration of the package one should choose, as Bluefish is packaged in two different repositories and there's no way (shy of actually adding the repository and taking your chances) to determine which version is preferable.

In general openSUSE is not quite at par with other major distributions for package depth when installed. Users will probably need to do a bit more legwork and research to get the most out of openSUSE, which suits longtime SUSE fans just fine — but may not help its chances with users new to Linux.

Hardware support for openSUSE 11.4 is excellent, or at least it's excellent on the selection of hardware I tried it on. I tested openSUSE on two laptops and under VMware. The primary machine I used for testing is a ThinkPad X61 with 4GB of RAM. One of the things that I like about openSUSE is that the x86 release still has support for 4GB of RAM with a vanilla install — no need to install a different kernel to have support for 4GB or more on a 32-bit install. Running the 64-bit release is also an option, of course — but it's not unusual to get a 32-bit only DVD or CD from magazines that ship openSUSE as a free DVD.

Rollin', Rollin', Rollin'

The release of openSUSE 11.4 marks the "official" introduction of Tumbleweed, the rolling release version of openSUSE, which we covered back in December when it was announced. openSUSE is not the first distribution to attempt a "rolling" release, of course, but it is the first major distribution to do so. The idea for Tumbleweed is to provide openSUSE users with the latest stable releases of software, rather than requiring users to wait for major openSUSE releases or ride the Factory (development) distribution.

It's early days for Tumbleweed, but if the first updates are any indication, it may be best for the openSUSE project to avoid steering new users towards Tumbleweed. Using Tumbleweed as directed by the openSUSE wiki requires manually adding the Tumbleweed repository and doing a distribution upgrade. This triggered several package conflicts on my system. The Zypper error messages are not particularly user friendly, though nothing catastrophic happened when I went ahead with the update and choose to keep the obsolete packages.

During the 11.4 release cycle, Project Evergreen has also taken off. Led by Wolfgang Rosenauer, Evergreen is an attempt to continue updates for openSUSE releases past their official 18-month lifecycle. Support for 11.1 ended in December, and Evergreen has provided updates for users since. Whether it will be around in 18 months for users of 11.4 to depend on is another question. Rosenauer says that the project has been able to keep up with all 11.1 updates since December, but "the project is not at the level I'd have liked it to be though. The reason is that if I cannot manage it anymore for whatever reason it'll most likely die since there are no other driving persons behind it yet." Despite that, Rosenauer says that 11.4 is "a good candidate for the next supported version and I'm planning towards it. I hope that more active people are joining in during the next months." Status for Evergreen is available on the openSUSE wiki.

Even though the openSUSE Project has never formally decided what it wants to be when it grows up, it would seem that it's settled into a role all the same. openSUSE is an experienced Linux user's distribution, with an interesting collection of tools (like OBS, SUSE Studio, and Tumbleweed) for well-informed users and developers. Though openSUSE is well-polished and very stable, it has enough rough edges that I would hesitate to recommend it to users who have no or very little prior Linux experience.

But developers and experienced Linux users should feel right at home on openSUSE. The project has focused on a lot of under-the-hood features, infrastructure, and developer tools. It also seems to have picked up a bit of steam and enthusiasm in the community over the last year. While Novell's attention has been largely focused on its pending acquisition by Attachmate, the existing openSUSE community seems to have been revitalized a bit. While Novell's fate was in limbo, openSUSE enthusiasts realized that openSUSE's continued existence — or at least its continued improvement — requires more than just making requests of Novell.

But the project is still, at least for the time being, largely dominated by Novell — though significantly less so than just a few years ago. It may become even more independent if, or when, the foundation comes to fruition. The foundation is still in discussion and most of the discussions seem to have tapered off in February — most likely the victim of pending release pressure. Those interested in the topic have an open invitation to contribute.

The next openSUSE release should come out in November of this year, though it's unclear right now what features are being targeted and whether it will be another minor update or a major jump from 11.4. Judging by the interest in 11.4, the project seems to be regaining some momentum and 2011 could be a very interesting year for openSUSE.

Comments (13 posted)

Distribution quotes of the week

-- Josh Boyer

-- Chris Richards

-- Thomas Kahle

Comments (1 posted)

openSUSE 11.4 has arrived

The announcement for the openSUSE 11.4 release has gone out; it contains a lot of information about what can be found in this release. "We are proud to announce the launch of 11.4 in the openSUSE tradition of delivering the latest technology while maintaining stability. The 11.4 release brings significant improvements along with the latest in Free Software applications. Combined with the appearance of new tools, projects and services around the release, 11.4 marks a showcase of growth and vitality for the openSUSE Project!"

Comments (1 posted)

Mageia 1 Alpha 2

Mageia has released the second alpha for Mageia 1. "Again, this has been a very busy month: our build system is now running at full capacity and the list of available software has grown to over 6,100 SRPMS, including full integration of LibreOffice and Eclipse, rpm 4.8.1, KDE 4.6.1."

Comments (none posted)

Slackware 13.37 release candidate 2

The March 16, 2011 entry in the Slackware-current changelog (x86, x86_64) has a brief announcement that Slackware 13.37 rc2 is available for testing.

Comments (2 posted)

DPL elections 2011: Candidates

The nomination period is over for this year's Debian Project Leader election. There is only one candidate this year, Stefano 'Zack' Zacchiroli. His platform has been updated for the 2011 election.

Full Story (comments: none)

Bits from the Debian Installer Team

We haven't heard from the Debian Installer Team since before the 'squeeze' release, but there's quite a bit going on. These bits summarize the changes within the team and in the installer. Some include a move from Subversion to git, improvements in kfreebsd and hurd support, and, of course, new features for 'wheezy'. "The Debian Installer team are looking for more people to help. The installer development is a huge but quite pleasant task and nowadays it is centered on a small number of people. The right time to start contributing is "Right Now(TM)" as we're starting a new development cycle and many new and exciting features are being developed. Come and join us!"

Full Story (comments: none)

Bit from the SRMs: the "stable-updates" suite

Adam D. Barratt covers the replacement of the "volatile" archive with the "stable-updates" archive for Debian 6.0 (Squeeze). "The suite contains a subset of the packages available via the "proposed-updates" suite which many users may wish to be able to install without having to cherry-pick them or wait for the next point release."

Full Story (comments: none)

Mingle Today! openSUSE Connect Released

openSUSE Connect is a social networking site for openSUSE contributors. "Connecting faces to names, nicknames and to email addresses. In the end connecting all our other tools to this one so we can share the data. These are the fundamental ideas behind our newest openSUSE tool: connect"

Full Story (comments: none)

Distribution newsletters

• Debian Misc Developer News #26 (March 10)
• Debian Project News (March 14)
• DistroWatch Weekly, Issue 396 (March 14)
• Fedora Weekly News Issue 266 (March 9)
• openSUSE Weekly News Issue 166 (March 12)

Comments (none posted)

Top Linux Distros For Every Level User (Datamation)

Matt Hartley briefly covers several distributions and recommends: Gentoo, Arch and Slackware for advanced users; Fedora and Debian for intermediate users; and SimplyMEPIS, PCLinuxOS and openSUSE for newbies.

Comments (none posted)

Spotlight On Linux: CrunchBang (Linux Journal)

Michael Reed takes a quick look at CrunchBang. "CrunchBang is a lightweight Linux distribution based on Debian. It comes in OpenBox and XFCE editions, and a very dark visual theme. It's the OpenBox version that I took a look at."

Comments (none posted)

Version control with Fossil

Within a year or so after the nearly simultaneous debuts of Git, Mercurial, and Bazaar in 2005, another distributed version control system (DVCS) was introduced called Fossil. Unlike the other three, Fossil has maintained a much lower profile. A recent announcement that Tcl/Tk would be moving from the SourceForge CVS repositories to Fossil raised that profile a bit. So, what is Fossil and what distinguishes it from the other choices in the DVCS space?

Fossil was created by D. Richard Hipp, who also created SQLite, and its first release was in 2006. Since that time, a number of projects have started using Fossil for source code management, including, unsurprisingly, SQLite, but also the Mongrel2 web server, the PRITLOG blog system, and now Tcl/Tk. Others are undoubtedly using it as well, but as the project's "Questions and Criticism" page notes, "fossil does not yet have the massive user base of git or mercurial".

To start with, Fossil is more than just a DVCS, it also includes bug tracking, wiki, and blog, all of which are set up for distributed operation. It is, in some ways, an integrated project management system like Trac, but it has a different set of features that are meant to satisfy Hipp's requirements. He worked on an earlier project (CVSTrac) that inspired Trac, but Fossil is clearly an effort to scratch his particular set of itches. It would seem that other projects are starting to find that it works for them as well.

One of the main differences from Trac, and one that will be very familiar to DVCS users, is the idea of disconnected operation. But, beyond the usual ability to edit and commit code that a DVCS provides, Fossil allows editing bug tickets or changing the wiki locally, then synchronizing those changes to other repositories at a later time. Disconnected operation is the "killer feature" that Fossil provides over Trac, at least for Hipp's purposes.

But Fossil has some other characteristics that will be attractive to some. It is a single monolithic binary (fossil) that handles all of the source code management tasks, without using any external programs (like diff or patch). That binary also handles web requests for source code browsing, bug ticket handling, and repository synchronization. Because it is standalone, it can be easily installed in its own chroot() environment to isolate it from the rest of the system. Monolithic may give the wrong impression, however, as the binary is only around 800K in size.

All of the data is stored in SQLite (again, no surprise there) in what Hipp calls an enduring file format: "A fossil repository is intended to be readable, searchable, and extensible by people not yet born." The current implementation uses deltas and zlib-compressed blobs stored in the SQLite database. Like other DVCS programs, Fossil uses SHA1 hashes to identify the "artifacts" that are stored in the repository.

Unlike other DVCSs, Fossil has a repository that is stored separately from the working tree, rather than as hidden directories like the default for Git or Mercurial. Also unlike those systems, Fossil's repository is just a single SQLite file that can be easily copied or moved as needed. A working directory is associated with a specific Fossil repository (in a specific location), in some ways like the "origin" concept in Git.

Typically, each developer has their own repository on their local machine with one or more local source trees (or working directories) associated with it. Repositories can be synced with each other via HTTP. There are two different modes of operation for syncing, "manual-merge" mode (which is the way that Git and Mercurial work) or "autosync" mode (which is similar to how CVS and Subversion (SVN) work). One of the interesting aspects of Fossil is that it supports both of these modes.

In autosync mode, a commit essentially also does a push to the server where the code was cloned from or the one that was most recently used to sync. Then one uses the "update" command to pull the most recent changes into the local repository and to merge them into the local source tree. Manual-merge mode just decouples the commit from push, and update from pull so that users need to do each of those parts separately (and indeed can do those parts separately). The documentation says that the author believes autosync to be the proper default (in the "4.0 Workflow" section of the Fossil Concepts page):

Since many projects using DVCS are often running in disconnected mode (at least conceptually), it makes sense that Git, Mercurial, and others only support the manual-merge style. Fossil would seem to be targeted more at smaller projects, with fewer developers, possibly all working for the same company on a single project. In some ways, it is targeted at replacing CVS or SVN with a distributed tool while more-or-less preserving the workflow that those tools provide.

One thing that cannot be said about Fossil is that it lacks for documentation. There is voluminous documentation of Fossil including its design philosophy, a technical overview, a Quick Start Guide, a comparison to Git, and more, all found linked from the main Fossil project page. There is even an 87-page User Manual available in both PDF and Fossil repository format. There should be very few barriers to getting going with Fossil.

In some ways, Fossil sits in between the VCS and DVCS worlds. For projects that like the idea of keeping their bugs and wiki together with their code (and documentation), Fossil is definitely worth a look.

Comments (14 posted)

Quotes of the week

-- Guido van Rossum

-- Richard Stallman

-- Theo de Raadt

-- Wietse Venema

Comments (2 posted)

Drizzle 2011.03.13 GA released

Drizzle is a fork of the MySQL relational database manager; the 2011.02.13 release has been announced. This is a "general availability" release; new features include Sphinx-based documentation, log-based replication, a MySQL migration tool, Innodb tables by default, the "HailDB" engine, pluggable authentication, and more. "It's been a long and crazy road to get to this point and the team would like to thank everyone that has helped us get here. Every patch, bug report, and thought-provoking question has been invaluable in getting Drizzle this solid."

Comments (none posted)

FFmpeg fork becomes libav

The group of developers who took over maintainership of the FFmpeg project some months ago have belatedly decided that a rename is in order; the project will now be known as Libav. The project has also posted a set of rules about how maintainership is expected to work going forward.

Comments (19 posted)

GNU Free Call

The GNU Free Call project has announced its existence. "Our goal is to make GNU Free Call ubiquitous in a manner and level of usability similar to Skype, that is, usable on all platforms, and directly by the general public for all manner of secure communication between known and anonymous parties, but without requiring a central service provider to register with, without using insecure source secret binary protocols that may have back-doors, and without having network control points of any kind that can be exploited or abused by external parties. By doing so as a self organizing meshed calling network, we further eliminate potential service control points such as through explicit routing peers even if networks are isolated in civil emergencies."

Comments (47 posted)

MongoDB 1.8 released

MongoDB is a "document-oriented" database manager; the 1.8 release has been announced. New features include journaling, improvements to sharding performance, geographical searches using spherical coordinates, sparse indexes, and more.

Comments (none posted)

Pogo 0.4 released

Version 0.4 of the Pogo music player is out. "Pogo plays your music. Nothing else. It tries to be fast and easy-to-use. Pogo's elementary-inspired design uses the screen-space very efficiently. It is especially well-suited for people who organize their music by albums on the harddrive." The big changes in this release appear to be support for WAV files and the ability to export playlists.

Full Story (comments: none)

Sawfish 1.8.0 released

Version 1.8.0 of the Sawfish window manager is available. New features include "edge actions" (a way of invoking actions when the mouse pointer hits an edge of the screen), support (finally) for the --replace option, an "iconify on leave" feature, a new "style tab" functionality, and more.

Full Story (comments: none)

Development newsletters from the last week

• Caml Weekly News (March 15)
• Geany Newsletter (March 13)
• OpenOffice.org Newsletter (March)
• PostgreSQL Weekly News (March 13)
• Tcl-URL! (March 13)

Comments (none posted)

GNOME Journal bilingual edition launches

Issue 22 of the GNOME Journal is available; it includes five stories available in both English and Spanish. Topics covered include GNOME HISPANO, SyncTeX support, styling GTK+ applications with CSS, an interview with Federico Mena Quintero, and "GNOME and Andalusia."

Full Story (comments: none)

Mozilla Moves On (ComputerWorld)

Glyn Moody talks with Mozilla CEO Gary Kovacs for this ComputerWorld article. "'We've delivered a browser, 450 million users, the largest market share in Europe,' Kovacs points out before asking rhetorically: 'what else do we need to deliver that embodies the ideals of that mission to take use forward?' Well, I'm glad he asked: 'you'll see from us not only browsers, but an application framework all based on the open Web. You'll see from us services that enable your own life in a meaningful and secure and private way where you're in control, and it's not beholden to any one company or one business model, and the code truly open. And you'll see from us a push into a set of policies where the user has the transparency in their interaction that they should expect and increasingly over time will expect.'"

Comments (14 posted)

Neary: Lessons learned

Dave Neary has contributed an lengthy post to the discussion about the GNOME project and its relationship with others. "Here are the headline points: 1. FreeDesktop.org is broken as a standards body; 2. Mark Shuttleworth doesn't understand how GNOME works; 3. GNOME is not easy to understand; 4. Deep mistrust has developed between Canonical, GNOME & KDE; 5. Difficult people are prominent in each of these projects; 6. Behind closed doors conversations are poison; 7. For people to work together, they need to be in the same place."

Comments (44 posted)

Ada Initiative kicks off census

The Ada Initiative has announced that it is taking a census on the status of women in open technology and culture. "'We want to know what women are working on and how each community's culture affects women,' said Ada Initiative co-founder Valerie Aurora. 'We also want to know where to direct our work in helping women in the community.'" [Update: As pointed out in the comments, the census is meant to be filled in by everyone, regardless of gender.]

Full Story (comments: 24)

Bdale Garbee Joins FreedomBox Foundation Board

Bdale Garbee has joined the FreedomBox Foundation board of directors and will chair its technical advisory committee. "Garbee is a longtime leader and developer in the free software community. He serves as Chief Technologist for Open Source and Linux at Hewlett Packard, is chairman of the Debian Technical Committee, and is President of Software in the Public Interest, the non-profit organization that provides fiscal sponsorship for the Debian GNU/Linux distribution and other projects. In 2002, he served as Debian Project Leader."

Comments (none posted)

A note for Hotmail users

The wise folks at Hotmail have decided to start blocking email from LWN's server; they also have not really bothered to inform people of how to get themselves unblocked. As a result, anybody who has subscribed to an LWN mailing list from a Hotmail account has been unsubscribed. It must be said that we were surprised by just how many of those there were. Please accept our apologies for the inconvenience.

Comments (132 posted)

Kuhn: Thoughts On GPL Compliance of Red Hat's Linux Distribution

Bradley M. Kuhn shares some thoughts about Red Hat's GPL compliance. "Meanwhile, what people are actually complaining about is that Red Hat RHEL customers have access to better meta-information about why various patches were applied. Some have argued (quite reasonably) that this information is required under GPLv2§2(a), but usually that section has been interpreted to allow a very terse changelog. [Jon] Corbet's original article mentioned that the Red Hat distribution of the kernel named Linux contains no changelog. I see why he said that, because it took me some time to find it myself (and an earlier version of the blog post was therefore incorrect on that point), but the src.rpm file does have what appears to be a changelog embedded in the kernel.spec file. There's also a simple summary as well that in release notes found in a separate src.rpm (in the file called kernel.xml). This material seems sufficient to me to meet the letter-of-the-license compliance for GPLv2§2(a) requirements. I, too, wish the log were a bit more readable and organized, but, again, the debate isn't about whether there's optimal community cooperation going on, but rather whether this distribution complies with the GPL."

Comments (34 posted)

PLUG And The Fender Mustang (Linux Journal)

Dave Phillips gets a Fender Mustang amplifier and uses software from the PLUG project to control it. "PLUG is a simple Qt4-based application (Figure 4) that provides full control of the amplifier and effects stages of the Mustang I and II amps. It's not as flashy as Fender's software, but it weighs about 50 kilos less - no .NET or Silverlight required - and it does what it's designed to do without complications or crashes."

Comments (none posted)

Qt and Digia, facts and fiction (Qt Blog)

Sebastian Nyström separates fact from fiction regarding the future of Qt. "The Qt commercial business is important, but it is not the majority of Qt. An estimated 400,000 developers across more than 70 industries use Qt. There are 3500 commercial customers. Some have one license, some have dozens of licenses, but commercial customers do not make up the majority of the Qt user base. Nokia will continue to invest in the development of Qt, continues to hold the copyright and will continue to publish Qt under an open source license benefitting the whole community." (Thanks to Mats)

Comments (13 posted)

HP: The Linux Desktop Company (ZDNet)

Steven J. Vaughan-Nichols covers HP's decision to ship PCs with WebOS. "Regardless of the PC's maker the idea is always the same: Give Windows users a really fast-to-boot and safe operating system that they can use for Web browsing. In part, that's what HP will be doing with webOS. It's more than that though. HP really wants webOS to be more than just an also-ran in the new tablet operating system world. Indeed, with this move HP has made it clear that wants to be a desktop operating system power as well."

Comments (53 posted)

LPI Hosts Exam Labs in Poland and Greece

The Linux Professional Institute (LPI) has announced promotional exam labs for their Linux Professional Institute Certification (LPIC) in Poland and Greece. "LPI exams have long been available in these countries through online vendors Prometric and VUE, however this is the first time that local events and affiliate partners have offered public LPI testing events in Poland and Greece."

Full Story (comments: none)

Libre Graphics Meeting 2011 in Montreal

Libre Graphics Meeting will be held May 10-13, 2011, in Montreal, Canada. "Every year, Libre Graphics Meeting focuses on individual and group interaction, emphasizing free media and the software that produces it. This year's attendees can expect a similar format that will include workshops, presentations, demonstrations, and birds-of-a-feather (BOF) sessions. Leading up to the conference, special lab sessions bring developers and artists face-to-face to make improvements in creative software. Gaps can form when developers and their users don't meet face to face; LGM gives artists and educators a voice in the building of their tools. Both content creators and developers will present, bringing a diverse community together to advance the technologies and strategies of high-quality free creative software." The call for proposals is open until April 1.

Full Story (comments: none)

Events: March 24, 2011 to May 23, 2011

The following event listing is taken from the LWN.net Calendar.

Date(s)	Event	Location
March 22 March 24	OMG Workshop on Real-time, Embedded and Enterprise-Scale Time-Critical Systems	Washington, DC, USA
March 22 March 25	Frühjahrsfachgespräch	Weimar, Germany
March 22 March 24	UKUUG Spring 2011 Conference	Leeds, UK
March 22 March 25	PgEast PostgreSQL Conference	New York City, NY, USA
March 23 March 25	Palmetto Open Source Software Conference	Columbia, SC, USA
March 26	10. Augsburger Linux-Infotag 2011	Augsburg, Germany
March 28 April 1	GNOME 3.0 Bangalore Hackfest | GNOME.ASIA SUMMIT 2011	Bangalore, India
March 28	Perth Linux User Group Quiz Night	Perth, Australia
March 29 March 30	NASA Open Source Summit	Mountain View, CA, USA
April 1 April 3	Flourish Conference 2011!	Chicago, IL, USA
April 2 April 3	Workshop on GCC Research Opportunities	Chamonix, France
April 2	Texas Linux Fest 2011	Austin, Texas, USA
April 4 April 5	Camp KDE 2011	San Francisco, CA, USA
April 4 April 6	SugarCon ’11	San Francisco, CA, USA
April 4 April 6	Selenium Conference	San Francisco, CA, USA
April 6 April 8	5th Annual Linux Foundation Collaboration Summit	San Francisco, CA, USA
April 8 April 9	Hack'n Rio	Rio de Janeiro, Brazil
April 9	Linuxwochen Österreich - Graz	Graz, Austria
April 9	Festival Latinoamericano de Instalación de Software Libre
April 11 April 14	O'Reilly MySQL Conference & Expo	Santa Clara, CA, USA
April 11 April 13	2011 Embedded Linux Conference	San Francisco, CA, USA
April 13 April 14	2011 Android Builders Summit	San Francisco, CA, USA
April 16	Open Source Conference Kansai/Kobe 2011	Kobe, Japan
April 25 April 26	WebKit Contributors Meeting	Cupertino, USA
April 26 April 29	OpenStack Conference and Design Summit	Santa Clara, CA, USA
April 28 April 29	Puppet Camp EU 2011: Amsterdam	Amsterdam, Netherlands
April 29	Ottawa IPv6 Summit 2011	Ottawa, Canada
April 29 April 30	Professional IT Community Conference 2011	New Brunswick, NJ, USA
April 30 May 1	LinuxFest Northwest	Bellingham, Washington, USA
May 3 May 6	Red Hat Summit and JBoss World 2011	Boston, MA, USA
May 4 May 5	ASoC and Embedded ALSA Conference	Edinburgh, United Kingdom
May 5 May 7	Linuxwochen Österreich - Wien	Wien, Austria
May 6 May 8	Linux Audio Conference 2011	Maynooth, Ireland
May 9 May 11	SambaXP	Göttingen, Germany
May 9 May 10	OpenCms Days 2011 Conference and Expo	Cologne, Germany
May 9 May 13	Linaro Development Summit	Budapest, Hungary
May 9 May 13	Ubuntu Developer Summit	Budapest, Hungary
May 10 May 13	Libre Graphics Meeting	Montreal, Canada
May 10 May 12	Solutions Linux Open Source 2011	Paris, France
May 11 May 14	LinuxTag - International conference on Free Software and Open Source	Berlin, Germany
May 12	NLUUG Spring Conference 2011	ReeHorst, Ede, Netherlands
May 12 May 15	Pingwinaria 2011 - Polish Linux User Group Conference	Spala, Poland
May 12 May 14	Linuxwochen Österreich - Linz	Linz, Austria
May 16 May 19	PGCon - PostgreSQL Conference for Users and Developers	Ottawa, Canada
May 16 May 19	RailsConf 2011	Baltimore, MD, USA
May 20 May 21	Linuxwochen Österreich - Eisenstadt	Eisenstadt, Austria
May 21	UKUUG OpenTech 2011	London, United Kingdom

If your event does not appear here, please tell us about it.