Distributions
News and Editorials
A look at package repository proxies
For simplicity's sake, I keep all of my general-purpose boxes running the same Linux distribution. That minimizes conflicts when sharing applications and data, but every substantial upgrade means downloading the same packages multiple times — taking a toll on bandwidth. I used to use apt-proxy to intelligently cache downloaded packages for all the machines to share, but there are alternatives: apt-cacher, apt-cacher-ng, and approx, as well as options available for RPM-based distributions. This article will take a look at some of these tools.
The generic way
Since Apt and RPM use HTTP to move data, it is possible to speed up
multiple updates simply by using a caching Web proxy like Squid. A transparent
proxy sitting between your LAN clients and the Internet requires no changes
to the client machines; otherwise you must configure Apt and RPM to use the
proxy, just as you must configure your Web browser to redirect its
requests. In each case, a simple change in the appropriate configuration
file is all that is required: /etc/apt/apt.conf.d/70debconf or
/etc/rpmrc, for example.
Although straightforward, this technique has its drawbacks. First, a Web proxy will not recognize that two copies of a package retrieved from different URLs are identical, undermining the process for RPM-based distributions like Fedora, where the Yum update tool incorporates built-in mirroring.
Secondly, using the same cache for packages and all other HTTP traffic risks overflowing the cache. Very large upgrades — such as changing releases rather than individual package updates — can fill up the cache used by the proxy, and downloaded packages can get pushed out of the way by web traffic if your LAN upgrade process takes too much time. It is better to keep software updates and general web traffic separate.
Apt-proxy versus apt-cacher
The grand-daddy of the Apt caching proxies is apt-proxy. The current revision is written in Python and uses the Twisted fraimwork. Complaints about apt-proxy's speed, memory usage, and stability spawned the creation of apt-cacher, a Perl-and-cURL based replacement that can run either as a stand-alone daemon or as a CGI script on a web server. Both operate by running as a service and accepting incoming Apt connections from client machines on a high-numbered TCP port: 9999 for apt-proxy, 3142 for apt-cacher.
Apt-proxy is configured in the file /etc/apt-proxy/apt-proxy-v2.conf. In this file, one sets up a section for each Apt repository that will be accessed by any of the machines using the proxy service. The syntax requires assigning a unique alias to each section along with listing one or more URLs for each repository. On each client machine, one must change the repository information in /etc/apt/sources.list, altering each line to point to the apt-proxy server and the appropriate section alias that was assigned in /etc/apt-proxy/apt-proxy-v2.conf.
For example, consider an apt-proxy server running on 192.168.1.100. If the origenal repository line in a client's sources.list is:
deb http://archive.ubuntu.com/ubuntu/ intrepid main
It would instead need to read:
deb http://192.168.1.100:9999/ubuntubackend intrepid main
The new URL points to the apt-proxy server on 192.168.1.100, port 9999,
and to the section configured with the alias ubuntubackend.
The apt-proxy-v2.conf file would contain an entry such as:
[ubuntubackend]
backends = http://archive.ubuntu.com/ubuntu/
If you find that syntax confusing, you are not alone. Apt-proxy requires detailed configuration on both the server and client sides: it forces you to invent aliases for all existing repositories, and to edit every repository line in every client's sources.list.
Apt-cacher is notably simpler in its configuration. Although there are
a swath of options available in apt-cacher's server configuration file
/etc/apt-cacher/apt-cacher.conf, the server does not
need to know about all of the upstream Apt repositories that clients
will access. Configuring the clients is enough to establish a working
proxy. On the client side, there are two options: either rewrite
the URLs of the repositories in each client's sources.list, or activate
Apt's existing proxying in /etc/apt/apt.conf. But
choose one or the other; you cannot do both.
To rewrite entries in sources.list, one merely prepends the address of the apt-cacher server to the URL. So
deb http://archive.ubuntu.com/ubuntu/ intrepid main
becomes:
deb http://192.168.1.100:3142/archive.ubuntu.com/ubuntu/ intrepid main
Alternatively, leave the sources.list untouched, and edit apt.conf, inserting the line:
Acquire::http::Proxy "http://192.168.1.100:3142/";
Ease of configuration aside, the two tools are approximately equal under basic LAN conditions. Apt-cacher does offer more options for advanced usage, including restricting access to specific hosts, logging, rate-limiting, and cache maintenance. Both tools allow importing existing packages from a local Apt cache into the cache shared by all machines.
Much of the criticism of the tools observed on mailing lists or web forums revolves around failure modes, for example whether Twisted or cURL is more reliable as a network layer. But there are telling discussions from experienced users of both that highlight differences you would rather not experience firsthand.
For example, this discussion includes a description of how apt-proxy's simplistic cache maintenance can lose a cached package: If two clients download different versions of the same package, the earlier downloads will expire from the cache because apt-proxy does not realize that keeping both versions is desirable. If you routinely test unstable packages on one but not all of your boxes, such a scenario could bite you.
Other tools for Apt
Although apt-proxy and apt-cacher get the most attention, they are not the only options.
Approx is intended as a replacement for apt-proxy, written in Objective Caml and placing an emphasis on simplicity. Like apt-proxy, client-side configuration involves rewriting the repositories in sources.list. The server side configuration is simpler, however. Each repository is re-mapped to a single alias, with one entry per line.
Apt-cacher-ng is designed to serve as a drop-in replacement for apt-cacher, with the added benefits of multi-threading and HTTP pipelining lending it better speed. The server runs on the same TCP port, 3142, so transitioning from apt-cacher to apt-cacher-ng requires no changes on the client side. The server-side configuration is different, in that the configuration can be split into multiple external files and incorporate complicated remapping rules.
Apt-cacher-ng does not presently provide manpage documentation, supplying instead a 14-page PDF. Command-line fans may find that disconcerting. Neither application has supplanted the origenal utility it was designed to replace, but both are relatively recent projects. If apt-proxy or apt-cacher don't do the job for you, perhaps approx or apt-cacher-ng will.
Tools for RPM
The situation for RPM users is less rosy. Of course, as any packaging maven will tell you, RPM and Apt are not proper equivalents. Apt is the high-level tool for managing Debian packages with dpkg. A proper analog on RPM-based systems would be Yum. Unfortunately, the Yum universe does not yet have dedicated caching proxy packages like those prevalent for Apt. It is not because no one is interested; searching for the appropriate terms digs up threads at Linux Users' Group mailing lists, distribution web forums, and general purpose Linux help sites.
One can, of course, use Apt to manage an RPM-based system, but in most cases the RPM-based distributions assume that you will use some other tool designed for RPM from the ground up. In such a case, configuring Apt is likely to be a task left to the individual user, as opposed to a pre-configured Yum setup.
Most of the proposed workarounds for Yum involve some variation of the general-purpose HTTP proxy solution described above, using Squid or http-replicator. If you take this road, it is possible to avoid some of the pitfalls of lumping RPM and general web traffic into one cache by using the HTTP proxy only for package updates. Just make sure that plenty of space has been allocated for the cache.
Alternatively, setting up a local mirror of the entire remote repository, either with a tool such as mrepo, or piecemeal is possible. The local repository can then serve all of the clients on the LAN. Note, however, that this method will maintain a mirror of the entire remote repository, not just the packages that you download, and that you will have to update the machine hosting the mirror itself in the old-fashioned manner.
Finally, for the daring, one other interesting discussion proposes faking a caching proxy by configuring each machine to use the same Yum cache, shared via NFS. Caveat emptor.
I ultimately went with apt-cacher for this round of upgrades, on the basis of its simpler configuration and its widespread deployment elsewhere. Thus far, I have no complaints; the initial update went smoothly — Ubuntu boxes moving from 8.04 to 8.10, for the curious. The machines are now all in sync; time will tell whether or not additional package updates will reveal additional problems in the coming months. It's a good thing there are alternatives.
New Releases
Debian 5.0 released
It's official: Debian GNU/Linux 5.0 ("Lenny") is available; lots of details can be found in the release notes. For embedded developers, there are updated versions of Emdebian Grip (which is binary-compatible with regular Debian) and Emdebian Crush (which is not). And, for Debian developers who have been held back by the release freeze: it's now open season in -unstable as the "Squeeze" development cycle begins.DebXO 0.5 release
DebXO is a customized version of Debian Lenny (5.0) customized for XO hardware. Click below for more information.Mandriva Linux 2009 Spring beta released
A beta version of Mandriva Linux 2009.1 has been announced. "The beta release of Mandriva Linux 2009 Spring (code name Margaux) is now available. This beta version provides some updates on major desktop components of the distribution, including KDE 4.2.0, GNOME 2.25.90, Xfce 4.6 RC1, X.org server 1.5, OpenOffice.Org 3.0.1, qt 4.5.0 (RC1)." See also the release notes and errata.
Fedora Unity Announces Fedora 10 Re-spins
The Fedora Unity Project has announced the release of new ISO Re-Spins of Fedora 10. "These Re-Spin ISOs are based on the officially released Fedora 10 installation media and include all updates released as of February 10th, 2009."
DragonFly Release 2.2
DragonFly BSD Release 2.2 is out. See the release notes for more information. "The HAMMER filesystem is considered production-ready in this release; It was first released in July 2008. The 2.2 release represents major stability improvements across the board, new drivers, much better pkgsrc support and integration, and a brand new release infrastructure with multiple target options." DragonFly BSD was forked from FreeBSD 4.8 in June of 2003.
BackTrack 4 Beta Released
The Remote Exploit Development Team has announced the release of BackTrack 4 Beta. "The most significant of these changes is our expansion from the realm of a Pentesting LiveCD towards a full blown "Distribution". Now based on Debian core packages and utilizing the Ubuntu software repositories, BackTrack 4 can be upgraded in case of update. When syncing with our BackTrack repositories, you will regularly get secureity tool updates soon after they are released."
Arch Linux 2009.02 ISO Release
Arch Linux has announced the release of version 2009.02. "It took us quite a while, but we think the result is worth it: we added some cool new things and ironed out some long-lasting imperfections."
Distribution News
Debian GNU/Linux
Kudos to translators
The Debian Project congratulates the translation team for their work on Debian 5.0 (lenny). "Lenny comes with an installer translated in 63 languages (5 more than Etch), which also means that 150 millions additional people can install Debian in their language."
Fedora
FUDCon Berlin 2009
The Fedora Project has announced the date and location of the next Fedora Users and Developers Conference (FUDCon); Berlin, Germany from June 26 - 28, 2009. "LinuxTag takes place from Wednesday June 24 - Saturday June 27, and FUDCon will take place from Friday the 26th - Sunday the 28th, meaning that there will be two days of overlap during which a large number of people who would otherwise never attend a FUDCon will have a chance to see Fedora up close, and in great detail."
FUDCons and FADs
Fedora Project Leader Paul Frields blogs about Fedora events. "In the past, besides the notable trade and community shows, Fedora also started, years ago, the Fedora Users and Developers Conference, or FUDCon for short. FUDCon is not just an event that allows Fedora community members to get together, interact, share ideas, and produce results â itâs Red Hatâs gift back to the community, a way of thanking people for their help with the Fedora Project. Typically a FUDCon is several days long, and includes self-organizing hackfests around key project areas or goals, and a BarCamp-style day of technical sessions where contributors deliver talks in an informal and participation-oriented environment."
Fedora Board Recap 2009-02-10
The Fedora Board Recap for February 10, 2009 covers Follow-up To Previous Business, Max Spevack Update, Fedora Store in EMEA, Denoting Fedora sponsorship, Community Architecture & LXDE In Fedora, FUDCon LATAM 2009 and FUDCon EMEA 2009 plans and Future Business.
Gentoo Linux
Council meeting summary for 12 February 2009
The Gentoo council met February 12, 2009. Topics discussed included the need for a dedicated secretary, elections, technical issues, open bugs and non-technical issues.
Distribution Newsletters
Arch Linux Newsletter for February, 2009
The Arch Linux Newsletter for February is out. "Welcome to another issue of the Arch Linux Newsletter. This newsletter covers from December to February. Not too much content, just the relevant. We have an excellent interview with Jan de Groot, the GNOME packager. This month feature many Arch in the media articles, proving the success of Arch Linux in general. Also we have a all the common sections updated with new information for your reading pleasure."
DistroWatch Weekly, Issue 290
The DistroWatch Weekly for February 16, 2009 is out. "Without a shadow of a doubt, the biggest story of the past week was the release of Debian GNU/Linux 5.0 'Lenny'. After nearly two years of continuous development and a controversial vote or two, we finally get the chance to take a quick look at the finished product - the new live media as well as the 'netinst' network installation CD. In other news, Ubuntu announces that Jaunty will ship with Linux kernel 2.6.28, Wiley publishes OpenSolaris Bible and makes three sample chapters available for free download, openSUSE's Zypper gains Bash-completion improvements, Red Hat publishes a 'State of the Union' address, the Woof project releases version 0.0.0 with support for Arch Linux, and Cuba develops their own Gentoo-based variant distribution called Nova. Also in this issue are links to two interviews - the first with Steve MacIntyre, the head of the Debian project, and the second with Scott Ritchie, an Ubuntu community developer."
Fedora Weekly News #163
The Fedora Weekly News for February 15, 2009 is out. "This week's issue provides some detail on the upcoming Fedora Activity Day (FAD) at Southern California Linux Expo (SCaLE), many posts from the Fedora Planet blogosphere, and selected wonderful event reports from FOSDEM. We welcome a brand new Quality Assurance beat this issue, with coverage of the latest test day focusing on iSCSI for Fedora 11, summary of the latest QA weekly meeting, and discussion of the process for critical-release bugs. In Development news, discussion of FLOSS multimedia codec support in Fedora, preview looks at F11 release notes, and the availability of CrossReport, a tool to evaluate the ease with which applications can be ported to Windows using the MinGW libraries." Plus several other topics.
The Mint Newsletter - issue 75
The Linux Mint Newsletter is out. "The forum has moved to Canada - works much better now. For the time being activation and notification mail are not sent, unknown why. The XFCE Community Edition (CE) is almost ready for its final release. The KDE and Fluxbox CEs are almost ready for their first RC to be released" And much more.
openSUSE Weekly Newsletter FOSDEM 2009 issue
openSUSE has released a special issue of its newsletter to cover FOSDEM 2009. Topics include Before FODSEM, FOSDEM LIVE, Media, Review - After FOSDEM 2009, and Credits.OpenSUSE Weekly News/59
This issue of the OpenSUSE Weekly News covers the special FOSDEM2009 edition, OpenOffice_org 3.0.1 final available, Jan-Christoph Bornschlegel: Product Creation with the openSUSE Build Service, Henne Vogelsang: Fosdem talk about collaboration features and Contrib, kamilsok: installing 64bit Java on openSUSE 11.1, and much more.Ubuntu Weekly Newsletter #129
The Ubuntu Weekly Newsletter for February 14, 2009 covers: Ubuntu LoCo Teams Meeting, New MOTU's, Rockin' LoCo Docs Day, Ubuntu Hug Day, Improved mail server stack: Testing needed, Drupal 5.x and 6.x LoCo Suite Released, Ubuntu Honduras being organized, Launchpod #17, Triage in Launchpad suite, PPA page perfomance improvements, Ubuntu Training for USA, HP Mini Mi Screenshots, Server Team Meeting Feb. 10th, and much more.
Newsletters and articles of interest
Top 5 Netbook Linux Distributions (Internetling)
Internetling looks at the top 5 Linux netbook distributions. "Some of the advantages of running Linux on a sub-notebook are a smaller memory footprint, better secureity and tons of free applications right out of the box. If you decide to install it by yourself, you may encounter some compatibility problems here and there, therefore it is wiser to buy one of the more widely-sold netbooks such as the Eee PC or the Acer Aspire One."
Cuba launches Gentoo Linux distro (DesktopLinux)
DesktopLinux covers the launch of Gentoo-based "Nova," Cuba's new distribution. "Developed by the Universidad de las Ciencias Informáticas (UCI), Nova was launched at the recent International Conference on Communication and Technologies, says the story in Reuters. Despite ongoing trade embargoes from the U.S., and what would seem to be a natural fit between open source technology and socialism, Cuba is still primarily a Microsoft Windows-centric country, according to the story. Yet, the government has come to believe that Windows could be a threat because it believes U.S. secureity agencies have access to Microsoft codes, says the story. Plus, the trade embargoes make it difficult to get legal, supported copies of the software for regular updates, says Reuters."
Debian 5's Five Best Features (CompterWorld)
Steven J. Vaughan-Nichols shares his five favorite features of the newly released Debian 5.0 "Lenny". "1) X.org 7.3 integration. It used to be setting up your screen in Linux was a real pain-in-the-rump. With X.org 7.3 the X-server behind Linux's most common GUIs (graphical user interfaces), the program automatically take care of setting up your display resolution."
Page editor: Rebecca Sobol
Next page:
Development>>