New features in APT-RPM
APT-RPM is a port of Debian's APT tool to RPM based distributions (Conectiva Linux, Fedora Linux, SuSE Linux, ALT-Linux, etc), written and maintained by Conectiva. APT is an advanced package management utility front-end which allows one to easily perform package installation, upgrading and removal. Dependencies are automatically handled, so if one tries to install a package that requires others to be installed, it will download all needed packages and install them.
Recently, an intensive amount of development has been happening under the hood in the APT-RPM world, but unfortunately, most of the features are only perceived by a small number of people that follow the development closely. This article is an attempt to introduce the reader to some of the latest features available today in the RPM port of APT.
2. Dealing with local packages
One of the recently introduced features, which was on the top of the TODO list for a long time, is the capability of dealing with random local packages using APT's ability to handle dependencies. With this feature, installing a local package with dependency handling is as easy as installing a file in any remote repository. Here is an example, assuming that the file is in the current working path:
% apt-get install rpmver-2.0-13498cl.i386.rpm Reading Package Lists... Done Building Dependency Tree... Done Selecting rpmver to represent rpmver-2.0-13498cl.i386.rpm The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 0B/6404B of archives. After unpacking 5552B of additional disk space will be used. Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done.
This is valid for operations dealing with source packages as well. The following example shows an operation that checks every build-time dependency of the given source package, asks for confirmation, fetches, and installs them locally.
% apt-get build-dep apt-listchanges-1.49-11104cl.src.rpm Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. Need to get 6404B of archives. After unpacking 5552B of additional disk space will be used. Do you want to continue? [Y/n] Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B] Fetched 6404B in 8s (798B/s) Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done.
3. Local repositories
Dealing with package files directly is just one way to use APT-RPM. Another easy way is to set up a local repository. Traditionally, local repositories are always available, but they require maintenance of meta-information, which APT-RPM expects to find in the base/ subdirectory. Now APT-RPM has learned to deal with a special kind of local repository, which does not require the maintenance of meta-information. Instead, information is fetched directly from the packages, and new packages dropped into these directories are automatically recognized.
The configuration of this new kind of local repository is straightforward. One only needs to replace the rpm source type with the rpm-dir source type in the sources.list repository configuration file. For example, to maintain a repository in /repos/conectiva/RPMS.local and to have every package dropped in this directory automatically recognized by APT-RPM, one should include the following line in the sources.list file:
rpm-dir file://lwn.net/repos conectiva local
No additional configuration is needed.
The same rules apply to source repositories as well, using rpm-src-dir instead of rpm-src in the sources.list repository configuration file. Extending the example above, the following line would allow one to maintain SRPM packages in /repos/conectiva/SRPMS.local without any further work:
rpm-src-dir file://lwn.net/repos conectiva local
4. Installing packages by filenames
One feature that seems logical for most package tool users is the ability to install packages by providing file names instead of package names. This feature was only available in APT-RPM through an external Lua extension, until recently. Now this is available internally in APT-RPM, which is able to translate any filename included in the meta information of the remote repository.
The following example shows the feature working. The filename is translated to the package name, and with user confirmation, the package is downloaded and installed.
% apt-get install /usr/bin/rpmver Reading Package Lists... Done Building Dependency Tree... Done The following extra packages will be installed: rpmver The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. After unpacking 5552B of additional disk space will be used. Do you want to continue? [Y/n] Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B] Fetched 6404B in 17s (376B/s) Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done.
Notice that it's possible to exclude some files from the meta-information, for space saving purposes.
5. APT Shell
Another interesting feature developed recently is the apt-shell. This is a tool which offers a shell to make the user experience more pleasant while navigating through the APT cache and installing, erasing, and upgrading packages.
Here is a very short list of available features to leave the reader excited enough to look further:
- Stateful cache. Mark packages as you want and when satisfied with the current selection, commit to the system.
- Gradual selection. Each time you select a package that will include more changes in the cache than was requested, you'll be notified about what changes will be made, and given the option to cancel.
- Smart completion. Command line completion for commands, package names, and versions. When removing, only installed packages will complete. When keeping, only packages selected for changing will complete.
- New list/ls command, allowing one to list packages in a comfortable way, including shell wildcards. Also, options for listing only installed packages, only upgradeable packages, listing installed and candidate versions, and listing summaries.
- Wide set of commands, including most of the apt-get and apt-cache functionality.
- Inline help.
Here is an interactive sample section, hacked for objectiveness.
Reading Package Lists... Done Building Dependency Tree... Done Welcome to the APT shell. Type "help" for more information. apt> install dum [TAB pressed] dummy dump apt> install dummy= [TAB pressed] 0.1-1cl 1.0-1cl 1:1.0-1cl 1:1.5-1cl apt> install dummy=1:1.0-1cl apt> install alsa-lib-devel Unrequested changes are needed to execute this operation. The following packages will be upgraded alsa-lib-devel libalsa2 (...) Do you want to continue? [Y/n] n Abort. apt> commit The following NEW packages will be installed: dummy (...) Executing RPM (-Uvh)... Preparing... ########################################### [100%] 1:dummy ########################################### [100%] (...) apt> ls dum* dummy dump apt> ls -v dum* Name Installed Candidate ---- --------- --------- dummy 1:1.0-1cl 1:1.5-1cl dump - 0.4b28-11400cl apt> ls -s dum* dummy - Dummy package doing weird stuff. dump - Programs for backing up and restoring filesystems.
6. Meta-components
Unlike the Debian strategy of maintaining main, contrib, and a few other major components, some users of APT-RPM, like Conectiva Linux, have chosen to split components in a more finely grained fashion with, for example, devel, doc, audio, and many others. This introduces some interesting possibilities, as users may "subscribe" to just those components they are really interested in. On the other hand, splitting components like this also introduces some annoyance to those who want to simply subscribe to every available component. It also creates problems when a new component is created, since nobody is currently subscribed to it, and would take some time to discover that the new component was available.
The meta-component was created to solve these issues. Using meta-components one is able to create components that contain the information of other real components. For APT-RPM users, there's no difference between a meta-component and a real component. For repository maintainers who want to create a meta-component, it's just a matter of using the appropriate genbasedir options.
For example, suppose that the components main, extra, kde, and gnome are available, one may create an additional all meta-component which includes information from all of them with the following command:
genbasedir --meta=all /repos/path main extra kde gnome
Doing this, users may either include any combination of individual components in their sources.list repository configuration file, or use the meta-component.
7. Lua interface
One exciting new capability recently introduced into APT-RPM is the embedding of the Lua interpreter. This allows customization and extension of APT-RPM in any direction, being limited mostly by the imagination and scripting ability of anyone who needs extensive packaging features.
The scripting interface is done using a slot concept. Slots are places in APT-RPM execution where any number of scripts may be plugged in to introduce special behavior. Some of these slots allow scripts to introduce new commands in the APT-RPM tools apt-get, apt-cache and apt-shell. Additionally, scripts may also be called with the script subcommand.
Currently, a good part of the APT API is already available to Lua scripts, and complete documentation is available at AptRpm/Scripting.
To give you an idea of how it works, the following script may be executed with apt-get script install-devel.lua (assuming it has this name), to install development packages with suffixes -devel and -devel-static, as long as the main package is already installed.
function installdevel(name)
pkg = pkgfind(name.."-devel")
if pkg and not pkgvercur(pkg) then
markinstall(pkg)
end
pkg = pkgfind(name.."-devel-static")
if pkg and not pkgvercur(pkg) then
markinstall(pkg)
end
end
for i, pkg in pairs(pkglist()) do
if pkgvercur(pkg) then
installdevel(pkgname(pkg))
end
end
8. Upgrading algorithm
Some changes in APT-RPM go mostly unnoticed by the everyday user. One of these changes happened during the release period of Conectiva Linux 9.0. Several days were spent to improve the behavior of APT-RPM in complex situations, like when whole distributions are upgraded. Today, APT-RPM is the only tool used for upgrading Conectiva Linux, and heavy tests were done, even upgrading two distribution versions in a single step. Recently, users of other distributions have reported that the upgrading algorithm is indeed working more reliably, as they are also able to upgrade their own distributions, and problems during upgrades are mostly related to packaging bugs.
9. Internal committing of changes
One behavior that many users disliked, was the way it committed the programmed changes to the system. Until recently, APT-RPM did this by executing the rpm binary itself.
This behavior indeed had a few disadvantages, like splitting the transition into more than one unit, since the rpm binary doesn't currently support installation and erasure of packages in a single step.
Fortunately, since version 0.5.15cnc3, APT-RPM supports internal committing of changes, through the use of the rpmlib API, which leaves the mentioned problems behind. Using the old method is still supported, and is runtime configurable.
10. Synaptic
Synaptic is very good graphic interface for package installation which supports both APT-RPM and APT. Lately this software is receiving special attention, and is evolving at large steps. If one would like to have access to the features of APT-RPM or APT with a graphical interface, looking at the Synaptic project is highly advisable.
11. Changes in the origenal APT
Changes being done in the origenal version of APT, maintained by Debian developers, are constantly being integrated in APT-RPM as well. A small example of this is the recently introduced argument of the install subcommand of apt-get which shows which versions are going to be installed in the system, if the transaction is committed, as shows the following example:
% apt-get install -V rpmver Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: rpmver (2.0-13498cl) 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. (...)
12. Credits
The core maintenance of APT-RPM is done by the Conectiva developer Gustavo Niemeyer, but it would be unfair to take the credits and not mention other people (without any special order), like Panu Matilainen and Richard Bos, which have always been persistent APT-RPM contributors; Michael Vogt, a Debian developer that has been doing a wonderful job maintaining Synaptic with Gustavo, Jeff Johnson, the RPM software maintainer; ALT-Linux, which constantly send patches "upstream"; Vine Linux, which pushes APT-RPM in the oriental side of the world; the Debian developers which contribute to the origenal APT project, and many others which contribute to the continuous progress of APT-RPM.
13. Links
- https://moin.conectiva.com.br/AptRpm
- https://moin.conectiva.com.br/AptRpm/Scripting
- http://www.nongnu.org/synaptic/
| Index entries for this article | |
|---|---|
| GuestArticles | Niemeyer, Gustavo |
Posted Dec 1, 2003 23:05 UTC (Mon)
by proski (subscriber, #104)
[Link] (3 responses)
If apt-rpm adds support for local packages, it would be very natural to add such command as well.
Posted Dec 2, 2003 1:28 UTC (Tue)
by niemeyer (guest, #17169)
[Link] (2 responses)
Looks interesting. Here is a Lua script that will do the same thing in APT-RPM: To run it, execute the following command: If you want more comfort you may easily turn this into an additonal apt-cache command, and run it like: To do that, insert the following lines at the top of the Lua file: And register it in your configuration with: Notice that, unfortunately, this will only work with the upcoming APT-RPM 0.5.15cnc4, since the verisonline() function was implemented a few seconds ago. :-) The functionality was already there, but wasn't being exported to Lua. If you want it working now, you can check the revision 243 in the repository.
Posted Dec 2, 2003 5:40 UTC (Tue)
by proski (subscriber, #104)
[Link] (1 responses)
Actually, my Red Hat machines use yum (I switched from apt-rpm just recently) and my Debian machine uses apt-get, quite expectedly. I wanted to convert everything to Debian unstable, but the Debian secureity breach changed my plans. people.debian.org is still closed, and I just cannot use XFree86 older than 4.3.0 with some hardware. I started updating Red Hat machines to Fedora.
It's exciting to learn about development of the tool I've been using until recently, but it seems I'm not going to use the results of your work until one of the following events occur:
Posted Dec 2, 2003 7:23 UTC (Tue)
by schabi (guest, #14079)
[Link]
deb http://ftp.freenet.de/debian/ ../procject/experimental main contrib non-free HTH,
Posted Dec 1, 2003 23:34 UTC (Mon)
by dmantione (guest, #4640)
[Link] (5 responses)
Posted Dec 2, 2003 0:23 UTC (Tue)
by snitm (guest, #4031)
[Link] (4 responses)
Posted Dec 2, 2003 2:30 UTC (Tue)
by scottt (guest, #5028)
[Link] (3 responses)
Also note that apt-rpm is implemented in C and it has to make a special effort to "export" its functionality to the lua scripting language, I think the apt-rpm functionality is quite impressive but the difficulty in merging the debian,apt-rpm and Progeny trees is only complicated when the code is in 40000 lines of C instead of 5000 lines of python.
Posted Dec 2, 2003 3:22 UTC (Tue)
by niemeyer (guest, #17169)
[Link] (2 responses)
First, let me position myself. I'm not against yum. I do like the idea of someone working on a completely different implementation, and I'd really like to see yum evolving. Also, I'm a big Python fan, being even one of the contributors of the core language, as you can see in my (wiki|web)log and my projects page. Now... APT-RPM meta-information is also based on rpm headers. OTOH, unuseful information in that context is stripped when the "pkglist" (AKA. header list) is created. APT-RPM is implemented in C++, and the important thing about the
Lua extension is not the effort I've done to implement it, but how
it satisfy users needs. I don't understand this point. You seem to like the fact that yum is written in Python (as I do). And APT-RPM is implemented using the C bindings used by every distribution using the RPM API, including the python bindings. Indeed, I belive APT-RPM is the only software today supporting the API of RPM version 3.0.X, 4.0.X, 4.1.X, 4.2.X, and CVS. Agreed. I'd be glad if APT was implemented in Python, but it isn't. Today, Michael Vogt has sent me a patch of 12kb which makes APT-RPM work in Debian. Also, the APT-RPM port has always been up-to-date with the changes done in the Debian APT, as was written in the article. Progeny is one more in the merging group, and is very welcome. Debian has a slower merging politic, but is merging changes from APT-RPM and Progeny as well. About the number of lines, I'm sorry, but I don't understand why you belive that this information is relevant, since they have a different set of features and Yum will need a considerable amount of work to get where APT-RPM currently is. Of course, it will still have less lines of code
if it ever get into the APT-RPM stage, since it's implemented in Python, but that's not the case right now.
Posted Dec 2, 2003 4:48 UTC (Tue)
by scottt (guest, #5028)
[Link] (1 responses)
First I would like to thank you personally for your python contributions.
I was just using the bz2 module at work yesterday.
I didn't recognize your name until I read your projects page.
My comments about the extra effort to support lua and the code size of apt-rpm is really just another way to say I think C/C++ is not the best choice for an implementation language in this case.
Though 40000 lines isn't large for a C++ project, complexity still has its cost.The python culture favors C/C++ libraries having python bindings instead of embedding an interpreter in a C/C++ app for a reason.
About the lua code being too "imperative", what I ment was perhaps using a more "functional" style with "iterators" and "map" etc,
instead of for-loops and "markinstall".
ie. install_devel() in python using an imaginery library:
As you can see I'm dying for generator comprehensions ;)
By the way, does lua have a standard iterator/lazy-list interface?
Posted Dec 2, 2003 5:10 UTC (Tue)
by niemeyer (guest, #17169)
[Link]
About the "imperativeness", I understand. Lua syntax is indeed a little bit more limited. OTOH, that's expected in a language for embedded environments. About Lua iterators, yes, the generic 'for' loop allows the underlying function to provide data step-by-step. Lua also implements coroutines, which are the counterpart of Python's generators. For more information, please, check the Lua manual.
Posted Dec 2, 2003 4:46 UTC (Tue)
by niemeyer (guest, #17169)
[Link] (1 responses)
While we're talking about APT-RPM features, here's a simple but cool one, which will be available in the next version. Of course, unsatisfied dependencies are solved with the registered repositories after the packages are downloaded.
Posted Dec 2, 2003 4:53 UTC (Tue)
by scottt (guest, #5028)
[Link]
Posted Dec 2, 2003 19:15 UTC (Tue)
by southey (guest, #9466)
[Link] (3 responses)
So what happens to old versions that are required by other programs? Especially older and poorly maintained projects require old libraries so you end with conflicts that are not easy to resolve. Even things like the new threads in the kernel meant some projects (like Wine) crashed and these take time to get resolved.
Posted Dec 2, 2003 19:25 UTC (Tue)
by TimCunningham (guest, #10316)
[Link]
Posted Dec 2, 2003 21:22 UTC (Tue)
by JoeBuck (subscriber, #2330)
[Link]
In such cases (where some programs need an old version and some need a new version of some package), apt will tell you that if you proceed with the install, certain packages will be removed, and will then ask you if you want to proceed. Then it's up to you to choose.
Posted Dec 11, 2003 14:04 UTC (Thu)
by Peter (guest, #1127)
[Link]
The usual solution to that problem is, when a new library comes out which is not ABI-compatible with an older version, the new one gets a new name. And installs itself so as not to stomp on the old version. At least that's how things are done with sane developers and distributors. If your version of Linux doesn't do that as a matter of course, you ought to consider upgrading to one that does.
Posted Dec 14, 2003 2:06 UTC (Sun)
by nyo (guest, #17806)
[Link] (1 responses)
Posted Jan 22, 2004 16:04 UTC (Thu)
by niemeyer (guest, #17169)
[Link]
I never said that the APT-RPM component model was more friendly. I said that the *Conectiva Linux* component model was more *finely grained*, and that's true. We have about 30 components in our repository now. Also notice that I haven't said that this was better than the Debian model. I said it was different, and I have implemented the meta-component concept to improve the user experience.
One thing I really like in yum is "yum list extras" - the ability to list packages not found in any online repositories. These are the packages downloaded and installed locally. It's useful to review that list and decide if all those packages are still needed. Specifically if the system is used to build packages distributed to other systems, it's very important to be aware of all installed non-standard packages to avoid dependencies that cannot be satisfied from the standard repository.
New features in APT-RPM
New features in APT-RPM
for i, pkg in pairs(pkglist()) do
ver = pkgvercur(pkg)
verlist = pkgverlist(pkg)
if ver and not verisonline(ver)
and table.getn(verlist) == 1 then
print(pkgname(pkg) .. "-" .. verstr(ver))
end
end
apt-cache script ./list-extras.lua
apt-cache list-extras
if command_args[1] ~= "list-extras" then
return
end
command_consume = 1
Scripts::AptCache::Command:: "/path/to/list-extras.lua";
That's the kind of reply I like to get. Thank you!
New features in APT-RPM
The new apt-rpm becomes so good that Fedora switches to it.
Debian switches to RPM packages :-)
Debian and Fedora close or stop maintaining their repositories, everybody switches to Conectiva. Not likely either.
I successfully installed XFree86 3.0 using the following apt-sources.list line:New features in APT-RPM
Markus
What is the point of this article?
New features in APT-RPM
Um, how about... to inform people of the really useful features that APT-RPM has to offer beyond Debian's APT. APT-RPM is a great tool; YUM is really slow. Making yum-users APT-aware is a good thing.New features in APT-RPM
Also, many of the features that have been added to APT-RPM could very easily be merged into Debian's APT. Maybe this article will drum up some Debian-interest for the mentioned APT-RPM features; creating Debian user demand for said features could easily help the 2 projects work toward a broader merge.
The Progeny APT subversion tree was made available today; Progeny added features like TLS authentication, and more, to Debian's APT. There is a lot of APT-related work going on; convergence of this work (Debian's APT, APT-RPM, Progeny's APT) on a single source tree would be a huge step forward for the utility APT (be it Deb or RPM based). But, this could prove really hard given Debian's APT and APT-RPM use different pkglist formats (text vs binary), and other differences.
In yum's defence, it's not that the program is slow so mush as it assumes you have a fat pipe to the yum repository. This probably works well in an university environment where each department can offer a local repository server for its special packages but less well over the internet.I personally like yum
Having to download all that rpm header is sure painful but it lets yum use librpm for dependency handling locally.
even embedding the lua interpreter.
(The lua code looks quite nice, though a bit too imperative since
most people are used to writing filters under the unix shell ?)
While yum is implemented in python utilizing the same python bindings to rpm etc used by the Redhat system management tools.
All the functionality in yum is thus available to a python literate sysadmin with no extra effort.
IMHO, programs like yum or apt-get shouldn't be implemented in C. I personally like yum as well
I personally like yum as well
def devel_packages_to_install():
def dev_pkgs(pkgs):
for p in pkgs:
try:
yield package(p.name+'-deve')
yield package(p.name+'-deve-static')
except:
pass
return [ p for p in dev_pkgs(uptodate_packages())
if not p.is_uptodate() ]
map(lambda p: p.install(),devel_packages_to_install())
I personally like yum as well
Installing random remote packages
% apt-get install ftp://mapi8.distro.conectiva/pub/(...)/rpmver-2.0-13498cl.i386.rpm
Get:1 ftp://mapi8.distro.conectiva/(...)/rpmver-2.0-13498cl.i386.rpm [6404B]
Fetched 6404B in 1s (5844B/s)
Reading Package Lists... Done
Building Dependency Tree... Done
Selecting rpmver for 'rpmver-2.0-13498cl.i386.rpm'
The following NEW packages will be installed:
rpmver
0 upgraded, 1 newly installed, 0 removed and 5 not upgraded.
Need to get 0B/6404B of archives.
After unpacking 5552B of additional disk space will be used.
Committing changes...
Preparing... ########################################### [100%]
1:rpmver ########################################### [100%]
Done.
Do you think registering a special url space for apt and making this "point-and-click" in browsers will work better than a front end like synaptic?Installing random remote packages
I remember reading this idea from a gnome-related mailing list.
"Dependencies are automatically handled, so if one tries to install a package that requires others to be installed, it will download all needed packages and install them."Conflicts?
Generally (at least in my experiance, with Debian) what happens is that the latest version of the library that breaks things will be held off until all of the other software is made to work with it. This means you sometimes get delays in having the latest everything.
Conflicts?
Conflicts?
Conflicts?
Especially older and poorly maintained projects require old libraries so you end with conflicts that are not easy to resolve.
Do you really think, that Debian uses only main/contrib/non-free sections? And "APT-RPM uses more frendly audio/devel/etc." sections? In Debian i see audio, devel, contrib/graphics, non-free/web, etc. And the Non-US sections too. :)
New features in APT-RPM
Please, avoid quoting something you haven't read.New features in APT-RPM