[lfs-dev] Once more: Package Management
dj at linuxfromscratch.org
Sat May 19 10:15:23 PDT 2012
On 05/19/2012 08:26 AM, Jeremy Huntwork wrote:
> I've been holding back bringing this up on-list for a while because I
> intended to do the bulk of the work and then present a working system to
> the community for comment and review. I still intend to do that, but
> given some recent discussions, I think the time is right to bring this
> up and see where it goes.
> (Sorry for the cross posting, but since it involves both projects, I
> wanted to make sure all saw it - if possible, please reply on lfs-dev.)
> 1. Adjust LFS/BLFS to auto-generate build recipes for individual
> packages that a packaging tool can use to create binary packages with
> meta information included such as dependency tracking.
> 2. Store 'official' copies of those binary packages in a developer
> package repository so that when developing (primarily BLFS) a dev can
> automatically pull and install into a build environment the dependencies
> he needs to build and create an individual package.
> 3. Create a standard workflow and tools whereby a developer can
> accomplish #2 and edit the book accordingly in an efficient, reliable way.
> (B)LFS-style development by hand becomes a huge undertaking. BLFS _is_ a
> huge undertaking - and it's a difficult beast to maintain. In order to
> create a stable reference page in BLFS an editor has to have spent the
> time to build all of LFS, tweak it, go through current BLFS, manually
> install dependency packages and then carefully document the package he
> builds. No two developers are guaranteed to have the same environment,
> either, so accuracy and stability becomes an issue.
> The same is true of the LiveCD project, and is the main reason why it
> sits dead today. It is difficult to maintain when there are no packaged
> binaries to draw from and the entire thing is a scripted source build.
> Let me just say now that because (B)LFS is primarily (and probably
> should always be) about educating, I don't think we should require
> end-users to use package management. Mainly the proposal is intended to
> assist in development. However, if we have taken the time to incorporate
> PM in our dev workflow, we can make the option of building via PM tools
> available to readers if they wish to use it for themselves and build
> their own package repository.
> (The following details assume pacman is the package manager chosen, but
> it could conceivably be another, such as rpm5.)
> 1. The end of LFS chapter 5 is modified to (optionally) build the
> packaging tool and any dependencies it has.
> 2. LFS chapter 6 is modified so that for each package a build recipe
> (e.g. PKGBUILD file) is generated from the book's source. A reader is
> given the option of carrying out the build manually or via the packaging
> 3. LFS devs create official copies of the binary packages and install
> them to a semi-public package repository
> 4. BLFS is modified to also generate build recipes (PKGBUILD files) from
> its source
> 5. As BLFS devs work on individual packages, they can use the defined
> workflow and tools to generate a chroot environment with only the
> packages they need for build-time dependencies - they can then both
> produce a binary version of the package as well as document what they've
> done, the end product being a BLFS page which will generate/correspond
> to the PKGBUILD file.
> 6. BLFS dev updates the BLFS binary package repository with the
> 'official' package and other devs can now draw from and use those when
> working on their respective package.
> There are, I'm sure, both positives and negatives to this proposal, and
> I'd like to hear everyone's thoughts.
> I intended to do all the development in the jh branch, but if there are
> more parties interested in helping this development, then I'm also open
> to sharing the workload and perhaps creating an environment where this
> can be done together.
I do believe this was consistent with Gerard's vision a few years back.
I have (as I'm sure, most other long timers have) been using a
home-grown system for some time. In my own case, that involves manually
editing my scripts on every pass to avoid if-then constructs, and I have
actually kept it that way so that I have no surprises. I do this because
I am forced to reevaluate if a package build breaks and that forces me
to keep in touch with the readers' experience, at least somewhat.
The immediate downside of your proposal for me, is that the optional
dependencies must be accounted for in some cases by individual configure
flags, which results in a logical if-then in the build description file,
and sometimes even compound flags which result in nested if-then
constructs. Fortunately, that is not a deal breaker for me if the
readers get the same treatment (which seems to be the case), but this
does hard code optional dependencies for the pre-packaged installations.
This is both good and bad. From a development standpoint, it won't take
me a week to build a fairly standard system to test a simple package
upgrade, but that still means manual (or maybe only partially automated)
builds for omitting optional deps in the build process up to the point
that I need. Please prime flame throwers now! Does a recommended
dependency change meaning once again? :-)
I've never used pacman, so I can't speak for the complexity of it, but
from my fuzzy memory, it did fit fairly well with our purposes (optional
deps were supported). On the surface, I'm game. A few more questions though.
What separates LFS from say Arch, Gentoo, T2... at that point? No mile
long USE flags or complex switching scripts I presume, but I know little
about the other two. I've included some of their work in BLFS in the
past, but that is about it.
My hope is that build order is still a manual process where the user
determines build order herself. Dependency checking is done only at
build time and that optional deps remain optional. If there will be
automation, how do we determine what optional deps to pull in?
What benefits can we expect for users who have already done the leg work
to use other packaging methods?
For instance, with what is put into the book, could a logical parser be
made to genarate a spec file? How about dpkg?
While I admit that the .d/ directory reduces this concern immensely,
what happens to configuration files that must be modified by additional
installations? Do we keep diffs indefinitely, or do we create a comment
marker for each package's additions, modify the package to support some
sort of include directive? I can see scripting getting somewhat
difficult here, not impossible, but difficult.
If it is to be done, it should be designed with all goals in mind from
the start. I have long suggested that LFS and BLFS move to installations
from DESTDIR (or equivalent) as this does about 60% of building your src
rpm without defining a particular package manager. The other 40% is just
transposing what is in the book to spec, and could be automated, but
even this has had lackluster support in the past despite the obvious
Sorry I didn't use the bullet points...out of time for today.
-- DJ Lucas
This message has been scanned for viruses and
dangerous content, and is believed to be clean.
More information about the lfs-dev