[lfs-dev] Concerning iana-etc

Bruce Dubbs bruce.dubbs at gmail.com
Wed Jul 3 08:06:07 PDT 2019


On 7/3/19 1:17 AM, DJ Lucas via lfs-dev wrote:
> 
> 
> On 6/28/2019 10:59 AM, Bruce Dubbs via lfs-dev wrote:
>> On 6/28/19 10:16 AM, Marty Jack via lfs-dev wrote:
>>>
>>> You could consider doing it the way I have been doing it.  In my view 
>>> the iana-etc package in the book is in unmaintained status.
>>>
>>> If you download
>>> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xml 
>>>
>>> https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml
>>>
>>> and run a script over them, you get /etc/protocols and 
>>> /etc/services.  I can provide the script if there is interest in 
>>> changing over.  I believe the script came from Arch originally.
>>> The latest revision of these files is May 31, 2019.
>>
>> I would be interested in seeing the script.  It looks like what we 
>> have dates back to 2008 although we've seen no issues about missing 
>> updates.
>>
> 
> Good call Marty. Seth's script still works on the current protocols file 
> but it's not as clean as the Arch awk script. Unfortunately, it comments 
> out the whole of the service-names-port-numbers.txt. Any files from 
> IANA, other than the xml files, are a mess to work with. I did 
> eventually get to the output that Marty suggested that way (with the 
> commented additions below). Services from the .csv took around 80 
> seconds to process for all the special cases using only sed and was 36 
> lines long! Fun exercise, but obviously not very efficient. :-) In comes 
> Marty's suggestion (with a couple of minor modifications):
> 
> Per the Arch PKGBUILD at 
> https://git.archlinux.org/svntogit/packages.git/plain/trunk/PKGBUILD?h=packages/iana-etc 
> for LFS it would look something like this to get us close to Seth's 
> original output - I added the commented descriptions, fixed spacing 
> (field length should be also be 15 for proper alignment in protocols, 
> not 13), added auto downloading over FTP so that it can run from a cron 
> job or systemd timer, and finally added local modifications (example 
> data file below the script, but its just properly formatted lines). Here 
> is a first take:
> 
> ===================================/usr/sbin/update-iana-etc=================================== 
> 
> #!/bin/sh
> # Begin /usr/sbin/update-iana-etc
> 
> # Simple script to update IANA protocols and services files
> 
> if [ $# -ne 0 ]; then
>          echo "usage: update-iana-etc" >&2
>          exit 2
> fi
> 
> uid=$(id -u)
> if [ "$uid" != 0 ]; then
>          echo "update-iana-etc: running as non-root user! Exiting..." >&2
>          exit 0
> fi
> 
> DOWNLOADED=false
> DATE=`date -I`
> TEMPDIR=`mktemp -d` &&
> cd $TEMPDIR &&
> 
> # Protocols
> ftp -inv ftp.iana.org << "EOF" &&
> user anonymous anonymous
> pasv
> cd assignments
> cd protocol-numbers
> get protocol-numbers.xml
> bye
> EOF
> gawk -F"[<>]" '
> BEGIN { print "# IANA protocols for LFS\n" }
> (/<record/) { v=n="" }
> (/<value/) { v=$3 }
> (/<name/ && $3!~/ /) { n=$3 }
> (/<description/) {d=$3}
> (/<\/record/ && n && v!="") { printf "%-15s %3i %-15s\t# %s\n", 
> tolower(n),v,n,d }
> ' protocol-numbers.xml > protocols &&
> 
> # Services
> ftp -inv ftp.iana.org << "EOF" &&
> user anonymous anonymous
> pasv
> cd assignments
> cd service-names-port-numbers
> get service-names-port-numbers.xml
> bye
> EOF
> gawk -F"[<>]" '
> BEGIN { print "# IANA services for LFS\n" }
> (/<record/) { n=u=p=c="" }
> (/<name/ && !/\(/) { n=$3 }
> (/<number/) { u=$3 }
> (/<protocol/) { p=$3 }
> (/<description/) { d=$3 }
> (/Unassigned/ || /Reserved/ || /historic/) { c=1 }
> (/<\/record/ && n && u && p && !c) { printf "%-15s %5i/%s\t# %s\n", 
> n,u,p,d }
> ' service-names-port-numbers.xml > services &&
> echo "Successfully downloaded...." &&
> DOWNLOADED=true
> 
> # Sanity checking
> if [ "${DOWNLOADED}" != "true" ]; then
>      echo "Somthing went wrong obtaining the upstream files. Exiting..."
>      exit 3
> fi
> 
> # Add custom protocols
> if [ -f /etc/sysconfig/protocols.add ]; then
>      cat /etc/sysconfig/protocols.add >> protocols
> fi
> 
> # Add custom services
> if [ -f /etc/sysconfig/services.add ]; then
>      cat /etc/sysconfig/services.add >> services
> fi
> 
> # Install them...
> mv -v /etc/protocols /etc/protocols-${DATE}
> install -vm644 protocols /etc/protocols
> mv -v /etc/services /etc/services-${DATE}
> install -vm644 services /etc/services
> 
> echo "Update complete!"
> 
> # Clean up
> rm -rf ${TEMPDIR}
> 
> # End /usr/sbin/update-iana-etc
> =============================================================================================== 
> 
> 
> 
> 
> ==================================/etc/sysconfig/services.add================================== 
> 
> smtps             465/tcp       # Simple Mail Transport Protocol over TLS
> =============================================================================================== 
> 
> 
> Just a quick (but functional) mock-up using update-ca-certificates as an 
> example. Can clean it up if this would be good for the book. Would need 
> to move to chapter 7, after networking configuration (won't work without 
> /etc/resolv.conf).

This is an interesting idea, but I think for LFS it would be better to 
host the files on anduin for download.  The update is really not very 
critical.  After all we have been working with the current version since 
2008 without issue.

(Looking at the current XML files, protocols was last updated 2017-10-13 
but services was updated 2019-07-02 (yesterday). )

Right now we create a tarball with all packages that is available for 
download.  After that, for LFS, an internet connection is not needed. 
I'd like to keep it that way.

We could but the script on anduin to keep the files updated like we do 
with firmware.  We could also put the update script into BLFS similar to 
make-ca.

Another thought for BLFS is to just download the xml files with wget and 
use xsltproc and a custom .xsl file to parse the xml.

   -- Bruce


More information about the lfs-dev mailing list