[lfs-dev] Concerning iana-etc

Marty Jack martyj19 at comcast.net
Wed Jul 3 07:41:08 PDT 2019



On 07/03/2019 02:17 AM, DJ Lucas via lfs-dev wrote:
> 
> 
> On 6/28/2019 10:59 AM, Bruce Dubbs via lfs-dev wrote:
>> On 6/28/19 10:16 AM, Marty Jack via lfs-dev wrote:
>>>
>>> You could consider doing it the way I have been doing it.  In my view the iana-etc package in the book is in unmaintained status.
>>>
>>> If you download
>>> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xml
>>> https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml
>>>
>>> and run a script over them, you get /etc/protocols and /etc/services.  I can provide the script if there is interest in changing over.  I believe the script came from Arch originally.
>>> The latest revision of these files is May 31, 2019.
>>
>> I would be interested in seeing the script.  It looks like what we have dates back to 2008 although we've seen no issues about missing updates.
>>
> 
> Good call Marty. Seth's script still works on the current protocols file but it's not as clean as the Arch awk script. Unfortunately, it comments out the whole of the service-names-port-numbers.txt. Any files from IANA, other than the xml files, are a mess to work with. I did eventually get to the output that Marty suggested that way (with the commented additions below). Services from the .csv took around 80 seconds to process for all the special cases using only sed and was 36 lines long! Fun exercise, but obviously not very efficient. :-) In comes Marty's suggestion (with a couple of minor modifications):
> 
> Per the Arch PKGBUILD at https://git.archlinux.org/svntogit/packages.git/plain/trunk/PKGBUILD?h=packages/iana-etc for LFS it would look something like this to get us close to Seth's original output - I added the commented descriptions, fixed spacing (field length should be also be 15 for proper alignment in protocols, not 13), added auto downloading over FTP so that it can run from a cron job or systemd timer, and finally added local modifications (example data file below the script, but its just properly formatted lines). Here is a first take:
> 
> ===================================/usr/sbin/update-iana-etc===================================
> #!/bin/sh
> # Begin /usr/sbin/update-iana-etc
> 
> # Simple script to update IANA protocols and services files
> 
> if [ $# -ne 0 ]; then
>         echo "usage: update-iana-etc" >&2
>         exit 2
> fi
> 
> uid=$(id -u)
> if [ "$uid" != 0 ]; then
>         echo "update-iana-etc: running as non-root user! Exiting..." >&2
>         exit 0
> fi
> 
> DOWNLOADED=false
> DATE=`date -I`
> TEMPDIR=`mktemp -d` &&
> cd $TEMPDIR &&
> 
> # Protocols
> ftp -inv ftp.iana.org << "EOF" &&
> user anonymous anonymous
> pasv
> cd assignments
> cd protocol-numbers
> get protocol-numbers.xml
> bye
> EOF
> gawk -F"[<>]" '
> BEGIN { print "# IANA protocols for LFS\n" }
> (/<record/) { v=n="" }
> (/<value/) { v=$3 }
> (/<name/ && $3!~/ /) { n=$3 }
> (/<description/) {d=$3}
> (/<\/record/ && n && v!="") { printf "%-15s %3i %-15s\t# %s\n", tolower(n),v,n,d }
> ' protocol-numbers.xml > protocols &&
> 
> # Services
> ftp -inv ftp.iana.org << "EOF" &&
> user anonymous anonymous
> pasv
> cd assignments
> cd service-names-port-numbers
> get service-names-port-numbers.xml
> bye
> EOF
> gawk -F"[<>]" '
> BEGIN { print "# IANA services for LFS\n" }
> (/<record/) { n=u=p=c="" }
> (/<name/ && !/\(/) { n=$3 }
> (/<number/) { u=$3 }
> (/<protocol/) { p=$3 }
> (/<description/) { d=$3 }
> (/Unassigned/ || /Reserved/ || /historic/) { c=1 }
> (/<\/record/ && n && u && p && !c) { printf "%-15s %5i/%s\t# %s\n", n,u,p,d }
> ' service-names-port-numbers.xml > services &&
> echo "Successfully downloaded...." &&
> DOWNLOADED=true
> 
> # Sanity checking
> if [ "${DOWNLOADED}" != "true" ]; then
>     echo "Somthing went wrong obtaining the upstream files. Exiting..."
>     exit 3
> fi
> 
> # Add custom protocols
> if [ -f /etc/sysconfig/protocols.add ]; then
>     cat /etc/sysconfig/protocols.add >> protocols
> fi
> 
> # Add custom services
> if [ -f /etc/sysconfig/services.add ]; then
>     cat /etc/sysconfig/services.add >> services
> fi
> 
> # Install them...
> mv -v /etc/protocols /etc/protocols-${DATE}
> install -vm644 protocols /etc/protocols
> mv -v /etc/services /etc/services-${DATE}
> install -vm644 services /etc/services
> 
> echo "Update complete!"
> 
> # Clean up
> rm -rf ${TEMPDIR}
> 
> # End /usr/sbin/update-iana-etc
> ===============================================================================================
> 
> 
> 
> ==================================/etc/sysconfig/services.add==================================
> smtps             465/tcp       # Simple Mail Transport Protocol over TLS
> ===============================================================================================
> 
> Just a quick (but functional) mock-up using update-ca-certificates as an example. Can clean it up if this would be good for the book. Would need to move to chapter 7, after networking configuration (won't work without /etc/resolv.conf).
> 
> --DJ
> 

Waiting until Chapter 7 would be an issue because quite a few test systems rely on having this in /etc.
My analyzer detects Python Python2 coreutils git glib glibc libevent openssl perl tcl
Wouldn't it work to download the initial copies as if they are package tarballs and have them in the sources directory and then only use the network when you wanted to refresh?
Thanks for taking the time to work on getting the script in better shape.


More information about the lfs-dev mailing list