[lfs-dev] Concerning iana-etc

DJ Lucas blfs-dev at lucasit.com
Sun Jul 7 15:23:20 PDT 2019



On 7/3/2019 10:06 AM, Bruce Dubbs via lfs-dev wrote:
> On 7/3/19 1:17 AM, DJ Lucas via lfs-dev wrote:
>>
>>
>> On 6/28/2019 10:59 AM, Bruce Dubbs via lfs-dev wrote:
>>> On 6/28/19 10:16 AM, Marty Jack via lfs-dev wrote:
>>>>
>>>> You could consider doing it the way I have been doing it. In my 
>>>> view the iana-etc package in the book is in unmaintained status.
>>>>
>>>> If you download
>>>> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xml 
>>>>
>>>> https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml
>>>>
>>>> and run a script over them, you get /etc/protocols and 
>>>> /etc/services.  I can provide the script if there is interest in 
>>>> changing over.  I believe the script came from Arch originally.
>>>> The latest revision of these files is May 31, 2019.
>>>
>>> I would be interested in seeing the script.  It looks like what we 
>>> have dates back to 2008 although we've seen no issues about missing 
>>> updates.
>>>
>>
>> Good call Marty. Seth's script still works on the current protocols 
>> file but it's not as clean as the Arch awk script. Unfortunately, it 
>> comments out the whole of the service-names-port-numbers.txt. Any 
>> files from IANA, other than the xml files, are a mess to work with. I 
>> did eventually get to the output that Marty suggested that way (with 
>> the commented additions below). Services from the .csv took around 80 
>> seconds to process for all the special cases using only sed and was 
>> 36 lines long! Fun exercise, but obviously not very efficient. :-) In 
>> comes Marty's suggestion (with a couple of minor modifications):
>>
>> Per the Arch PKGBUILD at 
>> https://git.archlinux.org/svntogit/packages.git/plain/trunk/PKGBUILD?h=packages/iana-etc 
>> for LFS it would look something like this to get us close to Seth's 
>> original output - I added the commented descriptions, fixed spacing 
>> (field length should be also be 15 for proper alignment in protocols, 
>> not 13), added auto downloading over FTP so that it can run from a 
>> cron job or systemd timer, and finally added local modifications 
>> (example data file below the script, but its just properly formatted 
>> lines). Here is a first take:
>>
>> ===================================/usr/sbin/update-iana-etc=================================== 
>>
>> #!/bin/sh
>> # Begin /usr/sbin/update-iana-etc
>>
>> # Simple script to update IANA protocols and services files
>>
>> if [ $# -ne 0 ]; then
>>          echo "usage: update-iana-etc" >&2
>>          exit 2
>> fi
>>
>> uid=$(id -u)
>> if [ "$uid" != 0 ]; then
>>          echo "update-iana-etc: running as non-root user! Exiting..." 
>> >&2
>>          exit 0
>> fi
>>
>> DOWNLOADED=false
>> DATE=`date -I`
>> TEMPDIR=`mktemp -d` &&
>> cd $TEMPDIR &&
>>
>> # Protocols
>> ftp -inv ftp.iana.org << "EOF" &&
>> user anonymous anonymous
>> pasv
>> cd assignments
>> cd protocol-numbers
>> get protocol-numbers.xml
>> bye
>> EOF
>> gawk -F"[<>]" '
>> BEGIN { print "# IANA protocols for LFS\n" }
>> (/<record/) { v=n="" }
>> (/<value/) { v=$3 }
>> (/<name/ && $3!~/ /) { n=$3 }
>> (/<description/) {d=$3}
>> (/<\/record/ && n && v!="") { printf "%-15s %3i %-15s\t# %s\n", 
>> tolower(n),v,n,d }
>> ' protocol-numbers.xml > protocols &&
>>
>> # Services
>> ftp -inv ftp.iana.org << "EOF" &&
>> user anonymous anonymous
>> pasv
>> cd assignments
>> cd service-names-port-numbers
>> get service-names-port-numbers.xml
>> bye
>> EOF
>> gawk -F"[<>]" '
>> BEGIN { print "# IANA services for LFS\n" }
>> (/<record/) { n=u=p=c="" }
>> (/<name/ && !/\(/) { n=$3 }
>> (/<number/) { u=$3 }
>> (/<protocol/) { p=$3 }
>> (/<description/) { d=$3 }
>> (/Unassigned/ || /Reserved/ || /historic/) { c=1 }
>> (/<\/record/ && n && u && p && !c) { printf "%-15s %5i/%s\t# %s\n", 
>> n,u,p,d }
>> ' service-names-port-numbers.xml > services &&
>> echo "Successfully downloaded...." &&
>> DOWNLOADED=true
>>
>> # Sanity checking
>> if [ "${DOWNLOADED}" != "true" ]; then
>>      echo "Somthing went wrong obtaining the upstream files. Exiting..."
>>      exit 3
>> fi
>>
>> # Add custom protocols
>> if [ -f /etc/sysconfig/protocols.add ]; then
>>      cat /etc/sysconfig/protocols.add >> protocols
>> fi
>>
>> # Add custom services
>> if [ -f /etc/sysconfig/services.add ]; then
>>      cat /etc/sysconfig/services.add >> services
>> fi
>>
>> # Install them...
>> mv -v /etc/protocols /etc/protocols-${DATE}
>> install -vm644 protocols /etc/protocols
>> mv -v /etc/services /etc/services-${DATE}
>> install -vm644 services /etc/services
>>
>> echo "Update complete!"
>>
>> # Clean up
>> rm -rf ${TEMPDIR}
>>
>> # End /usr/sbin/update-iana-etc
>> =============================================================================================== 
>>
>>
>>
>>
>> ==================================/etc/sysconfig/services.add================================== 
>>
>> smtps             465/tcp       # Simple Mail Transport Protocol over 
>> TLS
>> =============================================================================================== 
>>
>>
>> Just a quick (but functional) mock-up using update-ca-certificates as 
>> an example. Can clean it up if this would be good for the book. Would 
>> need to move to chapter 7, after networking configuration (won't work 
>> without /etc/resolv.conf).
>
> This is an interesting idea, but I think for LFS it would be better to 
> host the files on anduin for download.  The update is really not very 
> critical.  After all we have been working with the current version 
> since 2008 without issue.
>
> (Looking at the current XML files, protocols was last updated 
> 2017-10-13 but services was updated 2019-07-02 (yesterday). )
>
> Right now we create a tarball with all packages that is available for 
> download.  After that, for LFS, an internet connection is not needed. 
> I'd like to keep it that way.
>
> We could but the script on anduin to keep the files updated like we do 
> with firmware.  We could also put the update script into BLFS similar 
> to make-ca.
>
> Another thought for BLFS is to just download the xml files with wget 
> and use xsltproc and a custom .xsl file to parse the xml.
>
>   -- Bruce
https://github.com/djlucas/iana-etc <-- In the interim. I emailed Seth, 
haven't heard from him in several years.

--DJ



More information about the lfs-dev mailing list