[lfs-dev] Possible problem with grep-2.23

Bruce Dubbs bruce.dubbs at gmail.com
Mon Feb 15 14:33:24 PST 2016


Bruce Dubbs wrote:
> Ken Moffat wrote:
>> This is my logging problem.  A while ago, I reworked my logging to
>> get a list of everything "interesting" before a package was built,
>> and again after.  This uses a long pipeline to exclude things which
>> are not interesting (in chroot I build in /building, log in /logs and
>> keep internal markers in /misc).  I also use ls -l on all the files of
>> interest, to get changes.
>>
>> What seems to be happening is that grep-2.23 used with -v discards
>> too much, grep-2.22 was ok.
>>
>> There is a run just before each package, and another at the end.
>> When I completed chapter 5 I did not pay much attention to the
>> screen, I merely checked that it had all completed - but I think
>> that there was no problem there (the logs contain both programs and
>> symlinks, e.g. xz shows /tools/bin/lzcat - symlink, and
>> /tools/bin/xz - program)
>>
>> I then left it building chapter6, with a command to suspend when
>> that script finished.  Came back to it this afternoon, saw error
>> messages interspersed with the normal messages telling me what it
>> was doing, e.g.
>>
>> testing man-db-2.7.5.tar.xz
>> km_identify: directory is man-db-2.7.5
>> ls: cannot access 'Binary file (standard input) matches': No such file
>> or directory
>> configure
>> make
>> check
>> install
>> ls: cannot access 'Binary file (standard input) matches': No such file
>> or directory
>>
>> At first I thought these messages were just an annoyance, but on
>> looking further I discovered that files (but perhaps not symlinks)
>> in /bin and /usr/bin were not being listed.
>>
>> When I completed chapter 5 I did not pay much attention to the
>> screen, I merely checked that it had all completed - but I think
>> that there was no problem there (the logs contain both programs and
>> symlinks, e.g. xz shows /tools/bin/lzcat - symlink, and
>> /tools/bin/xz - program)
>>
>> My commands have evolved, the listing comes down to the following,
>> reformatted to one part per line with numbers added at end
>>
>> find -H / -xdev -type f -o -type l | \    # 001
>> sed 's%^/mnt/lfs%%' | \            # 002
>> grep -v '^/logs' | \            # 003
>> grep -v '^/misc' | \            # 004
>> grep -v '^/building' | \        # 005
>> grep -v '^/tmp' |  \            # 006
>> sed 's/\x20/\\\ /g' | \            # 007
>> xargs ls -l | \                # 008
>> sed 's%\([^/]*\)\(.*\)%\2 \1%'        # 009
>>
>> Taking the files present after xz has been installed in chapter 6, I
>> ran those commands to produce output in /tmp : first only the find
>> (list001], then find and sed [ list002], and so forth.  When I got
>> to adding the xargs command I saw the error message
>>
>> ls: cannot access 'Binary file (standard input) matches': No such file
>> or directory
>>
>> so I tried adding 2>/dev/null and confirmed that silenced it and
>> threw nothing away.  But that is a side issue - without the message,
>> I would not have looked.
>>
>> All is fine in the first two steps, but removing any line which
>> starts '/logs' does far more.
>>
>> root in chroot /# wc -l /tmp/list002 /tmp/list003
>>    37121 /tmp/list002
>>     1495 /tmp/list003
>>
>> and the diff starts:
>>
>> --- /tmp/list002        2016-02-15 20:38:16.966065542 +0000
>> +++ /tmp/list003        2016-02-15 20:38:39.717790746 +0000
>> @@ -1492,35630 +1492,4 @@
>>   /usr/share/doc/kbd-2.0.3/iso8859-4.txt
>>   /usr/share/doc/kbd-2.0.3/utf/utflist
>>   /usr/share/doc/kbd-2.0.3/utf/ethiopic
>> -/usr/share/doc/kbd-2.0.3/utf/<E2><99><AA><E2><99><AC>
>> -/usr/share/doc/kbd-2.0.3/utf/utfdemo
>> -/usr/share/doc/kbd-2.0.3/utf/README
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-21.html
>> -/usr/share/doc/kbd-2.0.3/iso8859-6.txt
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-9.html
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-1.html
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-17.html
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-13.html
>> -/usr/share/doc/kbd-2.0.3/kbd.FAQ-16.html
>> -/usr/share/doc/kbd-2.0.3/iso8859-14.txt
>> -/usr/share/doc/kbd-2.0.3/iso8859-1.txt
>> -/usr/share/doc/kbd-2.0.3/iso8859-10.txt
>>
>> and finally gets to
>> -/bin/cat
>> -/bin/cp
>> -/bin/bzcat
>> -/bin/lsattr
>> +Binary file (standard input) matches
>>
>> Can anybody confirm this sort of problem with grep-2.23 ?  Or am I
>> misusing those grep -v commands ?
>>
>> FWIW, the final sed rearranges the fields so that the filename comes
>> first - that aids the comparison.
>
> I do subscribe to the grep mailing list.  There was a change in 2.23 that
> IIRC was related to this.  What I see as the problem is the line:
>
> /usr/share/doc/kbd-2.0.3/utf/<E2><99><AA><E2><99><AC>
>
> What happens if you prefix your grep commands with LC_ALL=en_US.utf8?
> Or the GB equivalent.
>
> What I get is:
>
> $ ls /usr/share/doc/kbd-2.0.3/utf | grep -v utf
> README
> ethiopic
> Binary file (standard input) matches
>
> $ ls /usr/share/doc/kbd-2.0.3/utf | LC_ALL=en_US.utf8 grep -v utf
> README
> ethiopic
> ♪♬

And the contents of that weird file:

The file with a strange name in this directory is no mistake.
It shows that Linux handles filenames in Unicode (UTF-8)
without ever realizing that they are in Unicode.
[However, some old tar's are unhappy with this filename.]

If you want to see it (the filename consists of two musical notes):
% setfont               # load PC font, just to be sure
% setfont -u def.uni    # load Unicode mapping table
% echo -e '\033%8'      # set Unicode output mode
% ls -N                 # show filenames (raw)
% echo -e '\033%@'      # set Latin1 output mode again.

If you want to remove it:
% rm ./??????

If you want to input Unicode:
% kbd_mode -u           # Be careful! Do this only with a kbd_mode
                         # from kbd-0.88.tar.gz or later!
and next tell loadkeys what key combinations correspond to which
Unicode values. (See also the contents of that musical file.)

See also the scripts unicode_start and unicode_stop in ../../src.
------

Note that the rm command above will also remove the README file.

   -- Bruce




More information about the lfs-dev mailing list