View Issue Details

IDProjectCategoryView StatusLast Update
0000548tcshnlspublic2017-05-16 16:13
ReporterIzaac 
Assigned ToChristos Zoulas 
PrioritynormalSeveritycrashReproducibilityalways
Status assignedResolutionopen 
PlatformMultipleOSMultipleOS VersionMultiple
Product Version 
Target VersionFixed in Version 
Summary0000548: Corruption of UTF-8 command lines in .history
DescriptionHistory gets violently corrupted and multiplied over time when dealing with UTF8 command lines.

A reproduction is provided demonstrating the (d)evolution of the situation. Ultimately, the .history file grows large enough to cause unreasonable load times and eventually exceeds process memory allocation:

tcsh current memory allocation:
free: 0 112 61 31 40 29 9 3 2 2 2 2 0 1 1 1 1 1 2 2 2 2 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
used: 0 16 515 737 280 27 19 5 7 3 2 3 2151 2 2 2 2 2 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
        Total in use: 276087456, total free: 199251296
        Allocated memory from 0x672988 to 0xffffffffffffffff. Real top at 0x1cbc4400
nbytes=67108880: Out of memory

I presently have a .logout hook to mangle it back into shape, but ...
Steps To Reproduceizaac@fire> ssh izaac@library 'rm .history'
izaac@fire> ssh izaac@library
Last login: Fri May 20 14:27:10 2016 from xxx.xxx.xxx.xxx
 _ _ _
| (_) |__ _ __ __ _ _ __ _ _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | | | |_| |
|_|_|_.__/|_| \__,_|_| \__, |
                          |___/
You have new mail.
izaac@library> echo é
é
izaac@library> history
     1 14:29 echo é
     2 14:30 history
izaac@library> exit
logout
Shared connection to library closed.
izaac@fire> ssh izaac@library
Last login: Fri May 20 14:29:54 2016 from xxx.xxx.xxx.xxx
 _ _ _
| (_) |__ _ __ __ _ _ __ _ _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | | | |_| |
|_|_|_.__/|_| \__,_|_| \__, |
                          |___/
You have new mail.
izaac@library> history
     1 14:29 echo é
     2 14:30 history
     3 14:30 exit
     4 14:30 history
izaac@library> xxd ~/.history
00000000: 232b 3134 3633 3736 3839 3938 0a65 6368 #+1463768998.ech
00000010: 6f20 c3a9 0a23 2b31 3436 3337 3639 3030 o ...#+146376900
00000020: 300a 6869 7374 6f72 790a 232b 3134 3633 0.history.#+1463
00000030: 3736 3930 3034 0a65 7869 740a 769004.exit.
izaac@library> exit
logout
Shared connection to library closed.
izaac@fire> ssh izaac@library
Last login: Fri May 20 14:30:05 2016 from xxx.xxx.xxx.xxx
 _ _ _
| (_) |__ _ __ __ _ _ __ _ _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | | | |_| |
|_|_|_.__/|_| \__,_|_| \__, |
                          |___/
You have new mail.
izaac@library> history
     1 14:29 echo é
     2 14:29 echo é
     3 14:30 history
     4 14:30 exit
     5 14:30 history
     6 14:30 xxd ~/.history
     7 14:30 exit
     8 14:30 history
izaac@library> xxd ~/.history
00000000: 232b 3134 3633 3736 3839 3938 0a65 6368 #+1463768998.ech
00000010: 6f20 c3a9 0a23 2b31 3436 3337 3638 3939 o ...#+146376899
00000020: 380a 6563 686f 20c3 83c2 a90a 232b 3134 8.echo .....#+14
00000030: 3633 3736 3930 3030 0a68 6973 746f 7279 63769000.history
00000040: 0a23 2b31 3436 3337 3639 3030 340a 6578 .#+1463769004.ex
00000050: 6974 0a23 2b31 3436 3337 3639 3030 370a it.#+1463769007.
00000060: 6869 7374 6f72 790a 232b 3134 3633 3736 history.#+146376
00000070: 3930 3130 0a78 7864 207e 2f2e 6869 7374 9010.xxd ~/.hist
00000080: 6f72 790a 232b 3134 3633 3736 3930 3136 ory.#+1463769016
00000090: 0a65 7869 740a .exit.
izaac@library> exit
logout
Shared connection to library closed.
izaac@fire> ssh izaac@library
Last login: Fri May 20 14:30:17 2016 from xxx.xxx.xxx.xxx
 _ _ _
| (_) |__ _ __ __ _ _ __ _ _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | | | |_| |
|_|_|_.__/|_| \__,_|_| \__, |
                          |___/
You have new mail.
izaac@library> history
     1 14:29 echo é
     2 14:29 echo é
     3 14:29 echo ÃÂé
     4 14:30 history
     5 14:30 exit
     6 14:30 history
     7 14:30 xxd ~/.history
     8 14:30 exit
     9 14:30 history
    10 14:30 xxd ~/.history
    11 14:30 exit
    12 14:30 history
izaac@library> xxd ~/.history
00000000: 232b 3134 3633 3736 3839 3938 0a65 6368 #+1463768998.ech
00000010: 6f20 c3a9 0a23 2b31 3436 3337 3638 3939 o ...#+146376899
00000020: 380a 6563 686f 20c3 83c2 a90a 232b 3134 8.echo .....#+14
00000030: 3633 3736 3839 3938 0a65 6368 6f20 c383 63768998.echo ..
00000040: c283 c382 c2a9 0a23 2b31 3436 3337 3639 .......#+1463769
00000050: 3030 300a 6869 7374 6f72 790a 232b 3134 000.history.#+14
00000060: 3633 3736 3930 3034 0a65 7869 740a 232b 63769004.exit.#+
00000070: 3134 3633 3736 3930 3037 0a68 6973 746f 1463769007.histo
00000080: 7279 0a23 2b31 3436 3337 3639 3031 300a ry.#+1463769010.
00000090: 7878 6420 7e2f 2e68 6973 746f 7279 0a23 xxd ~/.history.#
000000a0: 2b31 3436 3337 3639 3031 360a 6578 6974 +1463769016.exit
000000b0: 0a23 2b31 3436 3337 3639 3032 300a 6869 .#+1463769020.hi
000000c0: 7374 6f72 790a 232b 3134 3633 3736 3930 story.#+14637690
000000d0: 3233 0a78 7864 207e 2f2e 6869 7374 6f72 23.xxd ~/.histor
000000e0: 790a 232b 3134 3633 3736 3930 3239 0a65 y.#+1463769029.e
000000f0: 7869 740a xit.
izaac@library> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=
izaac@library> set | ggrep -e shell -e version -e dspmbyte -e history -e savehist
dspmbyte utf8
history 8192
savehist (8192 merge)
shell /usr/pkg/bin/tcsh
version tcsh 6.19.00 (Astron) 2015-05-21 (i386-sun-solaris) options wide,nls,dl,al,kan,rh,color,filec
izaac@library> exit
Additional InformationThis appears in SunOS 5.11.i86pc, NetBSD-6.1.5-amd64, NetBSD-7.0-amd64, NetBSD-6.1.5-i386, NetBSD-5.1.5-i386, and a variety of other platforms.
TagsNo tags attached.

Relationships

Activities

Christos Zoulas

Christos Zoulas

2016-11-27 15:50

manager   ~0001447

Can you try if 6.20 fixes it. I cannot reproduce it.
Izaac

Izaac

2017-05-16 16:13

reporter   ~0001519

Alas, no dice:

izaac@shatter> ssh izaac@library 'rm .history' 
izaac@shatter> ssh izaac@library
Last login: Tue May 16 12:00:20 2017 from 24.39.116.178
 _ _ _                          
| (_) |__  _ __ __ _ _ __ _   _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | |  | |_| |
|_|_|_.__/|_|  \__,_|_|   \__, |
                          |___/
You have new mail.
izaac@library> echo $version
tcsh 6.20.00 (Astron) 2016-11-24 (i386-sun-solaris) options wide,nls,dl,al,kan,rh,color,filec
izaac@library> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=
izaac@library> echo é
é
izaac@library> exit
logout
Shared connection to library closed.
izaac@shatter> ssh izaac@library
Last login: Tue May 16 12:00:47 2017 from 24.39.116.178
 _ _ _         
| (_) |__  _ __ __ _ _ __ _   _ 
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | |  | |_| |
|_|_|_.__/|_|  \__,_|_|   \__, |
                          |___/ 
You have new mail.
izaac@library> history
     1  12:00   echo $version
     2  12:00   locale
     3  12:01   echo é
     4  12:01   exit
     5  12:01   history
izaac@library> xxd ~/.history
00000000: 232b 3134 3934 3935 3034 3439 0a65 6368  #+1494950449.ech
00000010: 6f20 2476 6572 7369 6f6e 0a23 2b31 3439  o $version.#+149
00000020: 3439 3530 3435 310a 6c6f 6361 6c65 0a23  4950451.locale.#
00000030: 2b31 3439 3439 3530 3436 300a 6563 686f  +1494950460.echo
00000040: 20c3 a90a 232b 3134 3934 3935 3034 3636   ...#+1494950466
00000050: 0a65 7869 740a                           .exit.
izaac@library> exit
logout
Shared connection to library closed.
izaac@shatter> ssh izaac@library
Last login: Tue May 16 12:01:08 2017 from 24.39.116.178
 _ _ _                          
| (_) |__  _ __ __ _ _ __ _   _
| | | '_ \| '__/ _` | '__| | | |
| | | |_) | | | (_| | |  | |_| |
|_|_|_.__/|_|  \__,_|_|   \__, |
                          |___/
You have new mail.
izaac@library> history
     1  12:00   echo $version
     2  12:00   locale
     3  12:01   echo é
     4  12:01   echo é
     5  12:01   exit
     6  12:01   history
     7  12:01   xxd ~/.history
     8  12:01   exit
     9  12:01   history
izaac@library> xxd ~/.history
00000000: 232b 3134 3934 3935 3034 3439 0a65 6368  #+1494950449.ech
00000010: 6f20 2476 6572 7369 6f6e 0a23 2b31 3439  o $version.#+149
00000020: 3439 3530 3435 310a 6c6f 6361 6c65 0a23  4950451.locale.#
00000030: 2b31 3439 3439 3530 3436 300a 6563 686f  +1494950460.echo
00000040: 20c3 a90a 232b 3134 3934 3935 3034 3630   ...#+1494950460
00000050: 0a65 6368 6f20 c383 c2a9 0a23 2b31 3439  .echo .....#+149
00000060: 3439 3530 3436 360a 6578 6974 0a23 2b31  4950466.exit.#+1
00000070: 3439 3439 3530 3437 310a 6869 7374 6f72  494950471.histor
00000080: 790a 232b 3134 3934 3935 3034 3734 0a78  y.#+1494950474.x
00000090: 7864 207e 2f2e 6869 7374 6f72 790a 232b  xd ~/.history.#+
000000a0: 3134 3934 3935 3034 3737 0a65 7869 740a  1494950477.exit.
izaac@library> exit
logout
Shared connection to library closed.

Issue History

Date Modified Username Field Change
2016-05-20 18:40 Izaac New Issue
2016-11-27 15:49 Christos Zoulas Assigned To => Christos Zoulas
2016-11-27 15:49 Christos Zoulas Status new => assigned
2016-11-27 15:50 Christos Zoulas Status assigned => feedback
2016-11-27 15:50 Christos Zoulas Note Added: 0001447
2017-05-16 16:13 Izaac Note Added: 0001519
2017-05-16 16:13 Izaac Status feedback => assigned