View Issue Details

IDProjectCategoryView StatusLast Update
0000582tcshgeneralpublic2016-11-30 11:22
ReporterMasanari Iida 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
Product Version6.20.00 
Target VersionFixed in Version 
Summary0000582: Variable in UTF8 character conversion failure during LANG variable change
DescriptionOS: Fedora 24
tcsh-6.19.00-10.fc24
And tcsh-6.20.00 Astron 2016-11-24 (compile from source)

Value in variable seemed to be mangled after LANG change and
change back to original value.
I thought the value totally corrupted, but I found a workaround.
Set the LANG one more time.

This symptom can be reproduced all the time.
Steps To Reproduce$ cat utf8.txt
あああ

$ od -x utf8.txt
0000000 81e3 e382 8281 81e3 0a82 000a
0000013

(1) Set variable in UTF8
   $ setenv LANG ja_JP.utf8
   $ setenv USERVAL `cat utf8.txt`
   $ echo $USERVAL | od -x
    0000000 81e3 e382 8281 81e3 0a82 <= Original value
    0000012

(2) Change LANG to EUC
   $ setenv LANG ja_JP.eucJP
   $ echo $USERVAL | od -x
    0000000 ab8f 81aa 8f82 aaab 8281 ab8f 81aa 0a82 <= (A)
    0000020

(3) Change LANG back to UTF8
   $ setenv LANG ja_JP.utf8
   $ echo $USERVAL | od -x
    0000000 a4c2 a2c2 a4c2 a2c2 a4c2 a2c2 000a <= (B)
    0000015

(4) Set the LANG to UTF8 again
   $ setenv LANG ja_JP.utf8
   $ echo $USERVAL | od -x
    0000000 81e3 e382 8281 81e3 0a82 <= Same as original value
    0000012
Additional Informationあ in EUC code is A2A4.
As you see incorrect output B, you can find 3 set of A4 A2.
So It seemed the tcsh tried to output in EUC, but still wrong.
But the output should have display when I set the LANG in EUC
in step 2.

TagsNo tags attached.

Relationships

Activities

Masanari Iida

Masanari Iida

2016-11-30 11:11

reporter  

test_case.tgz (752,582 bytes)
Masanari Iida

Masanari Iida

2016-11-30 11:22

reporter   ~0001453

$ md5sum test_case.tgz
b48f890c86cbbfdfe5dc9275bab18c2d test_case.tgz

$ tar tvf test_case.tgz
-rwxrwxr-x iida/iida 1903328 2016-11-30 20:08 ./test/tcsh tcsh-6.20 binary
-rwxr-xr-x iida/iida 242 2016-11-30 20:08 ./test/test.sh script
-rw-rw-r-- iida/iida 10 2016-11-29 16:25 ./test/utf8.txt UTF8 data


$ more test.sh
#!./tcsh

setenv LANG ja_JP.utf8
setenv USRUTF8 `cat utf8.txt`
echo $USRUTF8 |od -x
echo ""
setenv LANG ja_JP.eucJP
echo $USRUTF8 |od -x
echo ""
setenv LANG ja_JP.utf8
echo $USRUTF8 |od -x
echo ""
setenv LANG ja_JP.utf8
echo $USRUTF8 |od -x

Actual Result
$ ./test.sh
0000000 81e3 e382 8281 81e3 0a82
0000012

0000000 ab8f 81aa 8f82 aaab 8281 ab8f 81aa 0a82
0000020

0000000 a4c2 a2c2 a4c2 a2c2 a4c2 a2c2 000a <= Not Expected
0000015

0000000 81e3 e382 8281 81e3 0a82
0000012

Expected Result

$ ./test.sh
0000000 81e3 e382 8281 81e3 0a82
0000012

0000000 ab8f 81aa 8f82 aaab 8281 ab8f 81aa 0a82
0000020

0000000 81e3 e382 8281 81e3 0a82 <=
0000012

0000000 81e3 e382 8281 81e3 0a82
0000012

Issue History

Date Modified Username Field Change
2016-11-30 10:21 Masanari Iida New Issue
2016-11-30 11:11 Masanari Iida File Added: test_case.tgz
2016-11-30 11:22 Masanari Iida Note Added: 0001453