What happened to the ß?

[JamesC]

I have a text file, x, containing some "ß" characters. When I use k=FileRead (x), k contains no "ß" characters. So if I loop though the file reading lines, and writing them to a second file, the second file will be shorter than the first, rather than identical.

This is a big problem if the file is in German!

What on earth is going on??

Thanks

James

[agraham]

You may have opened the file in ASCII mode rather than Unicode.

[JamesC]

Thanks for the quick reply agraham - I guess that must be it!

[JamesC]

Using FileOpen(x,filename,cRead,,cAscii), when I FileRead(x), the "ß" characters appear as "?" characters. So writing all the lines to a second file results in a file of the same length.

Using FileOpen(x,filename,cRead), the "ß" characters disappear altogether as I described earlier.

This is rather strange

[Erel]

FileOpen(x,filename,cRead) should be able to handle these characters.
Can you upload a text file with these characters?

[klaus]

If the original file was saved with ANSI coding, you cannot read the special characters.
If you read the text file with Notepad and save it back with UTF-8 coding
you will get the special characters in B4PPC.
I have made a test with Notepad, I created a file with special characters in, saved it with ANSI coding, and read this file in B4PPC with FileOpen(c1,"Test.txt",cRead). The special characters don't appear.
Reading back the same file with Notepad and saving it with UTF-8 coding, the special characters appear normaly in B4PPC.
The strange thing is that with Notepad the special characters are normaly displayed whatever the coding is.

Best regards

[JamesC]

Originally Posted by Erel

FileOpen(x,filename,cRead) should be able to handle these characters.
Can you upload a text file with these characters?

I created pretty much a random file in Pocket Word to test what was happening, like this:

Aßßß
Bßcßßß
C???ß
de

This is saved as a txt file, saying No to the dialog which asks if I want to save it as word instead. When it is reopened in Pocket Word, it appears unchanged.

[JamesC]

Attached txt file, although it looks different in Notepad on the PC, with "ß" characters changed to "฿"!! I guess this is something to do with pocket word.

A฿฿฿
B฿c฿฿฿
C???฿
dc


Also attached is the beginning of original file where I first noticed the problem.

[JamesC]

attached.

[Erel]

The last file attached was saved encoded with ASCII and not UTF-8.
It doesn't contain any "special" characters.

You can use Notepad to create the file and then choose Save As - Encoding - UTF8.