Finding invalid characters in UTF-8 files

Albert Wiersch

When loading a UTF-8 file (or any file with a certain encoding), does SyntaxEdit have a way of reporting any bad or invalid characters in the file/document?

Does anyone have any suggestions on the best way of detecting invalid characters in files (particularly UTF-8 files because they are the most popular)?

Thank you.

Eugene Balabuev (LMD)

Hi,

LMDSyntaxEdit does not perform any extensive checking. There are some internal checking happens in used WinAPI functions, but there no more that in standard TStrings, for example.

You probably should look at WinAPI conversion functions; they should report errors, I think.

Albert Wiersch

So I would have to read in the document myself and check it with the WinAPI conversion functions (or something else) and then load it into SyntaxEdit? It sounds like it would work but I don't like doing the "double load" (it seems inefficient).

Thanks.

Eugene Balabuev (LMD)

You can crete smething like proxy stream to do this with single loading from file.

Eugene Balabuev (LMD)

I remember we alredy discussed this some time ago...

Albert Wiersch

Yes, I think this or a similar issue came up awhile back. I am not sure how to do a proxy stream but will look into it if I get around to trying to implement better character checking.