SyntaxEdit crash with syntax highlighting

Albert Wiersch

A customer sent me an HTML document that seems to cause SyntaxEdit to crash when it is opened with syntax highlighting enabled (stack overflow error).

I have already increased the stack size in the C++ Linker->Output setting to 0x400000 to address another issue that I think I had awhile back.

I'd like to send you this file and I received permission from the customer if "provided that they do not further
distribute the file and that the file is deleted after their evaluation."

Can I send this file to someone via email to check and then delete the file afterwards? Thank you.

Here's the top of the call stack:

:009efaba Lmdarrayutils::_16387 + 0x6

:009efc4c ; LMDCharArrayCopy

:00a2b1ed Lmdseddocument::_16582 + 0x29

:009d1f75 ; MatchPrim

:009d1d6a ; MatchPrim

:009d1dd5 ; MatchPrim

:009d1cdf ; MatchPrim

:009d1fc3 ; MatchPrim

:009d1d6a ; MatchPrim

:009d1dd5 ; MatchPrim

:009d1cdf ; MatchPrim

:009d1fc3 ; MatchPrim

:009d1d6a ; MatchPrim

:009d1dd5 ; MatchPrim

:009d1cdf ; MatchPrim

:009d1fc3 ; MatchPrim

:009d1d6a ; MatchPrim

:009d1dd5 ; MatchPrim

:009d1cdf ; MatchPrim

:009d1fc3 ; MatchPrim

... bunch of MatchPrim lines..... calling itself too many times?

balabuev

"guard variable" - yes, sure.

Eugene Balabuev (LMD)

Hi,

Please send the file to our support mail. We guarantee that it will not be distributed or shared in any way.

Albert Wiersch

Thank you. I've emailed the file. Please let me know if you don't receive it.

Eugene Balabuev (LMD)

Hi,

As was previous time, the error happens due to very long lines in the file (lines: 577 and 600). This time the line is so long that no stack increase can help.

Unfortunately, there is very low probability that that piece of LMD RegExp engine is possible to rewrite in iterative manner without recursion.

Albert Wiersch

So there is no fix for this? Can anything be done to prevent the stack overflow from crashing the program? This seems like a pretty bad problem.

Albert Wiersch

Well it looks like another customer just ran into this issue. The editor component SHOULD NOT CRASH. Long lines should not cause it to crash. It is terrible for the customer and for me. Isn't there something you can do about this so that it doesn't cause my entire application to crash?

:008af807 Lmdarrayutils::_16387 + 0x7

:008af998 ; LMDCharArrayCopy

:009282bd Lmdseddocument::_16582 + 0x29

:008c3102 ; MatchPrim

:008c2efb ; MatchPrim

:008c2f66 ; MatchPrim

:008c2e70 ; MatchPrim

:008c3150 ; MatchPrim

:008c2efb ; MatchPrim

:008c2f66 ; MatchPrim

:008c2e70 ; MatchPrim

:008c3150 ; MatchPrim

:008c2efb ; MatchPrim

:008c2f66 ; MatchPrim

:008c2e70 ; MatchPrim

:008c3150 ; MatchPrim

:008c2efb ; MatchPrim

:008c2f66 ; MatchPrim

:008c2e70 ; MatchPrim

:008c3150 ; MatchPrim

:008c2efb ; MatchPrim

:008c2f66 ; MatchPrim

:008c2e70 ; MatchPrim

...and lots more MatchPrim lines

balabuev

Hi,

So far, we have still no working solution. We will take alook again after our upcoming major release.

Albert Wiersch

OK, thanks. I hope you can find a away to at least avoid a crash.

I don't know if it's possible or not, but perhaps you can stop the recursion before you run out of stack space or something if you can't find a better solution.

Albert Wiersch

Well, this problem keeps popping up occasionally... and is terrible because it crashes my entire program.

If the recursion cannot be converted into an iterative design to avoid overflowing the stack, then can you add a "guard variable" that keeps track of the recursion depth and stops/aborts the recursion before it is likely to overflow the stack?

Albert Wiersch

Thanks. If that can prevent the stack overflow then it will be very helpful.

balabuev

Hi,

I've added the following code to MatchPrim method:

  if FMatchPrimDepth >= MaxMatchPrimDepth then

  begin

    if RaiseMatchPrimDepthError then

      Assert(False);

    Result := False;

    Exit;

  end;

So, you can set RaiseMatchPrimDepthError global variable to False (Its True by default) to ignore deep recursion. As well, you can change MaxMatchPrimDepth global variable value (I've set it to 1000 by default). However, I guess you realize, that this can produce side effects.

Albert Wiersch

Great. I'll "play" with this when the update is available and see if I can use it to avoid the stack overflow crashes.

balabuev

I've checked, it aviods stack overflow. But, the rest of a long line become un-parsed, and thus - uncolored (drawn with black default color). However, same regexps are used to process code folding, and potentially, in some other places. And its hard to predict what can happens in that cases; may be nothing bad, but I just can't predict.

Albert Wiersch

Isn't just about anything better than crashing?

Long lines that are unparsed/uncolored are unfortunate, but it's better than the program crashing.

Of course I wish there were a way to parse so that it could do what it needs to do without recursing so much as to cause a stack overflow... but I guess that would take a lot of redesign that you can't justify?

Anyway, I look forward to trying these changes and appreciate the workaround.