George: This was puzzling me, so I did some step-by-step testing to see what happens. (I bet you wish I weren't stuck at home with Covid
)
All of this should be seen in context.
I'm working with an unusual part of the available character set, including a bunch of 'control' characters.
Hardly a 'normal' situation.
That said, any crash is undesirable and one brought about by some content in the data is especially worrying.
The easy and quick answer is "If it hurts, don't do that", but there'll always be the nagging question - WHEN will it strike again?
You will no doubt approach this however you see fit and I'm fine with that, but there are some creepy-crawlies in the less frequented alleyways of the character set.
If you understand why and decide it doesn't warrant a code fix, a suitable Health Warning to users in the documentation may be appropriate.
You can skip to '
Observations' below or, should you wish to reproduce results, I attach a
DODGY.TST file.
dodgy.TST (11.23 KB)
I recommend
1) Use the RASTER font, because it includes visible graphics for many of the unusual and control codes.
2) Ensure the PROFILE for the .TST filetype specifies XTABS 0 to avoid the x'09' (HT) expanding to blanks
3) The DIFF settings I used are:
Limit comp Bounds: 1,MAX
Lines to Exclude: None
Options: All clear, except CASE Insentive
1 Min # matching lines
View: Single column view
The interesting bits are specifically the DELIMS line (13) and the WORD entries that follow it.
When testing,
1) Load any file you like.
2) Load (or RELOAD) DODGY.TST (Remember to change the XTABS to 0)
3) Modify DODGY.TST to take out various characters, but don't SAVE
4) Issue DIFF 2 3.
5) Retry from either 2) or 3)
Here's the trace after starting SPFLite, FM select any-file , FM select DODGY.TST, Type DIFF 2 3, Click <Done>
SPFLite V(2.7.22333) @ 2022-12-01 15:45
SPFLite has encountered an execution exception (C0000005)
Last Interactions were:
K=ENTER
P=diff 2 3
Module Back Trace:
09 PCMDPASTE
08 INITAFILE
07 TABADD
06 LOADTHETEXT
05 PCMDDIFF
04 CMDASSIGN
03 POSTKEYBOARD
02 MAINDKEYPROCESS
01 MAINCDOKEYSTRING
00 REALPBMAIN
Observations:
The DIFF crash is caused by the x'0A' (LineFeed) byte in column 16 on line 27 of DODGY.TST
Change that byte to something else and there is no crash.
That was relatively simple, but it raises more questions.
(1) Why "no-likey" a line feed character? EOL is set to CRLF and SPFlite has no issue with a x'0D' CR byte on its own.
I'll let you figure that out.
(2) If the x'0A' byte in line 27 crashes the system, why did the x'0A' byte in column 48 on line 13 not do so as well?
I reckon SPFLite never 'encountered' that LF byte because line 13 is truncated after column 46.
If I place a x'0A' byte nearer the front of Line 13, the system crashes as expected.
(3) What is causing the truncation to line 13?
No idea, but that also explains why many of the various WORD entries do not produce the expected colorisation effect when they are part of a string.
They are no longer considered 'delimiters'. A string like X'0E0E0E0E' should draw a continuous line in colour 6, but it doesn't, even though a single x'0E' appears correctly.
(4) And what's happening here, then?
As you can see, DODGY.TST started life as an .AUTO file.
Some of the less common characters cause ructions with the normal AUTO-colorisation process as well.
Having replace the x'0A' (LineFeed) byte in column 16 on line 27 with another character, rename DODGY.TST to TXT.AUTO.
Load any XXX.TXT file and you'll be informed via pop-up that...
S:\SPFLite\AUTO\TXT.AUTO
File contains an invalid statement at Line: nn
WORD 6
Auto Colorize has been turned off
[OK]
In short, colorisation dislikes x'0D' and x'1A' in WORD statements. Remove those two statements and colorization occurs normally.
Well almost normally, because it also issues
WNOTE> WARNING: Duplicate 'words' detected in the S:\SPFLite\AUTO\TXT.AUTO file.
WNOTE> >Š Œ Ž
These three chars correspond to x'8A', x'8C' and x'8E' respectively.
It's not clear to me why these are considered to be duplicate WORD entries. Each only occurs once in the file.
)