|
Post by rprinceton on Nov 2, 2023 7:01:49 GMT -5
Hello, During the course of my day I use SPFLite to edit large files. These files reside on a remote servers and based on network traffic can take awhile to load. SPFLite's window title will report "Not responding" and then display a modal window giving me a choice to terminate or continue.
I want to avoid having to respond to this window. I know that the file load will take some time.
Is there any way to tell SPFLite to not worry about a file load taking too long. Perhaps a setting in the config file? I am familiar with SPF_Loop_Check(option-num). This is for edit macros. I am referring to large file load times not done via a macro i.e., SPFLite simply loading it for edit.
Thank you in advance. Regards. Randall Princeton
|
|
|
Post by Robert on Nov 2, 2023 8:11:58 GMT -5
Randall, based on your description, it seems like what is needed is a new feature in SPFLite to "register" names associated with networks, and then to "mark" those networks that are "slow". Then, there would need to be a flag you could set somewhere to say "ignore slow access time" or something.
Network names that SPFLite knew about would be either certain drive letters or UNC names like \\MYSERVER.
The idea you are presenting is more general than the macro Loop Check, but it seems like a useful concept to add, if George could find the right place to add it to the code. It would also take a fair amount of code to add the user-interface part needed to define drives and network paths, finding a place to store all the configuration data, etc. Since drive latency like this is a hardware attribute rather than "instance" related, perhaps this is data that should be stored in the Registry rather than SPFLite.CFG ?
George, what do you think?
R
|
|
|
Post by George on Nov 2, 2023 10:30:06 GMT -5
This an RTFM moment. Check command line option -NOLOOP.
George
|
|
|
Post by Robert on Nov 2, 2023 12:22:51 GMT -5
George, I did Read The Fabulous Manual (just now), and it says,
If '-NOLOOP' appears as the command line, SPFLite will NOT activate it's normal loop detection logic.
Yes, that seems like it would do the trick. But, it is important to ask, is this the right trick to do?
What I mean is, if I'm not mistaken, your normal 'loop' detection doesn't really detect "loops", does it? It just detects a 'time-out' condition, some flag (or something) doesn't change or get reset (I am guessing here) within a certain timeout value (I think it's 10 seconds). Is that much correct, so far?
But, doesn't -NOLOOP disable all timeout testing? I see that as a possible problem, because there are three main causes of timeouts that I know. These are:
#1. Macro loops. These could be either long running loops, perhaps operating on a large file, or because of a macro bug.
#2. Macro popups asking for a user response. If the user takes longer than 10 seconds, SPFLite thinks that it has "timed out".
#3. Network I/O latency. Right now, Randall is having issue with this. It's no one's "fault", but is just how his network functions.
Now, if we use -NOLOOP to stop the 'not responding' #3 message, you are also going to suppress the #1 and #2 messages.
I think most people wouldn't care of #2 messages were always suppressed. Personally, I'd love it. But if #1 messages were suppressed, and a macro had a bug, it would be hard to manually intervene.
I have some recommendations that could help make it easier to manage these timeout situations. Comments invited.
1. Right now, timeout detection is either at the global level or on the macro level. I believe it could be useful to allow this at the primary command level also. That way, you wouldn't have to alter the properties of a desktop icon and restart to change things. Perhaps a command like this:
TIMEOUT [ ENABLE | DISABLE ]
If neither ENABLE or DISABLE were specified, the command would report on the current setting, as usual.
2. For macro calls that could cause a timeout, I recommend an SPFLite-provided "front-end" that would temporarily save then disable the timeout state, issue the normal function, then restore the timeout state. The only function that fits this - I think - is MSGBOX (and MSGBOXw). You'd make a new function SPF_MSGBOX which would take all the current parameters of the "real" MSGBOX, so all a macro writer would have to do is rename the function call.
3. For the network I/O latency, that would be more work. For Randall's scenario, the delay is caused by I/O to a network drive during read and write of an edit file. To quiet the warnings in this case, you'd have to somehow "front-end" your I/O of a file, so that you temporarily disabled the TIMEOUT test until the read or write operation were completed.
And yes, all this would require some work; how much, only you know. But, if this were possible (and, not too hard), it would be a better, more robust solution, instead of the current "blunt instrument" approach.
R
|
|
|
Post by George on Nov 2, 2023 13:04:48 GMT -5
Robert: The macro 'problem' can be controlled by the macro writer - SPF_Loop_Check(ON/OFF), so that handles 1 and 2. The macro writer can turn it on and off as needed by what is going on (e.g around MSGBOX type stuff).
Leaving #3. I suppose a LOOPCHECK ON/OFF command would do it, I'm just not sure how useful this would be considering how often people have brought up this problem.
And a quick code check indicates this is not trivial either as the loop detect is already turned on and off by numerous existing code locations. There is no single 'flag' to control this. The -NOLOOP actually cripples the creation of the sub-Thread which catches the loops.
Simple solutions are not always so simple.
George
|
|
|
Post by Stefan on Nov 3, 2023 3:06:31 GMT -5
George,
I don't quite understand the 'no single flag' statement in combination with "turned on and off by numerous existing code locations". Do those code locations have their own loop detection? I assume Loop detection raises an event which needs to be handled (to pop up the message) Could you allow loop detection to execute as normal but the 'handling' routine blocks/ignores/auto-resets the event when required?
If that were possible....
How about a solution based on filesize or record count? SPFLite could check the filesize before loading and if it's over a certain amount, could disable the loop detection for the duration of the loading process and then enable loop detection after file load is complete.
I appreciate file size does not equal number of lines/records, but it's a start. If the record count is important, could a STATE ON setting help as the line count would then be known before loading (assuming the file has been loaded/saved before), but I don't know if STATE info would be available at loading time. (I assume Profile data is available, given the need to know 'DCB', SOURCE, COLLATE, etc. info)
I appreciate neither approach is sensitive to whether the source location is local or network-base, but it wouldn't greatly matter.
The inverse of all this might also be useful during SPFLIte start-up, which is very, very s-l-o-w whenever there is an entry in the RECENT Files.FLIST which is on a network-server which in turn is not currently available (long wait for WIndows to time-out looking for the resource). An SPFLite internal timer to get it to 'stop waiting for Windows move on' would be helpful (but I know - probably small user demographic).
|
|
|
Post by Jo on Nov 3, 2023 8:40:42 GMT -5
Randall, your description does not clearly describe what "modal window" pops up. Is that really the SPFLite-loop-protection-window? Or is it a Microsoft Windows Message stating "The application is not responding. ..." ? in this case, you may try to modify some windows timeout values (HungAppTimeout) in the registry or simply wait until the file is loaded.
Jo
|
|
|
Post by George on Nov 3, 2023 10:17:23 GMT -5
Further explanation is needed here.
There are two different loop checks.
First, for macros. Macros run in their own thread, and the timeout WAIT is part of the normal 'WAIT for thread completion'. It honors the macro function SPF_Loop_Check(ON|OFF) setting.
For all others in normal SPFLite execution, the 10 seconds is measured from the last keystroke to the final return to Windows marking completion of the Windows message. This testing is suspended internally by several routines such as during crash handling, during Printing, during reading or writing of any file > ~ 3 MB, during FIND/CHANGE ALL when the line count > 50,000, during some RUN and SUBMIT processing, and during Windows OPEN activity.
I had thought of just using the same suspend checking used by those routines for a new LOOPCHECK command, but then if any of those functions are used after a LOOPCHECK OFF command, we'd end up back with LOOPCHECK ON.
So a different way is needed. I just need to figure out the best way.
George
|
|
|
Post by mueh on Nov 3, 2023 10:26:41 GMT -5
Hi George ! In THREAD FUNCTION DoLoopDetect(BYVAL dummy AS LONG POINTER) AS LONG IF gLoopCtr > 10 + (TP.LastLine / 750000) THEN ' Do we consider this a loop? (10 seconds + 1 per 750,000 lines) you give one additional second for 750000 lines . Maybe it should be for less lines ?
|
|
|
Post by George on Nov 3, 2023 13:35:43 GMT -5
MUEH: That kind of approximation is really tricky to determine. A lot is based on the power of the system it's running on. A big difference between some 5 year old Pentium system with real hard drives and a newer i9 (or i10, i11 - whatever the latest is) with SSD drives. I doubt there is any simple way to determine what limit should be set.
George
|
|
|
Post by Robert on Nov 3, 2023 14:08:34 GMT -5
No, there isn't a "simple" way to do it, but there is a way.
It would require the code to "calibrate" the performance of the host system. Windows has some high-precision timer API's so you can use as the "stopwatch", and then arbitrarily pick two points in the code and see how long it takes to run.
That way, you can calculate a conversion factor to equate CPU time vs. lines of SPFLite code that get executed in that time interval.
No, not simple, but not impossible. Main limiting factor is, I think, how important is this, really? I believe that such calculations are probably the wrong approach. It would be better if you fine-tuned your timing rationale. You need to segregate MSGBOX delays, network delays and Macro loop errors, and not try to solve them all with one all-size-fits-all solution, because it's not working well.
IMHO.
R
|
|
|
Post by George on Nov 3, 2023 15:22:27 GMT -5
Robert: Yes, I'm sure there's a way to come up with some kind of 'performance index' and then use it to determine an improved 'loop detected' interval.
But really, this is an editor, how many other editors are there that go to such levels to even detect loops, let alone loops based on measured performance standards.
Any effort here would truly be 'going down the garden path' to being a purely academic exercise.
George
|
|
|
Post by Stefan on Nov 6, 2023 3:51:34 GMT -5
<Snip> I had thought of just using the same suspend checking used by those routines for a new LOOPCHECK command, but then if any of those functions are used after a LOOPCHECK OFF command, we'd end up back with LOOPCHECK ON. So a different way is needed. I just need to figure out the best way. George
If you have a "status variable/location/whatever" of some kind that allows those other processes to temporarily disable/ignore the LoopCheck timer, could you implement this as a mini-stack? You'd still only need one variable (the stack pointer) and of course, a persistent stack accessible by the LoopCheck routine.
That way, whatever the sequence of the processes that enable/disable the LoopCheck, you PUSH/POP the status (ON/OFF) for each 'process' so each one returns to the previous status quo.
If you also wanted to implement an all-overriding LOOPCHECK OFF/ON command, perhaps the override ON/OFF could be the first entry in the stack. If set to OFF, no PUSH or POP occurs.
|
|
|
Post by George on Nov 6, 2023 10:32:45 GMT -5
Stefan: Main problem is that most of the 'suspend checking' activity is not matched by a corresponding 'resume checking' activity.
Every UI interaction eventually returns to Windows when complete. All the KB initiated stuff and mouse initiated stuff end up returning back through a common exit where 'resume checking' is issued - even if it was never set to 'suspend checking'.
A stack only works well if everyone uses it in pairs of requests.
Maybe I can create those 'pairs', TBD.
George
|
|
|
Post by George on Nov 6, 2023 12:02:49 GMT -5
Well, we now have a LOOPCHECK [ ON | OFF | ? ] command. Default is ON unless the command line operand -NOLOOP is used.
If -NOLOOP is used, a LOOPCHECK ON will override it.
LOOPCHECK is not saved anywhere, each run starts with it ON (unless -NOLOOP).
George
|
|