|
Post by George on Oct 17, 2020 14:22:29 GMT -5
Hi, Based on Stefan's problems / suggestions after having to re-create his CFG file and all it's settings, I've created a tool which should be of some help. I have a working version (which I'll attach below), but I first wanted to release this trial version and get people's suggestions, comments etc. So what is it. I'm calling it CFGMaint, I'm open to any alternative suggestions. It's a separate program from SPFLite itself, in fact it will refuse to run if a copy of SPFLite is executing. Command line: CFGMaint [-EXPORT [-EXPMAX n] ] [ -IMPORT ] [ -REPAIR ]
N.B. Default if no operands is -EXPORT -EXPMAX 10 ExportThis will create a export of all data in the CFG file in a readable TXT file format. The file will be created in the normal SPFLite DATA folder. The format of the name is SPFLiteCFGMaintExport.2020.10.17.12.39.TXT The EXPMAX value specifies the maximum number of copies of the export files to retain. CFGMaint will remove older, extra versions. Specifying EXPMAX 0 will turn off this automatic cleanup. If any reportable errors are encountered during Export, a LOG file will be created in the same folder. It's name will be D:\Documents\SPFLite\SPFLiteCFGMaintLog.2020.10.17.12.39.TXT. The timestamp will match the Export filename. These will be 'cleaned up' in the same manner as the actual Export files. My intent was that CFGMaint could just be run (via Task Scheduler) on whatever frequency you like to do periodic CFG file backups. ImportThis will prompt you to select an Export file to be used for Import. The file will be read and its values loaded into your current CFG file. Note the CFG file must exist, Import will NOT create one. If any reportable errors are encountered during Import, a LOG file will be created in the same folder. RepairThis will invoke a "repair in place" function which will evaluate all entries in the CFG file (as best as I could manage) and if errors are found it will prompt you individually for each error for permission to correct the error. You can proceed, deny the individual error or cancel the remaining part of the Repair run. If any reportable errors are encountered during Repair, a LOG file will be created in the same folder. Warnings
Obviously, while providing some useful features, this tool opens up the possibility for creating problems. - The Export files are simple TXT files, feel free to browse them.
- You can obviously EDIT these files, I will not be providing any details of what the entries are, most of you are more than capable of determining what most of them mean.
- If you do edit the file and use Import to reload it, Import will verify the new values to the best of its ability, but many of the values cannot be validated (like pathnames)
- The Export file is in sections matching the CFG tables. Please DO NOT:
- Rename the Table Names
- Add or remove keyword=value items within a Table group. Import replaces the CFG table with the Export file contents, it does not simply update individual keyword=value items. A Table group must be a complete set.
- Be particularly careful with Keyboard entries.
- Repair will almost certainly report missing keyword values in Profile entries. New options have been added over the various releases and if the Profile has not been used since the introduction they will be reported as missing. If you give permission, the keyword will be added with the appropriate default.
So ... If you try out Import or Repair - Backup your CFG file separately first. This is still an early version George CFGMaint.EXE (201.5 KB)
|
|
|
Post by Stefan on Oct 25, 2020 2:30:08 GMT -5
Quick Questions.... (1) What's the granularity for IMPORT, please? Is IMPORT always an all-or-nothing affair or... Is the user allowed to tailor/edit the .TXT file so that a subsequent IMPORT only 'refreshes/replaces' a subset of values? Examples: - just the KeyMap data by specifying just the [KDEFAULT] section - just the colour scheme by specifying just the [ODEFAULT] heading line followed by only the SCHEME... lines - a subset file profiles - all settings for a given instance (2) If I do not specify values for every setting in a given section, will IMPORT just refresh/replace the values of those settings I did specify and leave the rest ASIS, or will it replace all other settings' values (not mentioned in the TXT file) with default values?
Example: - Update a bunch of file profiles to COLS=ON by specifying each [...] profile header followed by COLSFLAG=1
- Set the same TABS for a bunch of profiles
(3) Will IMPORT add stuff to the existing CFG file?
Example:
- 'bulk-add' a new Instance with all its settings, - Copy a KeyMap from one CFG file to another as a new map? (not sure this is 'legal' without a separate Instance so may be a poor example)
|
|
|
Post by George on Oct 25, 2020 10:06:46 GMT -5
Hi Stefan:
The Import granularity is at the table level. So, yes, you can Import a file with just the keyboard definitions. The tables (as you've probably already browsed) are laid out:
[Tablename] keyword1=value keyword2=value . . ==========================
Each imported table must be complete, you cannot provide just selected keyword=value pairs. If you do, the table will be re-created with just those entries, and on it's first use all missing items are recreated with the normal SPFLite default.
Import will not create new tables, they must exist in the CFG file. Mind you, it's simple to create new entries (Instances, keyboard, Profiles etc.) with whatever defaults are normal. THEN you can Import any modified tables.
I suppose having the ability to add new tables could be put in, but then we get into explaining how different KB tables, Set tables etc. get pointed to, and that's not what I intended this to become.
Want to alter a bunch of Profiles? Just edit the Export file and do a bunch of CHANGE ALL commands.
George
|
|
|
Post by George on Oct 26, 2020 11:39:52 GMT -5
Thinking about Stefan's questions, I think I WILL alter CFGMaint Import to be able to cope with the various 'flavors' of missing pieces, from individual tables right up to a missing CFG file itself. Also to add tables. All the code pieces are laying around in various places, it's more a job of cobbling them together.
George
|
|
|
Post by George on Oct 27, 2020 14:15:15 GMT -5
Here's another release of the CFGMaint tool. Execution parameters are the same, enhancements are: - You can add new table items that do not currently exist (i.e. new Profiles, KB Tables, Instances). If you choose to use this function, you are on your own. CFGMaint will validate individual entries, but will NOT validate matching Instances to their related KB, SET, Retrieve tables. Although this function works, it is not the reason CFGMaint was created.
- CFGMaint will handle a complete absence of the CFG file. It will also handle missing Registry entries. Which means if you have the Export file, you can simply re-create the CFG file from only that.
As always, be careful, this is new stuff. I've been playing with it on my own CFG file, and nothing has broken, but still, back things up. George CFGMaint.EXE (203.5 KB)
|
|
|
Post by George on Oct 29, 2020 11:07:19 GMT -5
Here's a .DOC format file containing the contents of the HELP file for the CFGMaint tool. Comments, suggestions for improvements, clarification needed, are all welcome. George CFGMaint.doc (29 KB)
|
|
|
Post by Stefan on Nov 5, 2020 4:50:01 GMT -5
Hi George, I've taken the liberty of marking up the document with some suggestions as well as a few questions. Some of the questions may require another sentence in the document, some are more related to 'what if' user behaviour and some are suggestions you might like to consider.
Hope this helps.
Stef
CFGMaint.doc (40.5 KB)
|
|
|
Post by George on Nov 5, 2020 10:08:09 GMT -5
Stefan: Great, I'll review it and update as needed.
George
[Update]
Your editorial comments are fine. For the questions:
1. Adding to the system's PATH. The free Installer I use doesn't support doing that. I'll have to see if there's some other free one that does, or whether I can afford the upgraded version.
2. Sure, even though disk space is cheap nowadays, 10 is a bit much for EXPMAX default. 3 it is.
3. Saving EXPMAX is a good idea, I'll add that, saving the last used value.
4. If the Import file is empty, no problem. However it still issues messages after the validation phase and update phase, even though it did nothing. I'll add something to suppress that and issue a "Empty File" message.
5. Yes, there's no reason Import could not do the missing default entries, I'll look at that. I just never considered somebody doing an Import, immediately followed by an Export without an intervening real execution. However, it still would not have hurt, the file could have been Imported back and SPFLite would still have recreated the missing items.
6. The KBD table is fixed and should never be re-arranged, the order is critical.
The "O" and "P" tables (Options and Profiles) are old INI style, order is not important.
The Retrieve table also doesn't matter.
The BKP should also be left alone, there's nothing to be gained by modifying it but trouble.
The SET table should also be left alone, in its simplest form it seems trivial, but if PUSH and POP have been used, it isn't. Again, leave it alone.
[\UPDATE]
|
|
|
Post by George on Nov 6, 2020 16:38:54 GMT -5
|
|
|
Post by George on Nov 13, 2020 14:45:27 GMT -5
Stefan: Any feedback on this latest version? I've uncovered an unrelated nasty little bug to do with the maintain screen position enhancements we did, and I'd like to roll out a correction release. If possible I can incorporate any last minute adjustments to CFGMaint.
George
|
|
|
Post by Stefan on Nov 14, 2020 13:05:43 GMT -5
Hi George,
I trust you're still keeping safe and feeling fine. We're locked in again in the UK.
These comments refer to the versions you posted on 6 Nov 9:38pm.
Documentation
See attached CFGMaint SN.doc file for some more amendments.
These comments don't include whatever you may need to add in as a result of the testing below:
Running the CFGMAINT program
The testing threw up some anomalies. I ran 3 tests... -Export, -Repair, -Import
First I did an -EXPORT run of my current, long-existing CFG which seems to work fine. The log file showed: SPFLite CFGMaint Log CFGMaint - No command operands, -EXPORT -EXPMAX 3 will be assumed Fetching table names Exporting table: [BDEFAULT] BDEFAULT Export complete Exporting table: [KDEFAULT] KDEFAULT Export complete Exporting table: [ODEFAULT] Table: ODEFAULT, Entry: QUICKRENUM=99999 is Obsolete / Invalid, Dropped Table: ODEFAULT, Entry: DEFTYPES=* is Obsolete / Invalid, Dropped Table: ODEFAULT, Entry: DEFTYES= is Obsolete / Invalid, Dropped Table: ODEFAULT, Entry: SCHEME26FG2=&H000000 is Obsolete / Invalid, Dropped ODEFAULT Export complete
Exporting table: [P370ASM] P370ASM Export complete ... last two lines repeated for every profile.
Then I ran a -REPAIR on the same CFG. SPFLite CFGMaint Log Fetching table names Keyboard table for: KDEFAULT has too many entries, The extraneous entries will be removed Table: ODEFAULT, Entry: QUICKRENUM was invalid and deleted Table: ODEFAULT, Entry: DEFTYPES was invalid and deleted Table: ODEFAULT, Entry: DEFTYES was invalid and deleted Table: ODEFAULT, Entry: SCHEME26FG2 was invalid and deleted Table: ODEFAULT, Entry: AttnPos was missing and added to the table. Table: ODEFAULT, Entry: CutNew was missing and added to the table. Table: ODEFAULT, Entry: DiffMinMatch was missing and added to the table. Table: ODEFAULT, Entry: DiffMode was missing and added to the table. Table: P370ASM, Entry: AutoNum was missing and added to the table. Table: P370ASM, Entry: BOM was missing and added to the table. Table: P370ASM, Entry: XFORM was missing and added to the table. ... The last 3 lines were repeated for every profile table, some profiles also had NUMTYPE missing/added ... The missing DIFF entries were a surprise. I have run DIFF commands recently, so they should have been there. That got me thinking! See the BUG below! Repair complete
RESULTS:
ISSUE: As designed, the EXPORT and LOG files from CFGMaint are written to "Location for the SPFLite Data storage folders" shown on the OPTIONS - CONFIG tab. This folder was created to allow sharing of such data between several copies of SPFLite - for example, on separate machines. However each such copy requires a unique SPFLite.CFG file to avoid CFG corruption. I use SPFLite on multiple machines. Each has its own SPFLite.CFG on local storage, but all share the same "Location for the SPFLite Data storage folders" on my NAS. So if CFGMAIN is executed on different machines, the resulting EXPORT and LOG files become intermingled. Some will be deleted by others due to EXPMAX rules. Given that these EXPORT and LOG files relate to a specific SPFLite.CFG file, they should be in the the same folder as that SPFLite.CFG file. Obviously, IMPORT will need to look in the non-shared folder for its input. And folks like me who might use CFGMAINT to copy stuff from one CFG to another will have to physically place the Export file into the relevant machine's folder before executing the utility.
BUG: The missing DIFF entries in the REPAIR Log and the date/time stamps on the files confirm that neither the EXPORT nor the REPAIR processed the 'live' SPFLite.CFG on the local drive. Instead, both functions read, and in the case of REPAIR updated, an old SPFLite.CFG which I had careless left lying about in the in the shared data folder on the NAS. I also checked the registry entry: The HKCU\SOFTWARE\SPFLite\HomeFolder correctly references a local drive and HomeData points at the shared data folder on my NAS. (BTW, the CurrVersion is recorded there as 11.0.19235. Oops, but I digress - focus Stef!)
Removing the left-over CFG from the shared folder shows error message "SPFLite.CFG file does not exist in: S:\Documents\SPFLite\" in the log.
BUG: The documentation states that CFGMAIN will issue an error message if executed while SPFLite is active. I started SPFLite and then ran CFGMAINT. There was no error message and no LOG file was created. (Perhaps because it's looking in the wrong place for the .CFG file?)
ENHANCEMENTS: - If it cannot find the SPFLite.CFG, CFGMaint stops silently.
It would be good if the error message "SPFLite.CFG file does not exist in: S:\Documents\SPFLite\" in the log also appeared on the console. Similarly, a reassuring completion message like those below would be helpful
EXPORT completed
REPAIR completed <n> issues addressed, see LOG for details IMPORT completed, default value assigned to <n> items, see LOG for details
- It would be helpful if the LOG file listed which options/parameters were specified/assumed at invocation.
Especially as there is no easy way of finding out the (saved) EXPMAX value in effect otherwise.
Presently CFGMaint only lists assumed parameters if/when invoked without parms. - Filenames. To my eyes, SPFLiteCFGMaintLog.2020.10.27.13.42.TXT is hard to read and differentiate from others in the same format.
I suggest a format of CFGMaint Log 2020-10-27 13.42.TXT It's shorter and, crucially, the date/time separation is better. Similarly for 'Export'. - Invocation from command line requires user to navigate to C:\Program File(386)\SPFLite2 directory.
Messages are issued in transient MessageBox+<OK> format. This is neither a pure command line program (like XCOPY) nor a GUI program.
I'm comfortable that infrequent use doesn't require a GUI front end, so why not just issue messages and 'Yes/No' requests to the console? They would be persistent then.
Any plans to add C:\Program Files\Spflite2 (or \Program files (386)\ as required) to the Windows Environment "PATH" variable during installation?
-------------------------------
I do wonder what '...nasty little bug...' you found in the cursor positioning changes.
I have had a couple of odd ABENDs and observed sudden disappearances of a tab or two on one of my systems, but not frequent enough to really take SPFLite to task.
Now we have a good way to rebuild customisations in the CFG file, I may just delete the whole SPFLite structure from my systems and reload from scratch.
That should remove all the old test versions from C:\Program Files (386)\ as well as fix up the CFG files for good.
|
|
|
Post by George on Nov 14, 2020 14:31:38 GMT -5
Stefan: Just a quick reply, I'll follow up when I go through your note thoroughly, you provided some good stuff in there.
1. Next release will add SPFLite folder to the PATH, I broke down and paid for the Full version of CreateInstall, it was far simpler to stay with the Installer I know and am familiar with.
2. Switching to HomeFolder from HomeData for the files is fine, and handles your usage case much better.
3. Revised format for filenames is fine by me.
4. The nasty little bug triggered crashes if large # of lines were deleted and current cursor position was close to Top / Bottom of file. Don't remember exactly what I changed, but it should be corrected.
George
And yes, we're being put back into a more restricted Covid environment like you. This is one nasty little bugger.
|
|
|
Post by George on Nov 17, 2020 12:26:08 GMT -5
Stefan: I haven't got to the Doc file (yet)CFGMaint SN.doc (45 KB) I've altered the whole internal method for issuing messages, this should clean a lot up.ISSUE: As designed, the EXPORT and LOG files from CFGMaint are written to "Location for the SPFLite Data storage folders" shown on the OPTIONS - CONFIG tab. This folder was created to allow sharing of such data between several copies of SPFLite - for example, on separate machines. However each such copy requires a unique SPFLite.CFG file to avoid CFG corruption. I use SPFLite on multiple machines. Each has its own SPFLite.CFG on local storage, but all share the same "Location for the SPFLite Data storage folders" on my NAS. So if CFGMAIN is executed on different machines, the resulting EXPORT and LOG files become intermingled. Some will be deleted by others due to EXPMAX rules. Given that these EXPORT and LOG files relate to a specific SPFLite.CFG file, they should be in the the same folder as that SPFLite.CFG file. Obviously, IMPORT will need to look in the non-shared folder for its input. And folks like me who might use CFGMAINT to copy stuff from one CFG to another will have to physically place the Export file into the relevant machine's folder before executing the utility. CFGMaint will now use HomeFolder rather than HomeData for the Export and Log files.BUG: The missing DIFF entries in the REPAIR Log and the date/time stamps on the files confirm that neither the EXPORT nor the REPAIR processed the 'live' SPFLite.CFG on the local drive. Instead, both functions read, and in the case of REPAIR updated, an old SPFLite.CFG which I had careless left lying about in the in the shared data folder on the NAS. I also checked the registry entry: The HKCU\SOFTWARE\SPFLite\HomeFolder correctly references a local drive and HomeData points at the shared data folder on my NAS. (BTW, the CurrVersion is recorded there as 11.0.19235. Oops, but I digress - focus Stef!) Removing the left-over CFG from the shared folder shows error message "SPFLite.CFG file does not exist in: S:\Documents\SPFLite\" in the log. I could find nothing on this, there are some changes in there now due to the Folder/Data change, I'm hoping it has just 'gone away'.BUG: The documentation states that CFGMAIN will issue an error message if executed while SPFLite is active. I started SPFLite and then ran CFGMAINT. There was no error message and no LOG file was created. (Perhaps because it's looking in the wrong place for the .CFG file?) The changes to message handling should correct this.ENHANCEMENTS: - If it cannot find the SPFLite.CFG, CFGMaint stops silently. It would be good if the error message "SPFLite.CFG file does not exist in: S:\Documents\SPFLite\" in the log also appeared on the console. Similarly, a reassuring completion message like those below would be helpful This should also be corrected.
EXPORT completed REPAIR completed <n> issues addressed, see LOG for details IMPORT completed, default value assigned to <n> items, see LOG for details These messages have been enhanced. - It would be helpful if the LOG file listed which options/parameters were specified/assumed at invocation. Especially as there is no easy way of finding out the (saved) EXPMAX value in effect otherwise. Presently CFGMaint only lists assumed parameters if/when invoked without parms. Done- Filenames. To my eyes, SPFLiteCFGMaintLog.2020.10.27.13.42.TXT is hard to read and differentiate from others in the same format. I suggest a format of CFGMaint Log 2020-10-27 13.42.TXT It's shorter and, crucially, the date/time separation is better. Similarly for 'Export'. The new format is CFGMaint Exp 2020-11-17 11.23.TXT and CFGMaint Log 2020-11-17 11.23.TXT - Invocation from command line requires user to navigate to C:\Program File(386)\SPFLite2 directory. Messages are issued in transient MessageBox+<OK> format. This is neither a pure command line program (like XCOPY) nor a GUI program. I'm comfortable that infrequent use doesn't require a GUI front end, so why not just issue messages and 'Yes/No' requests to the console? They would be persistent then. Any plans to add C:\Program Files\Spflite2 (or \Program files (386)\ as required) to the Windows Environment "PATH" variable during installation? The next full release will add the install folder to the system's PATH variable. Some of the weird message handling was because I always intended the -EXPORT function would be run by some scheduling vehicle, and was therefore unattended. I killed that and added a -QUIET operand which will suppress 'most' of the popup MSGBOX messages. This can then be added to the scheduled Export runs.
So here's yet another iteration to play with. CFGMaint.EXE (206 KB)
|
|
|
Post by Stefan on Nov 17, 2020 12:51:36 GMT -5
Hi George, Thanks for this. Will look at it now. By the way.. is there something odd about the above link? It arrives with me as CFMaint.EXE.exe when I click to download it.
[Update] Arrives correctly named for me when I click on it. George [\UPDATE]
|
|
|
Post by Stefan on Nov 17, 2020 15:54:02 GMT -5
Ok George...
Behaviour observed when using the CFGMaint version you posted today (Nov 16th)... BUG - import file selection CFGMaint -IMPORT opens a selection dialog at the 'SPFLite DATA folder' not the live SPFLite.CFG file location.
Update: After running Import and manually re-navigating the selection dialog to the correct place and then re-running Import again, the dialog opens to the correct place. So I think the selection dialog simply opens to where-ever it opened last time, (possibly by another Windows application).
'Last-location' might just be a default if the specified location is invalid (missing quotes around filename with embedded blanks perhaps? My directory is "C:\$user\SPFLite CFG").
You may need to re-examine how you point it at the desired directory.
I also note that the selection dialog's 'Files of type' field shows EXPORT and will therefore include all files in that folder, because all relevant files are of type .TXT. I think you should rename the files so they end in .LOG and .EXP or even .EXPORT It discourages offline editing and users of CFGMaint should be expected to cope with this!
The "Files of Type" field can then be set appropriately so only files created by -EXPORT can be selected.
BUGS - command line operands
(1) There is an error in the handling of the command line operands. I'm pretty sure it relates to trailing blanks.
I have experienced cases where the command syntax looks fine, but the pop-up message reads "Cancelled, Command line operand: is not recognized"
(2) If CFGMAINT is invoked with invalid operands (e.g. CFGMAINT RUBBISH), the Log file shows CFGMaint - Log CFGMaint - Temporary Log location set to: C:\Users\Stef\Desktop\ CFGMaint - Log file location is reset to: C:\$user\SPFLite Cfg\ CFGMaint - Cancelled, Command line operand: RUBBISH is not recognized Switching between log destinations looks odd to a user - it does to this user, anyway.
(3) The new argument -QUIET behaves inconsistently when dealing with an invalid command parameter ie. CFGMaint -EXPOT -QUIET will issue a pop-up error msg, but CFGMaint -QUIET -EXPOT will not issue a pop-up error msg.
(4) It appears that the -EXPMAX operand is allowed with -IMPORT and -REPAIR also. Was that intended?
Should there be an upper limit to the -EXPMAX number?
BUG CFGMaint -IMPORT
(1) opens a pop-up window when complete, stating that "... n entries were corrected", but there is no LOG file to tell the user WHAT was corrected/changed.
(2) There are inconsistencies between the export and import processes. Repeatable example:
(a) Export a file - no errors listed. (b) Import the same file and the first message reads "Import file validation complete" and is followed by "CFG entries re-loaded successfully, 1 items were corrected". (c) Repeat steps a & b and the same result persists. So what's being corrupted/changed and when?
BUG
Following the IMPORT scenario above, I tested the -REPAIR option. There should not have been any errors. However,
CFGMAINT - REPAIR found that
Table: ODEFAULT, Entry LASTSCRY Contains an invalid value: -8 Replace with Default value: 5
As with IMPORT, there was no LOG file written to confirm this message / action taken.
I don't know what LASTSCRY is. I suspect it refers to window positioning.
My desktop extends across 2 displays and the SPFLite window could have been on either one of them or even straddle both.
On a subsequent repair attempt I received
Table: ODEFAULT, Entry SCRPOSITION Contains an invalid value: 1712,3207,-13,826 Replace with Default value: ?
I deliberately provoked the 2nd message by sliding the SPFLite window down so that the bottom edge of the window was off the 2nd display before closing SPFLite. It might therefore be advisable NOT to flag such negative values as errors.
I confirm that CFGMAINT now processes the 'live' CFG file for -EXPORT, -IMPORT, and -REPAIR.
Hope this helps.
|
|