|
Post by George on Jun 16, 2023 14:20:54 GMT -5
OK, here's what I'm looking at doing. So I started a new thread to 'drop off' ideas that got left behind.
I know this won't make everyone happy (Robert: I know you don't want commas)
Here's a first attempt at an extension to the EFT table to incorporate
the DEFAULT file-type list and the non-text-type and (W) table.
The current format is simply:
EFT-mask = profilename ; comments after the ;
The addition is simply to add the capability to specify additional operands
following the profilename.
i.e.
EFT-mask = profilename[,operand] [,operand] ... ; comments after the ;
1. Handling the former DEFAULT list entries
An entry would be:
EFT-Mask = DEFAULT e.g. CSV = DEFAULT
2. Handling the former non-Text file entries
An entry would be:
EFT-Mask = SKIP e.g. EXE = SKIP
3. Handling the former 'Open with Windows Default Program' entries
An entry would be:
EFT-Mask = SKIP, OPENWITH WINDOWS e.g. DOC = SKIP, OPENWITH WINDOWS
4. Handling an Open with a specific program
An entry would be:
EFT-Mask = SKIP, OPENWITH filename e.g. ABC = SKIP,OPENWITH "C:\Program Files\MyABC.exe"
5. Handling selective overrides to an existing Profile
An entry would be:
EFT-Mask = profname,profile-commands e.g. WXY = DEFAULT,RECFM F,LRECL 80,SOURCE EBCDIC
The Profile overrides are simply the normal Profile modifying commands, separated by commas.
Spaces may be inserted anywhere for readability.
e.g. WXY = DEFAULT, RECFM F, LRECL 80, SOURCE EBCDIC
Comments are still entered following the entry.
e.g. WXY = DEFAULT, RECFM F, LRECL 80, SOURCE EBCDIC ; Use DEFAULT to handle for an EBCDIC file
As always, comment, shoot down, whatever.
George
|
|
|
Post by Robert on Jun 16, 2023 20:23:37 GMT -5
I think this summary looks very good. You are right that I don't like the comma notation.
The reason I don't is because of a reason YOU first gave, which is to avoid creating conflicts if new-format EFT lines are combined with an old release that doesn't know anything about them. Using commas will not avoid those conflicts.
Here is my counter-proposal, which itself is different from what I first envisioned:
1. Instead of commas, use ; semicolons to separate each operand. People are used to using ; to separate primary commands, so they'd be doing the same thing they've always done.
That will cause all back-level versions to treat all new operands as comments.
2. If you want to have actual comments on EFT lines, you will have to designate them using ;; doubled semicolons.
The advantage of using ;; is that if anyone had commented EFT lines right now, they can go "fix" them ahead of time, to avoid any issues when these EFT extensions are implemented.
3. To protect users from innocent EFT commenting mistakes, you would also institute a "comment failsafe". Here's how that "failsafe" would work:
When you scan an EFT line, you'd look for the first token after the ; semicolon. If it is a keyword, you then check to see if it is on the "approved list" of acceptable EFT operands. This would be a list of new EFT options, like OPENWITH, plus any primary commands that you are going to allow.
There can't more than a few dozen allowable keywords in this context. You could just make a list of these and then look them up. If you wanted, you could make the EFT command validation more flexible, by using a SET symbol to define this list.
Imagine you defined a new system SET symbol of EFT.COMMANDS. That would contain a comma-separated list of commands you wanted to allow on an EFT line. You'd create an initial default list, and then users could revise that list themselves if they wished. If you ever started up SPFLite and found that the EFT.COMMANDS set name was missing, you would restore it to its default value.
Example: SET EFT.COMMANDS = "OPENWITH,RECFM,EOL,LRECL,SOURCE,BOM"
If you don't find one of the "allowable" keywords after the first semicolon (either it's not a keyword, or it is a keyword but just not one of the one you are going to allow), you ignore that EFT line's comments - but not ignore the part before the ; semicolon.
Then, perhaps the first time the EFT list is read, you'd issue a warning popup if any EFT lines have these "suspicious comments". It's only a warning, and WON'T prevent the rest of SPFLite from starting up.
--
Yes, this is a bit more coding work for you than you planned. But, what it accomplishes is it allows you to introduce this new format with the least possible impact when running it against old releases. And, it avoids problems running this new version with users' old EFT lines that might have problematic comments left in that they didn't get around to correcting yet.
Plus, if other users are like me, I suspect that the majority don't put any comments in their EFT. Personally, my own EFT list is quite short, and I don't have a single comment in it. (And I was the one who dreamed up the thing.) In all likelihood, that vast majority of users will have almost no comments, and the few they might have could be easily fixed. I did a test on the current release with an EFT line like:
STUFF = THINGS, JUNK
and, merely having this bogus line caused no problems at all.
Of course, for the release containing these extensions, a line beginning with a ; semicolon would not be validated in any way.
If we fully document the behavior of these new features, and do a decent job of making that "comment failsafe", I believe everything will be OK.
P.S. I have to honest. I think allowing inline comments on EFT lines is an unnecessary complication. If I had it to do over, I would only allow whole-line comments, and the in-line stuff would be removed.
===> One more thing.
Yes, allowing a 'failsafe' will complicate matters and add more code. It would be much simpler to not bother, and make the format the way you suggest. You can do that, at a cost. The cost would be, any user that took advantage of these new features would have to commit to not using back-versions of the code.
You don't usually "burn bridges" like that. But, it IS an option. Are users who take advantage of the latest features ALSO going to want to use back versions? I don't know. I tend to just use production until a new release comes out, then I use the new production. But that's me. I am used to the "old days", when "production" was all we had. The way SPFLite development is being done now has placed a MUCH higher emphasis on beta versions. So, maybe the back-release strategy needs to be given more attention.
|
|
|
Post by George on Jun 17, 2023 9:29:54 GMT -5
OK, I yield on the ; vs , You're right, back level support is more important.
As for valid commands, I've already done some prep, The command table now has another flag saying whether it's EFT 'eligible'. Far easier than creating yet another table to maintain. Nothing looks at it yet, but it was trivial to put in.
George
|
|
|
Post by Stefan on Jun 17, 2023 9:57:02 GMT -5
George, I think there is merit in both approaches, BUT... Personally, I don't like the need for users who are on a "pre-EFT-change" i.e. current release to manually rework their EFT statements to accommodate the new release. Such users are not currently using the new syntax, so it doesn't hurt them, unless they introduce it immediately (unlikely given it's specialist nature) AND then revert to an old version of SPFLite. I guess migration code to do this for them is reasonably simple (change all single ';' to ";;" unless double ';;' are already present - I consider R's proposal much, MUCH to complex).
But I wonder if this is even worth it just to allow backward compatibility of the newest EFT entries with old versions of SPFLite. Is this likely to happen and affect anyone not capable of realising the impact of the older version with a new-version CFG?
Are you also keeping the various "old" OPTION entries in the CFG just in case a user goes back to an older release?
Hence, I'd prefer using commas infuture, because they do not introduce a SEPARATE statement but simply signal a separate part of the SAME statement. In EFT statement terms, an additon like "DCB EOL LF" is not a separate EFT statement - it is an additional part to a single EFT statement.
Of course Robert is right when he says "the cost would be, any user that took advantage of these new features would have to commit to not using back-versions of the code." Not really correct as such though. Any such user is smart enough to take a copy of the CFG BEFORE updating - common practice for me, just in case. And if you want to be kind, you could do it for them with a simple file copy during the installation process.
No need to lumber everyone for eternity with a cludgy ';;' syntax which isn't used anywhere else by the product.
Update Oh well - too late again by 27 mnutes. /Update
|
|
|
Post by Robert on Jun 17, 2023 12:15:57 GMT -5
George, I hadn't thought of the update you did to the master command table, to add the EFT eligible flag. That is a great idea.
I am really liking this now. It seems like just about everything has been addressed. Simple, attractive syntax, and backward compatibility.
I am not sure I fully understand how you are going to handle multiple instances, but it seems like YOU understand it, and trust you can handle it. I will leave in your hands.
Bottom line: I am believing that this will actually work. Yay.
===> Just one more thing ...
Are you going to include DCB as an EFT-eligible command? For the very reason that you made DCB in the first place (avoiding conflicting Profile settings and being able to verify they are OK), I believe you'd need that same protection for commands on the EFT line. So, you'd need to do this:
L:\**.txt = TXT;DCB LRECL 0 RECFM U EOL LF
and not
L:\**.txt = TXT;LRECL 0; RECFM U; EOL LF
right?
UNLESS .. you are going to incorporate that "hack" you added for the command line, where you can "run together" the DCB stuff. IF SO, that would mean the DCB part would be implied, like this, where the other ; semicolons would then be dropped:
L:\**.txt = TXT;LRECL 0 RECFM U EOL LF
I don't know how you want to handle this part.
|
|
|
Post by George on Jun 17, 2023 17:13:44 GMT -5
Robert: I don't plan on making any changes to any of the Profile modifying commands. So LRECL, EOF and RECFM are still 'pass-thru' commands sent to the DCB command to handle. So, if a user used 3 old style RECFM, LRECL and EOL commands, and they didn't accomplish what was wanted - tough. The same result would have occurred if they had entered the commands on the command line.
And I'm waffling on the , ; thing. I simply hate a double character comment, it's ugly and not needed. We have ; now, there's no reason to switch to another format.
George
|
|
|
Post by George on Jun 18, 2023 9:22:18 GMT -5
There's a simple solution to the ; , problem.
I'm going to roll out a full release before I start any changes. I'll tweak the current EFT parsing, so that it will handle the Profile-name operand ending in a blank, comment, End-of-line or a comma. Then when the real EFT changes come along, there is no backward compatibility problem.
|
|
|
Post by Robert on Jun 18, 2023 9:48:08 GMT -5
That sounds like a workable solution. I had an idea overnight that might be of assistance.
Right now, you added the DCB command to allow RECFM, LRECL and EOL to be specified together, so that you could validate them together to ensure there are no conflicts. The DCB idea solved an otherwise hard to solve problem, and is a good idea. However, I believe for EFT lines, it is desirable to make them as simple as possible.
Here is my idea:
If I understand correctly where you are going with this, it is your intent to "grab" the extra operands on the EFT, store them somewhere, and then send them to the command processor at some point after the base profile is read, the file is opened, and all the other preparatory steps to get ready to edit. (Correct me if I'm wrong.)
What I think would "help" is if there were a "preprocessing step" that would be performed between the time you "grab" the EFT extra operands and the time you store them.
This preprocessing step would be used to consolidate and "normalize" all DCB-related commands.
Here's what I mean. The preprocessor would be single, relatively simple function that takes a string, scans and possibly modifies it, and returns a slightly different string.
The function would do this:
1. Scan for all instances of DCB, RECFM, LRECL and EOL (and anything that might need to be 'set aside').
2. The keywords and operands found by (1) would be 'extracted' from the input string, and saved off in a string variable in the function.
3. The extracted keywords and operands, NO MATTER HOW SPECIFIED, would be used to build a single DCB command.
4. Finally, a new EFT operand list would be created from the results obtained above, and returned.
5. You would take the results of this function, and use THAT to store for later handling by the command processor, rather than the 'raw' string you first capture off the EFT line.
By doing this, the user is free to specify the RECFM, LRECL, and EOF in any order they want, with or without semicolons, with or without an explicit DCB, and it will still work correctly and provide the DCB protects that have proven so useful.
If you like this idea, but don't want to code it, I can write it for you, if you wish.
|
|
|
Post by George on Jun 18, 2023 11:15:15 GMT -5
Robert: Appreciate the offer. But as I said "So, if a user used 3 old style RECFM, LRECL and EOL commands, and they didn't accomplish what was wanted - tough."
If the 3 separate commands are compatible, the result would be correct.
If the values are not compatible, then 1 or more of the 3 commands would surely fail. Putting them together in a single DCB command - they would still fail.
And any command failure will fail the file open. So merging them would have accomplished nothing.
Also, I have a hard time believing that a 'tweak' to a profile would be changing 3 values. To use the same profile, the files would naturally be almost identical in format.
Getting the EFT entry correct is a one time thing, and I'm sure any user would learn quickly to use DCB. It's not something that might happen over and over.
George
|
|
|
Post by Robert on Jun 18, 2023 12:02:25 GMT -5
The rationale was to avoid having to specify arcane, hard-to-remember rules that applied to extended EFT and no where else. But, if you aren't concerned about that, then I won't be either.
I should point out that most of these various EFT extensions I myself probably would not use, or would use infrequently. The things I have suggested were not selfish requests, but were devised to genuinely help the users. In particular, the idea of modifying things like TXT profile to change the EOL to LF was intended to help you, in the use case you gave of reading text files on a Linux volume.
|
|
|
Post by George on Jun 18, 2023 12:24:17 GMT -5
Robert: Not sure what you're trying to get across. What are these arcane, hard to remember rules?
We have a basic
operand-a = operand-1[,operand-2] ... [,operand-n] [ ; comments]
Where, other than the OPENWITH type, the optional operands are all familiar SPFLite Primary commands in normal SPFLite syntax.
Your last sentence seems to be saying that modifying the EOL of a TXT profile is somehow not going to be possible, while the whole purpose of this change is to MAKE that possible. i.e. mask = TXT,EOL LF ; Access Linux files.
Colour me confused.
George
|
|
|
Post by Robert on Jun 18, 2023 13:26:06 GMT -5
Sorry for the confusion, but I fear you have misunderstood me.
The "hard to remember rules" were punctuation rules, which have now changed several times since the idea was first proposed. You yourself made a point of saying you wanted to avoid prior-version conflicts, then changed this to use comma notation that would have caused a conflict, then relented and said you'd use semicolons, and now you are using comma again. Color ME confused.
My last sentence was, "In particular, the idea of modifying things like TXT profile to change the EOL to LF was intended to help you, in the use case you gave of reading text files on a Linux volume."
The intended emphasis was on the YOU. My point was that what I was proposing was not for my own benefit, but for YOURS, using your Linus use-case as an example. I would never suggest changing EOL was not possible. My whole proposal has been based on the idea that not only changing EOL should possible but that it HAD to be possible. That's the exact opposite of what you suggested above, and I'm puzzled why you would think so.
Just to be clear: The reason to avoid syntax conflicts is that if you were to release the EFT extensions, then some bug manifested itself in a release-version, users might not be able to use a down-version (release or beta) to circumvent a current-release bug while it was in the process of being addressed.
Under the current direction the design is going, no such circumvention would be possible. The only recourse users would have in that scenario is to yank any EFT extensions until the bug were fixed. How likely is all that to occur? Beats me. But, it is a foreseeable user inconvenience. It seemed prudent to avoid creating such user impasses, when we know it's an actual possibility (even if its likelihood is a small one).
It all falls under the category of doing due diligence. If you think I'm being overly cautious, well, you're the developer. All I can do is advise you.
|
|
|
Post by George on Jun 18, 2023 14:29:30 GMT -5
Robert: 3 or 4 posts ago I indicated there was a simple solution to the 'Back-level' problem, which would be put into the upcoming release (which BTW I just released a few minutes ago). That way when the next release with all these EFT additions takes place there will be NO problems going back to a prior release.
OK, yes I waffled on ; vs , but that was before coming up with the above solution. I never did like the ;; solution, it looked like and was, a big kludge, something that you hate. I was surprised when you proposed it.
And yes, I did interpret your last sentence incorrectly, I apologize for that.
But finally - there is NO backward compatibility problem - NOW. There is no reason we can't use commas, with semi-colon remaining the comment indicator. I don't know where you think the current design is faulty, but from my perspective it's doing fine.
George
|
|
|
Post by Robert on Jun 18, 2023 18:42:18 GMT -5
In that case, I believe I have advised you enough. You seem to have the issue in hand.
|
|
|
Post by Robert on Jun 19, 2023 9:39:58 GMT -5
As usual, I came up with an idea overnight. Not a suggestion per se, but a concern.
Suppose we have a drive E: that has EBCDIC data.
Then, suppose I want to edit an EBCDIC text file on E: and I define an EFT line like this:
E:\**\*.TXT = TXT, SOURCE EBCDIC
Can you see my concern? If E:\ABC.TXT is opened with profile TXT, and THEN its SOURCE is changed to EBCDIC, it will be too late, won't it? The EBCDIC data will have already been read in AS IF ANSI, and then turned into a big mess.
An awful lot depends on the exact timing and ordering of things.
I some ideas of how to possibly overcome this, but before I say anything, I want to hear what you think of this issue and how you might deal with it.
|
|