EFT extension

I think this summary looks very good. You are right that I don't like the comma notation.

The reason I don't is because of a reason YOU first gave, which is to avoid creating conflicts if new-format EFT lines are combined with an old release that doesn't know anything about them. Using commas will not avoid those conflicts.

Here is my counter-proposal, which itself is different from what I first envisioned:

1. Instead of commas, use ; semicolons to separate each operand. People are used to using ; to separate primary commands, so they'd be doing the same thing they've always done.

That will cause all back-level versions to treat all new operands as comments.

2. If you want to have actual comments on EFT lines, you will have to designate them using ;; doubled semicolons.

The advantage of using ;; is that if anyone had commented EFT lines right now, they can go "fix" them ahead of time, to avoid any issues when these EFT extensions are implemented.

3. To protect users from innocent EFT commenting mistakes, you would also institute a "comment failsafe". Here's how that "failsafe" would work:

When you scan an EFT line, you'd look for the first token after the ; semicolon. If it is a keyword, you then check to see if it is on the "approved list" of acceptable EFT operands. This would be a list of new EFT options, like OPENWITH, plus any primary commands that you are going to allow.

There can't more than a few dozen allowable keywords in this context. You could just make a list of these and then look them up. If you wanted, you could make the EFT command validation more flexible, by using a SET symbol to define this list.

Imagine you defined a new system SET symbol of EFT.COMMANDS. That would contain a comma-separated list of commands you wanted to allow on an EFT line. You'd create an initial default list, and then users could revise that list themselves if they wished.
If you ever started up SPFLite and found that the EFT.COMMANDS set name was missing, you would restore it to its default value.

Example:
    SET EFT.COMMANDS = "OPENWITH,RECFM,EOL,LRECL,SOURCE,BOM"


If you don't find one of the "allowable" keywords after the first semicolon (either it's not a keyword, or it is a keyword but just not one of the one you are going to allow), you ignore that EFT line's comments - but not ignore the part before the ; semicolon.

Then, perhaps the first time the EFT list is read, you'd issue a warning popup if any EFT lines have these "suspicious comments". It's only a warning, and WON'T prevent the rest of SPFLite from starting up.

--

Yes, this is a bit more coding work for you than you planned. But, what it accomplishes is it allows you to introduce this new format with the least possible impact when running it against old releases. And, it avoids problems running this new version with users' old EFT lines that might have problematic comments left in that they didn't get around to correcting yet.

Plus, if other users are like me, I suspect that the majority don't put any comments in their EFT. Personally, my own EFT list is quite short, and I don't have a single comment in it. (And I was the one who dreamed up the thing.) In all likelihood, that vast majority of users will have almost no comments, and the few they might have could be easily fixed. I did a test on the current release with an EFT line like:

STUFF = THINGS, JUNK

and, merely having this bogus line caused no problems at all.

Of course, for the release containing these extensions, a line beginning with a ; semicolon would not be validated in any way.

If we fully document the behavior of these new features, and do a decent job of making that "comment failsafe", I believe everything will be OK.

P.S. I have to honest. I think allowing inline comments on EFT lines is an unnecessary complication. If I had it to do over, I would only allow whole-line comments, and the in-line stuff would be removed.

===> One more thing.

Yes, allowing a 'failsafe' will complicate matters and add more code. It would be much simpler to not bother, and make the format the way you suggest. You can do that, at a cost. The cost would be, any user that took advantage of these new features would have to commit to not using back-versions of the code.

You don't usually "burn bridges" like that. But, it IS an option. Are users who take advantage of the latest features ALSO going to want to use back versions? I don't know. I tend to just use production until a new release comes out, then I use the new production. But that's me. I am used to the "old days", when "production" was all we had. The way SPFLite development is being done now has placed a MUCH higher emphasis on beta versions. So, maybe the back-release strategy needs to be given more attention.

George,

I think there is merit in both approaches, BUT...

Are you also keeping the various "old" OPTION entries in the CFG just in case a user goes back to an older release?

Robert: I don't plan on making any changes to any of the Profile modifying commands. So LRECL, EOF and RECFM are still 'pass-thru' commands sent to the DCB command to handle. So, if a user used 3 old style RECFM, LRECL and EOL commands, and they didn't accomplish what was wanted - tough. The same result would have occurred if they had entered the commands on the command line.

And I'm waffling on the , ; thing. I simply hate a double character comment, it's ugly and not needed. We have ; now, there's no reason to switch to another format.

George

That sounds like a workable solution. I had an idea overnight that might be of assistance.

Right now, you added the DCB command to allow RECFM, LRECL and EOL to be specified together, so that you could validate them together to ensure there are no conflicts. The DCB idea solved an otherwise hard to solve problem, and is a good idea. However, I believe for EFT lines, it is desirable to make them as simple as possible.

Here is my idea:

If I understand correctly where you are going with this, it is your intent to "grab" the extra operands on the EFT, store them somewhere, and then send them to the command processor at some point after the base profile is read, the file is opened, and all the other preparatory steps to get ready to edit. (Correct me if I'm wrong.)

What I think would "help" is if there were a "preprocessing step" that would be performed between the time you "grab" the EFT extra operands and the time you store them.

This preprocessing step would be used to consolidate and "normalize" all DCB-related commands.

Here's what I mean. The preprocessor would be single, relatively simple function that takes a string, scans and possibly modifies it, and returns a slightly different string.

The function would do this:

1. Scan for all instances of DCB, RECFM, LRECL and EOL (and anything that might need to be 'set aside').

2. The keywords and operands found by (1) would be 'extracted' from the input string, and saved off in a string variable in the function.

3. The extracted keywords and operands, NO MATTER HOW SPECIFIED, would be used to build a single DCB command.

4. Finally, a new EFT operand list would be created from the results obtained above, and returned.

5. You would take the results of this function, and use THAT to store for later handling by the command processor, rather than the 'raw' string you first capture off the EFT line.


By doing this, the user is free to specify the RECFM, LRECL, and EOF in any order they want, with or without semicolons, with or without an explicit DCB, and it will still work correctly and provide the DCB protects that have proven so useful.

If you like this idea, but don't want to code it, I can write it for you, if you wish.

The rationale was to avoid having to specify arcane, hard-to-remember rules that applied to extended EFT and no where else. But, if you aren't concerned about that, then I won't be either.

I should point out that most of these various EFT extensions I myself probably would not use, or would use infrequently. The things I have suggested were not selfish requests, but were devised to genuinely help the users. In particular, the idea of modifying things like TXT profile to change the EOL to LF was intended to help you, in the use case you gave of reading text files on a Linux volume.

Sorry for the confusion, but I fear you have misunderstood me.

The "hard to remember rules" were punctuation rules, which have now changed several times since the idea was first proposed. You yourself made a point of saying you wanted to avoid prior-version conflicts, then changed this to use comma notation that would have caused a conflict, then relented and said you'd use semicolons, and now you are using comma again. Color ME confused.

My last sentence was, "In particular, the idea of modifying things like TXT profile to change the EOL to LF was intended to help you, in the use case you gave of reading text files on a Linux volume."

The intended emphasis was on the YOU. My point was that what I was proposing was not for my own benefit, but for YOURS, using your Linus use-case as an example. I would never suggest changing EOL was not possible. My whole proposal has been based on the idea that not only changing EOL should possible but that it HAD to be possible. That's the exact opposite of what you suggested above, and I'm puzzled why you would think so.

Just to be clear: The reason to avoid syntax conflicts is that if you were to release the EFT extensions, then some bug manifested itself in a release-version, users might not be able to use a down-version (release or beta) to circumvent a current-release bug while it was in the process of being addressed.

Under the current direction the design is going, no such circumvention would be possible. The only recourse users would have in that scenario is to yank any EFT extensions until the bug were fixed. How likely is all that to occur? Beats me. But, it is a foreseeable user inconvenience. It seemed prudent to avoid creating such user impasses, when we know it's an actual possibility (even if its likelihood is a small one).

It all falls under the category of doing due diligence. If you think I'm being overly cautious, well, you're the developer. All I can do is advise you.

In that case, I believe I have advised you enough. You seem to have the issue in hand.