|
Post by George on Apr 2, 2020 13:53:27 GMT -5
Played with using XFORM and created a macro to "DUMP" a file in a format like the old mainframe dumps. (Idea wasn't mine, came from Stefan) Went fairly well, attached is a screen shot. Comments on layout etc. would be welcome. Right now it just does simple ANSI CRLF files. I was thinking of adding a pop-up to prompt for reading records options Like CRLF, LF, CR, VB, nnn (indicating FB records) and RAW (whole file as one 'record') This macro would become a a sample when this is ever released. George
|
|
|
Post by George on Apr 4, 2020 11:22:43 GMT -5
Robert: I thought I had replied to this, but I don't see it. I've pushed it around as suggested. I put the Hex offsets on the left closer to the Hex data and moved the decimal to the right.
|
|
|
Post by George on Apr 5, 2020 10:33:12 GMT -5
Thoughts on XFORM. As I've been 'playing' with the early support for this, and pondering the WRITE aspect, I think we may need to alter the way we've been thinking of it.
Like while using the File Dump option I've been playing with. To get to it, the Profile must be updated to have XFORM=macname added, where macname is the Dump formatter macro. But, if that's added to a Profile, ALL file opens for that profile get the Dump treatment. If I want a normal OPEN, I have to remove the XFORM=macname from the Profile.
Proposal: Add XOPEN, XBROWSE, XVIEW and XSAVE commands, which are just like the regular commands EXCEPT - if XFORM is specified, they will use the XFORM macro for reading/writing.
So if my .TXT profile has XFORM=XFORMDUMP specified, a normal OPEN will be just as before. If I invoke it via XOPEN, the XFORMDUMP macro will do the file reading.
And having XSAVE avoids all my problems with CREATE/REPLACE/SAVEAS/etc. Thoughts?
|
|
|
Post by George on Apr 6, 2020 10:09:18 GMT -5
1. Let's consider why someone would want an XFORM in the first place. Suppose they are looking at an EXE or a DLL and they want to know what's in it, and *maybe* patch it. If so, they are NEVER going to use SPFLite to edit the file directly. And, no doubt they already have EXE and DLL on the "do not edit" list. So, for cases like these, it is no hardship if they can't do an "end run" around using the XFORM. Why would they want to? And probably if they had an XFORM set up, they probably can't.
Agreed, I doubt anyone would attempt editing such files with SPFLite, even if some kind of XFORM macro was available, but I would bet that someone might like to browse the file, and a XFORM dump might be very handy.
2. Point one brings up another point. An XFORM may make it possible to edit a file (or, at least look at it) that is on the "do not edit" list. If so, you will have to coordinate XFORM vs. "do not edit". Otherwise, you would get locked out of using your XFORM to access an otherwise-inaccessible file.
The 'do not edit' list doesn't stop you, it just warns you and gets permission to proceed. It only automatically skips the file during File Manager FF commands.
3. That leaves files which CAN be normally edited in SPFLite but, for some reason, you want to look at them differently. Example: suppose you had a *very* structured text file consisting of some sort of "tables". Suppose these tables were things like "configuration entries" for some kind of software, where you had to be extremely precise about how the data was laid out. But, even so, it still was text data. Here, you'd have to make a decision whether you wanted to use your XFORM macro all the time, or only on occasion. For the case where you want to use it all the time, just add it to the PROFILE XFORM setting. For the case where you only want it once in a while, you name it as an overriding profile type on edit.
For example, you'd use something like EDIT MYFILE.TXT .XFORMLOAD
where .XFORMLOAD is the XFORM macro you've been playing with.
Using the .profilename override is fine for most normal ANSI Text files, but since the macro is reading the file, if a user had a lot of different files with different RECFM/LRECL/EOL settings, then they'd need multiple macros, one for each combination.
Ran into this with my test Dump macro XFORMDUMP, so had to add all the 'smarts' to the macro to handle the various options. I ignored SOURCE, that was getting just too much. This macro will be a sample, so users could easily plagiarize the read and deblock code.
The problem is, while the macro can handle RECFM=U/F/V etc. it STILL counts on the Profile correctly specifying these settings. Meaning a single .XFORMLOAD profile can't be used for multiple file formats. You'd need XFORMLOADU, XFORMLOADF, XFORMLOADV etc. Also a messy situation. Hence my XOPEN/XBROWSE etc thought.
4. As for all the new commands XOPEN, XBROWSE, XVIEW, XSAVE, it seems like a lot of complexity from the user side of things. I wouldn't recommend it. You already know if you are running in "XFORM mode", either based on the PROFILE XFORM or from an EDIT MYFILE.TXT .XFORMLOAD explicit profile-name override.
Based on that, when you start up an edit (etc.) session you could change the header on top from EDIT to XFORM EDIT, so users are reminded that's what is going on. If for some reason they didn't *want* to be in XFORM mode, they would just CANCEL to get out of it.
I don't really like this either, but we need to come up with something else that works.
5. There may be times when a profile has a defined XFORM setting but you don't want to use it. I think the best way to handle this is to have way to suppress it. Let us assume we created a reserved profile name of .NOXFORM so that if it were specified on a command line or in FM it means any predefined XFORM would not be used. I seriously doubt any user is going to make a "real" profile called .NOXFORM, so it should not be any great imposition.
Trying to pass a .NOXFORM through the Open process without it really being treated as a Profile name might be tricky.
6. As for trying to solve difficult problems like CREATE, REPLACE, SAVEAS etc. you'd have to decide how hard they really are. If they are *too* hard, then just disallow them in XFORM mode, and just say they can SAVE or END with Save in effect, and that's it. If they don't like it, and you can't/won't implement it, well tough, that's the breaks. I suspect that over time, you would eventually figure out how to do CREATE, REPLACE, SAVEAS etc. but doing everything all at once might be a little intimidating and you might want to defer it for a while.
Yeah, that's why I'm leaving the WRITE aside for now, the other stuff above has to get resolved first.
George
That's my take on it so far.
|
|
|
Post by George on Apr 6, 2020 15:38:52 GMT -5
1. I actually have had times where I wanted to patch a DLL or binary file that had embedded passwords, etc. For my keyboard work, the vendor used to embed the email address of the user (me) into the binary DLL's they made, which would have gone to MY users. Only, the "email" was actually my Pay Pal account. I needed a way to patch that file. The only tool at my disposal was a hex editor, but it's so much easier to look at files with SPFLite, I wish I really *could* both browse and update.
I still plan to have the WRITE ability to allow this to be done. But always remember that the support I'm putting in is to provide the ability for the MACRO to do this. If we want to 'Edit' EXE or DLL files, then somebody has to write the macro. I can provide sample skeletons for this, but the onus is still on the user to customize/tweak a sample into a usable macro. I would certainly hope that this would be one area where users would add stuff to the Contributed Files section of the forum.
2. It would be "helpful" if a user were doing an XFORM edit of a file type on the "do not edit" list that you would suppress the warning, since they set up their XFORM *because* they wanted to edit. They shouldn't have to be warned and have to dismiss a warning that doesn't apply to that situation. I know it might be hard to pull that off, but it would improve the user experience if you could.
Suppressing the message should not be a problem.
3. (a) The situation you describe in (3) might be a bit of over-worrying. I see two possible cases here. For the XFORMLOAD macro, you are (I think) trying to make a generic process to "dump" any file type. That's fine. Maybe you need to add more functions to return file type information? Maybe not; I'd have to check to confirm what's there, I don't use that stuff all the time.
All the filetype data is available already via Get_Profile requests. So RECFM/LRECL/EOL etc. are all available.
(b) Some things you said here didn't quite make sense. First, you said that your macro CAN handle RECFM U/F/V but then you say a single .XFORMLOAD profile CAN'T be used for multiple file formats. I am confused. I think you meant that it's *hard* to make a macro flexible enough to handle anything. That much I would agree with.
OK, I have a macro that does handle different RECFM/LRECL/EOL types, and it would probably make a good model macro. But remember, it counts on the Profile to properly reflect the real situation. You were indicating that it might be possible to just add .Profile-name to an OPEN request. I was trying to point out that customization was almost certainly going to be mandatory.
(c) Then, you said, "You'd need XFORMLOADU, XFORMLOADF, XFORMLOADV etc." Again, this is confusing, since you yourself wrote a macro in which you DIDN'T need to do that. But, applying the statement more generally, I think you meant that users might have to tailor their XFORM macros for multiple file formats if they had such. That might be so. However, George, in all likelihood that won't be the case, other than for ANSI vs. UTF-8 types of distinctions. If I had some very structured file that was input data to an application, it's unlikely that it would exist in more than one form. For instance, I need to edit files for my keyboard that are in UTF-16 format, because that's what the program wants. It's the ONLY format it will accept. So, for me, I would never require multiple XFORM macros to handle it, if I wanted to do so. I believe as a general rule, users simply won't need to do what you are describing. And, if they do, then they are certainly capable of cloning a "U" version of a macro into a "V" or or "F" version, if that's what they need. For instance, someone might want to read IBM mainframe files from Hercules. If they had supposedly "identical" data in different formats, they simply have to clone their macros as needed. Again, I doubt that users really *would* have "identical" data in truly different formats, and if they did, (a) it would be rare, and (b) that's their problem, not yours. You shouldn't try to create a general-purpose solution to a problem that will seldom happen.
4. Whether you can or do change the edit session header from EDIT to XFORM EDIT is up to you; I just thought it would be a useful reminder. I do believe a user will mostly KnOW they are doing an XFORM, since it's a sophisticated technique. Users are not generally going to do it by accident.
This is cosmetic stuff, shouldn't be a problem.
5. My thought about an overridding profile name of .NOXFORM was just a way to get this as a "flag" passed through the existing command parsing routines on the command line and FM line command area. Maybe you know a better way? The idea was you'd initially see .NOXFORM, immediately turn it into an internal "flag" and then discard it. That way, you would not actually try to use a real profile name of NOXFILE, since there is no such thing. Now, if you had some other way to pass a flag, like EDIT myfile.txt -NOXFORM that would be great, but EDIT doesn't currently allow syntax flags like that. And if you change the EDIT etc. commands to allow it, it would be a little tricky to make sure -NOXFORM was not treated like a file name. I am sure you could figure it out but it would take a little doing.
Thinking on this, how about the XFORM specification being 'macroname { PROMPT | NOPROMPT } So if someone wanted the XFORM macro all the time, they'd specify NOPROMPT. With PROMPT, a quick pop-up would ask whether a normal OPEN or XFORM OPEN is desired.
George
|
|
|
Post by erosolmi on Apr 7, 2020 4:24:53 GMT -5
Sorry George, I just saw this post.
If you can explain the problem again I can try to help.
thinBasic can manage any kind of bynary data inside strings but not from string in source code I mean you cannot insert a null byte in source code but you can load null bytes from a file or from a string expression
Example
uses "console"
string MyString = "ABC" + chr$(0,0,0) + "GHI"
printl "Len of MyString is: " + len(MyString) for lPos as long = 1 to len(MyString) printl "Char at position", lPos, "is ascii", ASC(MyString, lPos), " char:", mid$(MyString, lPos, 1) Next
WaitKey Output should be
Len of MyString is: 9 Char at position 1 is ascii 65 char: A Char at position 2 is ascii 66 char: B Char at position 3 is ascii 67 char: C Char at position 4 is ascii 0 char: Char at position 5 is ascii 0 char: Char at position 6 is ascii 0 char: Char at position 7 is ascii 71 char: G Char at position 8 is ascii 72 char: H Char at position 9 is ascii 73 char: I Let me know Eros
|
|
|
Post by George on Apr 7, 2020 9:31:03 GMT -5
Eros: At this point I'm not sure anymore, senility I guess. I was testing a new function for our macro language, and what appeared to be happening was that I was not getting a string passed properly when I used thinBasic_ParseString. It appeared that embedded LF CR etc. characters were being stripped out. I tried various things, but nothing seemed to work.
So I switched to using thinBasic_ParsePtrToStringAndLen which worked fine, that's what I continued on with.
Today I went back to confirm things before answering you, and re-tried the original thinBasic_ParseString and of course it worked perfectly.
So ... false alarm, sorry. I must have somehow mucked up my own code following the Parse.
Thanks for peeking in, thinBasic still amazes me, it's just so easy to extend (SPFLite has well over 150 functions defined).
George
|
|
|
Post by George on Apr 7, 2020 9:51:49 GMT -5
Robert: adding parse support to EDIT/BROWSE/VIEW etc. for 35+ new temporary override operands is almost certainly a no-go. Even reducing the allowable overrides to a much smaller number doesn't solve things.
Parsing would be a nightmare, I couldn't use any of the existing logic, and with many commands having optional operands we have the problem of determining if the next 'word' is an optional operand of the command, or the next actual command.
As to overriding the Profile, the Profile is not even loaded till way down in the OPEN path when the physical read is about to take place. If EDIT was able to manage the parse, the results would have to be tucked away until that stage where the Profile is loaded.
Profile data is kept as Properties in an Object, so the problem is that if a Profile Property is altered (by the override), the SET Property routine is where the Lock/Unlock is tested, meaning any change to an unlocked Profile becomes permanent, the exact opposite of what we want.
Yes, all the Set Property routines could be altered to METHODS and a new operand to indicate this is temporary override added, but now we're into some major impact changes.
I think we have to keep looking.
George
|
|
|
Post by George on Apr 7, 2020 12:13:22 GMT -5
Robert: Lots of problems in doing that. All the command processors are written expecting their operands to come from the common parser, the command router itself expects that. The command line keyword hi-lighter can't cope with a command with that many potential operands. And the thought of any command with optionally 30+ sub-operands, each with several operands of their own ...
I know typically only one or two would be entered.
If this is basically fudging up using a temporary profile, why not just CREATE the profile and invoke it with a .profile-name, which is already supported by primary commands and the File Manager? A user is probably only ever going to need a handful of them.
XFORM Profiles that must be unique because of the weird underlying file format, will always have XFORM set, they won't work without it. If a user wants a DUMP of a TXT file, he can create a TXTDUMP Profile with XFORM set, and just use a .TXTDUMP override when a Dump Format is wanted.
I don't think we have to make such significant changes, just think it out properly.
George
|
|
|
Post by erosolmi on Apr 7, 2020 14:33:06 GMT -5
Eros: At this point I'm not sure anymore, senility I guess. I was testing a new function for our macro language, and what appeared to be happening was that I was not getting a string passed properly when I used thinBasic_ParseString. It appeared that embedded LF CR etc. characters were being stripped out. I tried various things, but nothing seemed to work. So I switched to using thinBasic_ParsePtrToStringAndLen which worked fine, that's what I continued on with. Today I went back to confirm things before answering you, and re-tried the original thinBasic_ParseString and of course it worked perfectly. So ... false alarm, sorry. I must have somehow mucked up my own code following the Parse. Thanks for peeking in, thinBasic still amazes me, it's just so easy to extend (SPFLite has well over 150 functions defined). George It is a pleasure to know you are using thinBasic. If you need something just write a mail to: support at thinbasic dot com and I will try to help. Ciao Eros
|
|
|
Post by erosolmi on Apr 8, 2020 2:43:57 GMT -5
Thanks a lot.
We are really happy someone has used thinBasic as embedded script engine. And honored about the trust.
If you need some other native function developed directly in thinBasic Core engine just let me know. We try to keep thinBasic Core engine (thinCore.dll the dll you are using in SPFLite) very small and put more specific functionalities in satellite modules. But if requested functionality is enough general to be of help to different people, we do it.
Ciao Eros
|
|
|
Post by George on Apr 8, 2020 11:00:19 GMT -5
Robert: What I am wondering is where this idea of allowing Profile overrides on the Edit command came from. We were looking at XFORM and it's implications, and suddenly you want to add in the profile option override ability. It is simply extreme overkill. I don't see a single advantage it would buy us.
Our problem is the sometimes we do / sometimes we don't want the XFORM to be used, that's it, that's all we should be addressing.
How about what we do with the IMACRO where you can override it with a %imac-name. (Don't bother looking at Help Edit for this, somehow it never got into the Help file I'll correct that.
We could pick another character ($ ! ) to provide an XFORM override. $xform-name to set one, $NONE to kill an existing one.
I think this does all we need.
George
|
|
|
Post by George on Apr 8, 2020 14:18:29 GMT -5
Robert: OK, think we're getting there. What you describe should work fine. I'm not sure about - as the leading character. Like the %imacro-name, the string can also be used on the command line.
e.g. SPFLite2.exe -BROWSE MyText.txt %MyIMacro
I'd like to avoid it for that reason alone. Just adding a $operand is a pain since it has to be passed around from various locations, but hey! that's the job.
Do you think IMACRO should be modified to use the ON/OFF tecnnique the same way? As you say - consistency.
This is a good idea, thanks, if you can live with $ instead of -, I will go off in that direction.
BTW I was as surprised as anyone that %imacro wasn't documented under EDIT/BROWSE/VIEW - only under the command line options. Big oversight.
George
|
|
|
Post by George on Apr 9, 2020 10:24:02 GMT -5
Robert: I guess / will do.
If XFORM is really needed, and the user still enters /OFF then he'll get whatever SPFLite does right now based on the other file specifications in the Profile. You can't program around stupidity.
George
|
|
|
Post by George on Apr 9, 2020 12:43:20 GMT -5
Robert: No blanks in macro names, not going there, there is simply no need. It's not that hard to make up names. Sheesh, I coded Assembler for 30+ years with a restriction on variable and routine names of 8 characters.
As to /OFF putting a user in an untenable situation. The worst that can happen is the file gets loaded 'raw' like if you EDIT an EXE file. Not untenable, just CANCEL out and start over the right way.
Started having a look at the code for all this - getting messy, as it always does.
George
|
|