|
|
Post by Rava on Jun 7, 2024 5:35:59 GMT -5
Hi George, I'm trying to open a file of about 500Mb (466.000 rows lrec=1300) transferred to my PC in EIBCDIC mode, this is the profile information: Collate handling = ANSI Automatic file save at END = OFF PROMPT record length = 1300, File Record Format = F, End of Line delimiter type = NONE The program opens the file but does not display the data, i.e. it displays the plot with only spaces in ASCII (x'20'). SpfLite2_Bug1.docx (93.99 KB) My PC is equipped with Windows 11 PRO, 16Mb installed RAM, 10Mb used. I attach document with further details. Thanks for any help |
|
|
|
Post by Robert on Jun 7, 2024 9:28:20 GMT -5
Rava,
If you downloaded your file in EBCDIC mode, you should be setting the profile you use to edit the file as EBDCIC.
The usual way to do that is to decide on a file extension you want to associate with EBDCIC. Since you have a file that is RECFM F, LRECL 1300, EOL NONE, it sounds like you downloaded it from an IBM system like z/OS. If your download operation copied a mainframe file as-is, it would still be in EBCDIC. If it is, SPFLite can't edit it directly, unless you inform it that's what kind of file you have, so it knows it needs to do a character translation.
Let's say you call your PC file extension "IBM". (Just for example; you can make it any unused extension you wish.) If necessary, rename your file as needed to add this extension.
Next, on a File Manager screen, type the command PROFILE NEW IBM
Click on the Data Options tab.
In the Encoding and Collate fields, enter EBCDIC for both.
Set the RECFM, LRECL and EOL as needed.
Click on the DONE button.
Now, try opening your file again.
R
|
|
|
|
Post by George on Jun 7, 2024 9:48:24 GMT -5
There could be a large file bug, this is quite huge. My biggest test file is about 175 meg. It loads fine. The number of lines should be fine also, I have test files with 1,000,000 lines.
But yes, get the profile set up correctly for the EBCDIC and DCB parameters. Then lets see.
George
|
|
|
|
Post by Rava on Jun 7, 2024 11:13:03 GMT -5
Encoding parameter changed.
The collateral parameter doesn't let me change it and therefore remains set to ANSI.
by doing this the file opens but some records show x'20' instead of 'x40' and then if you press F3 it crashes reporting the following message:
"SPFLite Has encountered an execution exception (C0000005)"
This is the content of the log file:
SPFLite V(3.0.24069) @ 2024-06-07 18:10
SPFLite has encountered an execution exception (C0000005)
Last Interactions were:
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=Down
K=ENTER
P=End
Module Back Trace:
08 XREBUILD
07 DISPSCREEN
06 LINITTXTDATA
05 PCMDEND
04 CMDASSIGN
03 POSTKEYBOARD
02 MAINDKEYPROCESS
01 MAINCDOKEYSTRING
00 REALPBMAIN
|
|
|
|
Post by George on Jun 7, 2024 11:35:21 GMT -5
rava: I've been reviewing EBCDIC/ANSI stuff, and yes, there does seem to be something odd going on with the SOURCE and COLLATE settings. It doesn't account for your crash however.
Even with SOURCE EBCDIC and COLLATE ANSI, there should be no problem loading and viewing the file since COLLATE only affects viewing in HEX and the SORT command.
I'm continuing to investigate.
George
|
|
|
|
Post by George on Jun 7, 2024 12:28:38 GMT -5
rava: I found and corrected the little quirk with COLLATE, but the crash is still a problem. And the COLLATE problem is independent of the crash.
Is there any way you can create a smaller version of the file. Then send it and your SPFLite.CFG file to me. Actually if you can share the whole file via DropBox or GDrive I can fetch it (as long as there are no security concerns with sharing it).
George
|
|
|
|
Post by Rava on Jun 13, 2024 2:07:04 GMT -5
Hi George, thanks for the support
unfortunately we are talking about a data file with sensitive information and I can't share it, I'll see if possible mask it and send it to you.
|
|
|
|
Post by Robert on Jun 13, 2024 9:52:10 GMT -5
George, this seems to be just a capacity problem. If you haven't already done so, you should be able to create a 466,000 x 1300 lrecl file with no CR/LF and use that. If you didn't get a crash then it could be data related (which is unlikely unless the data has data < X'40' EBCDIC.
When you say SOURCE EBCDIC as a line command, it force the COLLATE to match. But in the Profile Edit panel, that does not happen. I am wondering if the default value for COLLATE should be NONE when first created. That way, if you do a PROFILE NEW name, and never set the COLLATE, you would see the NONE initial value and force it to whatever the SOURCE was.
Or, maybe just leave COLLATE to NONE. Then, when you open a file and get its profile, then see COLLATE NONE, you would force it to match the SOURCE at edit time.
R
|
|
|
|
Post by George on Jun 13, 2024 10:03:35 GMT -5
Robert: I have already tweaked those. SOURCE and COLLATE are now independent. Yes, they are normally the same, but we created them as separate items for some long ago forgotten reason. I can't remember why anymore. Regardless, SOURCE no longer forces it's choice into COLLATE.
George
Oh, and I have tried with a huge file. It's just what we already knew, a plain old out of memory error. Nothing to be done about that.
G.
|
|
|
|
Post by Robert on Jun 13, 2024 11:45:37 GMT -5
I believe the reason was that if you wanted a SORT to behave like ANSI order but if SOURCE forced COLLATE to match, you could never do it. I am sure this is my fault, since I was the one to suggest COLLATE to begin with. However, thinking about this current problem, I believe the correct thing to do when making a new Profile is to set the COLLATE to NONE, which would then be taken to mean "treat it the same as SOURCE is defined". The user could always set it to ANSI or whatever else if they wanted to, and most users would then get what they likely wanted in the first place.
R
|
|
|
|
Post by George on Jun 14, 2024 8:33:07 GMT -5
Robert: Other than separating them, I'm doing no more. Reading that IF/ELSE stuff handling SOURCE/COLLATE manipulation in Profile handling, SORT, DISPLAY routines, HEX handling etc. is weird already. Stuffing in support for NONE is too error prone for me to do properly anymore without stumbling.
George
|
|
|
|
Post by Robert on Jun 14, 2024 15:04:17 GMT -5
OK.
BTW, a fix that "shouldn't" be too hard might be to add a drop-down box in the Edit Profile panel so you could see a list of all SOURCE names. I found it surprisingly hard to type EBCDIC, even though I had 20 years working on mainframes.
Certainly not a priority, but it could be useful if you were looking for a particular SOURCE name and forgot it, so you didn't have to open up the SPFLite home folder and go look for them.
Of course, the same dropdown would need to appear on both the SOURCE and the COLLATE fields, if you were going to do it.
R
|
|
