Real (column-based) HLASM highlighting

Grog
Guest

Real (column-based) HLASM highlighting Jun 6, 2014 4:16:02 GMT -5

Quote

Post by Grog on Jun 6, 2014 4:16:02 GMT -5

Now that I might have to face doing HLASM coding at work in an IDE (as opposed to using a 3270 session) I'd like to have the HLASM syntax highlighting working for real in that scenario as well. (The IDE can invoke an external editor.) This means basing it on the statement syntax rather than matching strings to elements in lists of known language statement types.

While syntax highlighting makes the code more readable, its real power lies (I think) in its ability to detect and highlight syntax mistakes as the code is being written, saving on testing time down the track, and improving code quality.

Now, the syntax of some languages can be quite daunting to emulate in code, but 360/370/390/z assembler - and JCL for that matter - have a simple (old fashioned, I'm sure) column based syntax that can be simple to analyze in code. If implemented, it would remove any need to maintain word lists containing the instruction mnemonics as well as instructions for the assembler itself, not to mention any macros that might get used.

I imagine logic would be something like:
1. If column 1 is blank then there is no label, otherwise the first expression is the label.
2. The first non-blank expression which does not start in column 1 is the instruction.
3. The first expression following the instruction is the operand specification.
4. Any non-blanks after the operand specification are comments.
(For bonus points:)
5. If a non-blank is present in column 72 then the statement is continued on the next line.
6. If this line is a continuation the the first 15 columns must be blanks.

Internally, each line can have end-of-line status flags indicating
1. Is the statement continued?
2. Is the statement ending still in a comment?
3. Is the statement ending still in a quoted string?
The flags would set the initial conditions when commencing the parse of the following record.

Some or all of these flags could also be used in general to add support for block comments in other languages.
If you wanted to use this scheme for JCL, you would have to check for the // in columns 1 and 2, and use column 3 to check for the presence of a label. Since JCL has so few statement types, you could even highlight invalid JCL statement types.

Back to assembler, you also have to suppress starting a quoted string for item attributes such as L, N, K, O, T - er, that's all I remember at the moment. L for length is the most commonly used, of course - the others appear more in macros.

Considering the complexity of the product at the moment, I'd hazard that the above could be achieved in only a couple of multi-hour hack attacks.

The hardest part would be deciding how to request it for a file type extension in the PROFILE/AUTO spec.

Of course, you probably have your own ideas on how to do it which are far superior to the above.

BTW, I'm not keen on paying hundreds of dollars each year to license SlickEdit just for this feature.
:/

Thanks!

George
Administrator

Posts: 4,063

Real (column-based) HLASM highlighting Jun 6, 2014 12:07:02 GMT -5

Quote

Post by George on Jun 6, 2014 12:07:02 GMT -5

Grog:

Having pounded out ASM code for 30+ years, I fully understand the need for any and all assists to help out.

That being said, I'm reluctant to start down the path of language based syntax parsing to accomplish this. That's why the current AUTO design is structured in a somewhat weird manner, it really knows nothing about language syntax and doesn't want to know anything.

I think to support this, I'd go the route of providing an optional linkage to a language extension plug-in, and allow it to do the language based parsing. But even that can get awkward if you want to handle multi-line statements. How would colorize-as-you-type work? What would you pass to the plug-in, the line being typed? What if it's line 5 of a 10 line continued statement (not unusual for macros). To pass the whole statement then requires the mainline to understand parsing in order to 'back-up' to the start of the statement, but understanding parsing is the plug-in's job. Could get really messy.

George

robert
Guest

Real (column-based) HLASM highlighting Jun 6, 2014 16:24:57 GMT -5

Quote

Post by robert on Jun 6, 2014 16:24:57 GMT -5

The general shape and nature of this feature would operate as follows:

1. Instead of HILITE AUTO, there would be a new syntax, called HILITE RULES.

2. For a given file type, such as ASM, if the ASM.INI profile contained an entry for HILITE RULES, there would be associated "rule file", called ASM.HILITE.RULES.

3. In the RULES file, the first line would contain the name of a DLL that processed the highlighting for that file type.

4. In the RULES file, lines after the first line would contain parameters that were used by the rules DLL.

5. To keep the DLL simpler, SPFLite would read the rules parameters into memory and pass them as a parameter to the DLL.

6. It is possible that SPFLite would/could preprocess the various rules; perhaps they would be organized in typical INI file format.

7. As SPFLite needed to highlight a screen of a given file type, it would pass file-specific data (file name, file type, line number range, array of lines of text being displayed) along with a certain amount of context information.

8. The rules DLL is called, and returns an array of colorization lines, with R, G, B, Y and blank codes, with their usual meanings.

In this way, SPFLite provides the framework and architecture for user-defined colorization, while completely washing its hands of any knowledge of syntax. The design here also provides for files of differing types that have the same, or nearly the same syntax, being parsed by the same DLL, with the rules parameters being used to make any file-specific adjustments that were needed.

Grog
Guest

Real (column-based) HLASM highlighting Jun 6, 2014 23:40:41 GMT -5

Quote

Post by Grog on Jun 6, 2014 23:40:41 GMT -5

I agree, George, that you do not want to get into the business of having to understand the syntax of all languages. I think what you have now is a good fit for the vast majority of situations. And it's great that folks can customize the color schemes to suit themselves.

I guess I was hoping for a special dispensation for assembler since its syntax structure is different from just about everything else that folks are coding with today.

I see one aspect I had not grasped is that each word is processed as it is typed by SPFLite, where as ISPF (I expect) would scan a whole line at a time (due to the block mode nature of 3270 I/O).

But even with that process-words-as-you-go approach, the code would still have to be aware of "am I still in a quoted string? and "am I still in comment text?" to get the right highlight. A quick reference to the previous record's final status flag byte (I am suggesting) to set the initial conditions of *this* record would enable highlighting for block comments and wrapped quoted strings which might be possible in many languages.

There is no need to look at anything before the immediately previous record's end-status (in my suggested scenario). Yes, it is possible that the line with the /* which started the comment block 200 lines ago may get deleted, but the correct status will sort itself out as the user scrolls down and the syntax is rescanned. I am sure many users would appreciate such an enhancement.

"Why is my code all the comment color? Ah, forgot the trailing */ to end the comment block..."

Rather than special case code for assembler, Robert's idea is obviously more complete, and provides a framework for a general solution of which assembler is just a specific case, IMO.

Thanks for all your work.

George
Administrator

Posts: 4,063

Real (column-based) HLASM highlighting Jun 7, 2014 10:35:14 GMT -5

Quote

Post by George on Jun 7, 2014 10:35:14 GMT -5

I guess the whole colorizing thing will just have to be put into the ToDo list. Doing anything to handle muilti-line stuff is going to mean some rather large redesign. The current method is a do-it-when-needed process, only invoked as a line of text is displayed. I know some other editors do a colorize pass of the text on loading the file, which means carrying the color attributes around with the text. SPFLite's method retains nothing, the colors are determined right at the time the text is painted on the screen.

Knowing what's going on, as I rapidly PgDn in a colorized source file, I'm always stunned at the speed of todays procesors, it can't really be doing all that code, that quickly. I guess being brought up in the days of barely 1 mhz 1401's, twists your thinking.

George