Natural to CSV

Eduardo_Sisamon · February 4, 2008, 6:15pm

Hi Everyone.

First of all, sorry about my poor english. I’m from spain, and i’m learning how to speak and write english correctly.

I’ve trying for a long time to register on these forums, but i think the page had a problem, and I couldn’t do it until today.

I’ve got a problem in a program I’m making now.

I’m trying to make a CSV file from a Natural sequential file. My program has “column selections” (using begin of field, length of field, data type, output format and many other modifiers), and include an option to make filters to the input file. My problem is with these filters.

Let’s see how can i explain it. It’s difficult to explain it in spanish, but in inglish it seems to be impossible to me!

I want them to have the same look as JCL (i call the batch process using a JCL program). I’m trying to make a tree with the conditions read from the JCL file. The nodes can be both conditions and connectors (AND, OR …). The problem i found with this idea is to programate a “PARSER” to interpretate the filter read from the JCL, and translate it to a easy-use structure to evaluate the conditions and filter the input file.

Have you ever made anything like this? I know JCL has the SORT instruction with INCLUDE, and surelly it’s much faster than making the filter in Natural, but i want to try.

Thank you for your help.

Best regards.

Eduardo Sisamon

Steve_Robinson · February 5, 2008, 3:06am

Hi Eduardo;

Some questions:

Are you on a mainframe, or a PC?
When you say “Natural sequential file”, do you mean a Natural workfile?
When you say filter, do you just mean a set of conditions which determine what records should go to the CSV file? For example, SURNAME starts with a B or a T?

steve

Eduardo_Sisamon · February 5, 2008, 12:53pm

Hi Steve.

I’m on a mainframe. I’m running my program as a Batch process from a JCL program.
Yes, i mean Natural Workfile, sorry…
It’s something like that. I want to write the conditions using the same structure than the sentence INCLUDE in the SORT statement in JCL. The structure of this statement is:

INCLUDE COND=(a,b,c,d,e,and,…)
where:
a → Initial position
b → length of the substring
c → data type (numeric, string, packed and binary)
d → condition: EQ, LT, LE…
e → value to compare with the substring

I don’t make the filters using the fields I’m importing from the workfile. The filters are applied using several positions of the workfile (not neccesary included in the imported fields)

My program must be able to interpretate nested conditions, using the parentheses and giving them the same priority as JCL do.

I’ve finally made my tree of conditions, and now i want to convert this tree in a stack of conditions, in order to evaluate them in the correct way.

Thanks a lot.
Best regards.

Eduardo Sisamon

Steve_Robinson · February 5, 2008, 5:22pm

This is not a pretty task you have set for yourself.

Probably the easiest way to do this would be to use “ampersand variables”. For example:

FIND &FILE WITH &WITH
DISPLAY &DISPLAY
END-FIND
END

The program above cannot be STOW’ed. Basically, it is a “skeleton” of a program. At compile time, Natural will look for the Global Variables +FILE, +WITH and +DISPLAY, and substitute their values for the “ampersand variables”.

Important note: “ampersand variable” is not a Software AG term; I use it because I feel it is descriptive and therefore useful when teaching Natural. “ampersand variables” are not “variables” in the Natural sense, they have neither format nor length; they are merely text strings to be substituted for.

You would have a “driver program” that would be something like

INPUT +FILE / +WITH / +DISPLAY

followed by a lot of validation of what the user enters followed by RUN ‘REPORT’ where REPORT is the name of the program skeleton above (with the ampersands).

Thus, for your problem, you could code something like

READ WORK FILE
+CONDITION1
+SEPARATOR1
+CONDITION2
+SEPARATOR2
etc
generate CSV record
END-IF
END

You would have a driver program (the counterpart to the program above with an INPUT statement) that would read the conditional string and generate (using a COMPRESS) values such as:

COMPRESS ‘IF SUBSTRING ( #RECORD,’ #INITIAL-POS ‘,’ #LENGTH ‘)’ #condition
#VALUE ’ INTO +CONDITION1

yes, there is a lot more to this. +SEPARATOR1 2, etc would be AND’s or OR’s. You would have to check to see if the value is alpha so that you would actually compress something like ‘"’ #VALUE ‘"’ rather than just #VALUE, etc.

All in all, not very pretty; but a fun exercise.

An alternative to all of the above would be to create some sort of dummy REPEAT loop with conditions in it. This would probably be even messier than using the ampersand variables.

Have fun.

steve

Eduardo_Sisamon · February 5, 2008, 7:04pm

Thank your for your ideas, Steve.

I think i’ve solved my problem using my tree and stack of conditions. Here it is how i’ve solved it.

For example. I have this Filter:

((((1,1,N,EQ,1,AND,2,1,N,EQ,1,AND,3,1,N,EQ,1),OR,
(4,1,N,EQ,1,AND,(5,1,N,EQ,1,OR,6,1,N,EQ,1))),AND,
(7,1,N,EQ,1,OR,8,1,N,EQ,1)),AND,(9,1,N,EQ,1,OR,
(10,1,N,EQ,1,AND,(11,1,N,EQ,1,OR,12,1,N,EQ,1))))

This is exactly as in a JCL INCLUDE statement.

With a counter starting from 1 until 6 (five positions for the condition statement and one more for the connector), I read the input stream, looking for determinated data, depending on the value of the counter. I must always find these data in order, if not, the clausule is not ok. Only in the final condition i don’t find a connector after it.

Using this algorithm, I make a tree of conditions (see attached)

After that, using the correct algorithm, i construct a stack of conditions, putting first the connector, and after that the two conditions. One or both of these two conditions can also be a connector with two conditions… For the attached tree, here it is the stack.

1 → AND
2 → AND
3 → OR
4 → AND
5 → Cond 1
6 → AND
7 → Cond 2
8 → Cond 3
9 → AND
10 → Cond 4
11 → OR
12 → Cond 5
13 → Cond 6
14 → OR
15 → Cond 7
16 → Cond 8
17 → OR
18 → Cond 9
19 → AND
20 → Cond 10
21 → OR
22 → Cond 11
23 → Cond 12

Using this stack, i can solve the whole condition without solving every conditions inside. When i find an AND statement with a FALSE value, I don’t evaluate the conditions inside that condition (not all the conditions. Only the conditions under that one in the tree). The same with an OR and one of the statements with a TRUE.

I hope this can be helpful for some of you one day.

Best regards.

Eduardo.