Optional approach to hold-queue cursor processing

Hello out there,

I would like to know if anyone agrees with a proposal I’ve had in my mind for a long time. Here goes…

Background:
Most applications process a number of records in an iteration (by READ LOGICAL or FIND or whatever) and quite often in nested iterations too. Pretty much universally, each of these iterations have a notion of “current record”. Once the current record is processed the iteration moves onto the next (now current) record. There is no mystery why applications behave like this; the fact is doing it this way is simple, understandable, sensible. I’m describing nothing new here. In Natural programming the concept of current record (or current cursor) is represented quite nicely by *ISN.

Taking this further, in my decades of developing experience I have seen very few applications that have a notion of multiple current records (in the same iteration). Again, this is for good reason - processing things one at a time is easy to understand; and maintain so that’s how systems evolve. Keep it simple is a common phrase in our industry.

Side-effect:
A side-effect arises in hold-queue processing because an application may iterate through many records but only modify relatively few. This side-effect is “managed” by allowing finite hold-queue sizes; and all that. In addition, many applications are written to re-GET records that are to be modified to avoid this becoming a problem - but care still has to be taken to make sure decisions are taken on the content of the GET-hold record not the non-held access that precedes it. Many systems deal with this side-effect successfully in these sorts of ways - but I believe it is unnecessary complication. And we know that when something goes awry things can get very messy very quickly.

Proposal:
I propose an option to allow hold processing to be interpreted by Adabas in the simple, one at a time way (current record) that I first described. When Adabas is asked to progress to the next record in series (FIND. READ) then Adabas should simply decide if the record just processed was actually updated or not. If it wasn’t updated then Adabas should, when running with the optional setting I propose, release the non-updated record and put the next one in hold.

The benefit:
As far as I can see the benefits would be that hold processing would only ever be for one record at a time (per iteration) PLUS all the records updated since the last ET/BT. Overnight, many problems would be eradicated such as a) blowing hold-queues or b) hanging sessions due to inappropriate holds on records that haven’t actually been modified.

The implementation approach:
It’s not for me to dictate how this option would be implemented at the coding level, clearly. But what I would say is such an option should be introduced without the need for application program changes. That means the option would dictate how Adabas interprets existing command option settings rather than introducing new options. I guess a file attribute might be considered, since this can be modified by the DBA without impacting (as far as programming changes go) the application. A file attribute might seem a bit draconian where multiple application systems use the same file but that’s the sort of level where control should be allowed.

How do you know whether to use this option or not? Risk management.
This is always a difficult one. However, in this case I think the risks are pretty low simply because it seems to me to be really difficult to write applications that do anything other than adopt the “current record” approach. For that reason I’d be inclined to “just do it”.

However, a “find out” tool could be developed I suppose. Another xxxLOOK plug-in to the database to watch application activity for a file or files to predict if anything other than “current record” activity surfaces or not. If it doesn’t over a period of time then the option could be activated for the files that have been looked at.

A few complexities:
The changes in Adabas when running with this proposed option would need to allow an individual hold-queue element to distinguish between acting as a normal element (cursor-related) versus being an explicit one (created for any other reason such as by an explicit HI command). Only cursor-related elements would be automatically released when not updated when Adabas progresses from current to next record in series. This nuance would enable read-ahead processing (multi-fetch etc) to be tolerated in the same way they are today…until a way can be devised to automatically release read-ahead blocks of records from hold automatically too.

I’d be interested to know if you think I’m crazy, or of sound mind…or other! :smiley:

Ciao,

Michael

Hello Mike,
that looks like a sensible feature to have.

I’m not sure if the fairly new SHARED HOLD clause is able to cover part of what you envision - see http://techcommunity.softwareag.com/ecosystem/documentation/natural/nat823mf/sm/read.htm#read-shared-hold-clause.
There, the programmer can decide on how long to keep the records in SHARED hold with the option to upgrade to EXCLUSIVE hold.

http://techcommunity.softwareag.com/ecosystem/documentation/adabas/ada826mfr/adamf/comref/cmdover.htm#sharedhold

Cheers,
Marbod

Hi Marbod,

Part of the new shared-hold feature provides cursor-related automatic release of hold-queue elements along the “current record” paradigm. However, shared-hold is not directly related to update processing (even though you can upgrade shared to exclusive hold).

What I am proposing is for Adabas to interpret existing coding paradigms in a different way (optionally) so there is no need to alter the many, many millions of programs that are in the field already…that is the key part of my proposal.

Ciao,

Michael

Hi Michael;

No, you are not crazy; just somewhere between twenty and thirty years late with your suggestion.

There is a command, RI , in Adabas, which I believe does exactly what you want. I do not have Release notes that go back that far, but I believe RI was added to Adabas quite awhile ago.

RI is not a part of Natural. If I had to guess, it was not added there because of the number of Direct Call programmers who were “abusing” the facility. For example, you mentioned the fairly commonplace logic of reading a record without hold, then, if an update was required, re-reading the record with hold.

An alternative would be to always read with hold, then release if an update was not required. HOWEVER, holding records requires CPU time, as does releasing records. There were quite a number of horrendous performance problems associated with RI (which requires additional Adabas calls).

In addition, I would not want Adabas making the decision for me whether or not to release a record, especially based only on a criteria such as an update being executed. Suppose I want to read five records, then base an update on a sixth record on “current” values from the first five records, which do NOT get updated.

steve

Hi Steve,

Thanks for the feedback.

Yes, I’m aware of the RI command. However, the main point of my proposal is this option avoids having to alter the many millions of application programs out there. I also propose an option not a default behaviour. That means you can use it as needed - and clearly not use it where it wouldn’t be appropriate. But my experience tells me the vast majority of programs out there do limit themselves to the simple model of one record at a time and so this option would be of great benefit.

In addition, once this option became widely available then the habit of re-GET for update could be discarded completely. The re-GET approach only evolved to avoid problems of elongated hold-queues. Introducing this proposed option would eradicate the need for doing that entirely (okay - in the vast majority of cases anyway as all good options enable choice and control).

Ciao,

Michael