Good day,
We are using the attach manager run Natural RPC, and the number of the number of RPC services is fluctuating with the data volumes change.
Monitoring the environment we find that some conversations are broken due to the error “0003 0010: EOC due to DEREGISTER of partner” which is understandable considering the number of RPC services is decreasing at the time when the error occurs.
It is not clear if the parameter WaitTime could be used to complete all current conversations before releasing the RPC service.
Is it possible to complete the current conversations that are using a RPC service before removing it ? What is about the pending conversation ?
Thank you very much
Leonard
Hi Leonard,
the attach manager starts just the defined services if required.
The error 0003 0010 EOC due to DEREGISTER of partner indicates that an RPC Call was processed by the RPC Server but it seems to fail.
I don’t see that this is related to the attach manager.
Please open a support ticket for the issue as we need to analyze the issue. Please provide a RPC Server Trace of the situation.
regards
Peter
Hi Peter,
Thank you for the reply.
I understand that RPC call cannot be processed by the partner, a.k.a. RPC server, because that server is deregisted being stopped by the attach manager.
The Empower portal required to open SR seems still not working, i.e. we are rooted to the corporate website.
Regards
Leonard
Hi Leonard,
the attach manager does not stop services. There must be another issue with the RPC Server.
Please send a mail to softwareag-support-manager@posteo.de to open a support ticket and provide all information about your environment and your contact information.
regards
Peter
Hi Peter,
Ok, the Natural sessions expire in the Adabas user queue, and Attach manager is expected to up the number when required.
But what is very strange, I observed the drastic discrepancy between the number of registered service instances and Natural sessions, i.e. 34 and 11 ?
sag@dwfwprodNEW [ _env ] /var/fw/SAG> etbinfo -bDWNETB175 -d SERVICE -f “%SERVER-CLASS %SERVER-NAME %SERVICE %SERVER-ACT” |grep -i rpc
SAG ETBCIS RPCCIS 1
RPC RPCCIS CALLNAT 1
RPC SRVNTDB1 CALLNAT 34
RPC DAWNTEST CALLNAT 1
sag@dwfwprodNEW [ _env ] /var/fw/SAG> ps -ef|grep parm=dwnrpc |grep -v grep|grep -v dwnrpct |wc -l
11
Hi Leonard,
we should run this through support as we need to analyze your issue. Please open a support issue and provide the config file and logs from the attach manager, the uses versions and the output of etbinfo.
please also explain what you mean with “the Natural session expire in the Adabas user queue”
regards
Peter
Hi Peter,
I was concern the Natural RPC sessions are terminated due to the Adabas connections timeout. Although I do also think these Natural sessions might be terminated during a client request execution and the problem could be in the code.
I tested the default attach manager creating new Natural sessions to serve the RPC clients. I also realised that the customer is using the custom implementation of the current attach manager such as Natural ACI program, and this attach manager is responsible for the non-affinity between the RPC services and it’ registration on the broker as was shown in the previous mail.
I understand that this custom attach manager is the development issue, and is not responsibility of the product support, and trust that replacing this custom attach manager with the default I will address many operational and performance issues in the environment.
Thank you very much for your help, it is much appreciated.
Regards
Leonard
Good day Peter,
The same pattern is repeating with the default attach manager, i.e. the new services are not created quick enough when the load is high, but only after the volume drops.
Although the affinity between the registered services and Natural sessions is restored.
During the high water mark I noticed that the new services are pending to start for very long time before being able to start, and new registers remain very low or nothing.
Restarting the attach manager does not help.
2020-11-20 09:41:14 ActConv=194, PendConv=1, ActServ=4, Lookups=1
2020-11-20 09:41:14 Adjusted PendingStarts: 0 NewRegisters, 47 → 47 PendingStarts
for the configuration:
ServerName=SRVNTDB1
Service=CALLNAT
Min=10
Max=50
Increment=20
As I said earlier the sessions are decimated during the processing, perhaps due to the issue in the Natural code, and I hoped the atatch manager would compensate this while investigationg.
How can I make the attach manager to create new services faster ?!
If the issue does not have a known solution, I am happy to open an incident.
Kindly advise if I the client should mail the query to the email address you provided, or I can do it on their behalf ?
Thank you very much
Leonard
Hi Leonard,
The Attach Manager maintains a PendingStarts count (the number of times he started the bat file and the server did not yet register at the Broker). So he is waiting for the Servers to come up. When the Server registers at the Broker the PendingStarts count is updated (the number is decreased). This counter is needed to avoid too many servers to be started.
If the server start failed the server never registers at the Broker and the PendingStarts count is not updated.
Please check the command to start the server. Perhaps the start of new Servers failed?
Please open a support ticket if this doesn’t help
regards
Peter
Hi Peter,
The attach manager is using the command that I tested, and it is working but with these “PendingStarts” delays. The new servers start does not fail but is delayed …
I increased the niceness for the attach manager and the server startup command and monitoring.
Tahnk you
Leonard
Hi Peter,
This is the last question before openning the support ticket.
Apparently zombie ([exxatm] ) processes are created in hundreds which is preventing creating new active services.
These cannot be deleted unless restarting the attach manager that is their parent. In our environment the new rpc services are no more created when the number of zombies reaches approximately 700.
Increasing the max user processes and open files did not help.
This is EXX 9.10
Regards
Leonard
Hi Leonard,
EntireX 9.10 is quite old and out of support. I recommend to install a supported version.
Please try to reproduce it with a supported version and open a support ticket if you still have issues.
regards
Peter