ORA-00609: could not attach to incoming connection in database alert log

-

 Environment Details

----------------------------

Oracle Real Application Cluster 02 Nodes

Oracle Solaris 11.3 SPARC

Oracle version 19.6.1.0.0


Problem Description

---------------------------

Above error continuously reported in the database alert. In that case we have to check certain logs instead to database alert.

Before that, we must find for any firewall rule changes or any IP filter changes in between application server IP and database IPs with the internal network team.

So the error reported in the db alert as follows.

2021-11-23T15:20:09.766518+05:30

opiodr aborting process unknown ospid (9758) as a result of ORA-609

2021-11-23T15:20:09.770202+05:30

Errors in file /u01/app/oracle/diag/rdbms/db/db1/trace/db1_ora_9758.trc:

ORA-00609: could not attach to incoming connection

ORA-12547: TNS:lost contact


Considering the alert log error we must check listener logs as well. This can be done finding and comparing the similar timestamp in the listener logs.


23-NOV-2021 15:20:09 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=*******))(SERVICE_NAME=pdbsrv)(SERVER=dedicated)(INSTANCE_NAME=db1)) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.5.46)(PORT=23809)) * establish * pdbsrv * 0

Here listener log confirms that the connection has been established between the client and server process.


So to find out further sqlnet traces must be enable in RDBMS home  [ Warning : when sqlnet traces on, monitor the space because this will be generated many trace logs with in few minutes.]


Error in sqlnet traces as below.


Fatal NI connect error 12547, connecting to:

 (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.5.46)(PORT=23809))

 

  VERSION INFORMATION:

TNS for Solaris: Version 19.0.0.0.0 - Production

Oracle Bequeath NT Protocol Adapter for Solaris: Version 19.0.0.0.0 - Production

TCP/IP NT Protocol Adapter for Solaris: Version 19.0.0.0.0 - Production

  Version 19.12.0.0.0

  Time: 23-NOV-2021 15:20:09

  Tracing to file: /u01/app/oracle/product/19.0.0.0/db_home/network/log/svr_9758.trc

  Tns error struct:

    ns main err code: 12547

    TNS-12547: TNS:lost contact

    ns secondary err code: 12560

    nt main err code: 517

    TNS-00517: Lost contact

 

To match the listener log with sqlnet trace compare the unique port no. In this scenario it is 23809.




Comments

Post a Comment

Popular posts from this blog

ASM DISK MIGRATION [OCR/DATA/RECO DISK GROUPS] in RAC

-
Image
  Current Disk group as below. You must prepare candidate disks with proper permissions by storage level. Here we are going to add below new disks for each disk group with rebalance power 6 and we can do afd label below disks and assign to ASM disk group using asmca or sqlplus commands. After added new disks to relevant disk groups you may drop the old disks in DATA and RECO disk groups with rebalance power using asmca or sqlplus. But note here,  we do not drop old disk of OCR disk group due to existing voting disks. Therefore, you must need to transfer voting disks in to another ASM disk group which has sufficient space. Before moving voting disk  check OCR devices and existing voting disk location. Note you need to have OCR mirroring device before move voting disk to another ASM disk group and get a manual backup of OCR. To check OCR device = > ocrcheck To check voting disks => crsctl query css votedisk To check OCR backup=> ocrconfig -showbackup <AUTO|MAN...
Read more
-
How to remove old oracle home ?     -  Get tar file of old oracle home.               /<$BACKUP_LOCATION>/db_home.tar.gz $ORACLE_HOME -         Go to $ORACLE_HOME/oui/bin location   -         Check oratab file before remove file.   -         Execute below in silent mode to detach home.   -bash-4.4$ ./runInstaller -silent -waitforcompletion -detachHome ORACLE_HOME='/u01/app/oracle/product/12.2.0/dbhome_1' -local   -         After executed above check the oratab file.   Output of above command: Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB.    Actual 38817 MB     Passed The inventory pointer is located at /var/opt/oracle/oraInst.loc 'DetachHome' was ...
Read more