Showing posts with label OGG-01668 PROCESS ABENDING. Show all posts
Showing posts with label OGG-01668 PROCESS ABENDING. Show all posts

Friday, November 28, 2014

OGG-00869, ORA-04098: trigger 'SIOD.EMP' is invalid and failed re-validation (status = 4098).


##########################
## Error
##########################

2014-11-21 02:13:33  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, RTab.prm:  OCI Error
ORA-04098: trigger 'SIOD.EMP' is invalid and failed re-validation (status = 4098). INSERT /*+ RESTRICT_ALL_REF_CONS */ INTO "SIOD"."TAB_1" ("ID",

### Full Error

2014-11-21 02:13:27  INFO    OGG-00996  Oracle GoldenGate Delivery for Oracle, RTab.prm:  REPLICAT RTab started.
2014-11-21 02:13:32  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, RTab.prm:  No unique key is defined for table 'TAB_1'. All viable columns will be used to represent the key, but may not guarantee uniqueness.  KEYCOLS may be used to define the key.
2014-11-21 02:13:33  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, RTab.prm:  OCI Error ORA-04098: trigger 'SIOD.EMP' is invalid and failed re-validation (status = 4098).
2014-11-21 02:13:33  ERROR   OGG-01296  Oracle GoldenGate Delivery for Oracle, RTab.prm:  Error mapping from SIOD.TAB_1 to SIOD.TAB_1.
2014-11-21 02:13:33  ERROR   OGG-01668  Oracle GoldenGate Delivery for Oracle, RTab.prm:  PROCESS ABENDING.


##########################
#  Error Occurred
##########################

Oracle Golden Gate Replicat process ABENDED with the above error.

##########################
## Command Executed
##########################

start replicat RTab

**************************************** Step By Step Analysis ******************************************************

#########################################
# 1) Check ggserr.log file
#########################################

2014-11-21 02:13:27  INFO    OGG-00996  Oracle GoldenGate Delivery for Oracle, RTab.prm:  REPLICAT RTab started.
2014-11-21 02:13:32  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, RTab.prm:  No unique key is defined for table 'TAB_1'. All viable columns will be used to represent the key, but may not guarantee uniqueness.  KEYCOLS may be used to define the key.
2014-11-21 02:13:33  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, RTab.prm:  OCI Error ORA-04098: trigger 'SIOD.EMP' is invalid and failed re-validation (status = 4098).
2014-11-21 02:13:33  WARNING OGG-01003  Oracle GoldenGate Delivery for Oracle, RTab.prm:  Repositioning to rba 154752882 in seqno 7060.
2014-11-21 02:13:33  ERROR   OGG-01296  Oracle GoldenGate Delivery for Oracle, RTab.prm:  Error mapping from SIOD.TAB_1 to SIOD.TAB_1.
2014-11-21 02:13:33  ERROR   OGG-01668  Oracle GoldenGate Delivery for Oracle, RTab.prm:  PROCESS ABENDING.

=====================================================================================================================

#########################################
# 2) Reason for Failure
#########################################

Above error indicates that a trigger is in INVALID status due to which REPLICAT is failing to start.

=====================================================================================================================

#########################################
# 3) Check Trigger Status
#########################################

SET LINES 200
COL OBJECT_NAME FOR A30
COL OBJECT_TYPE FOR A25
COL Owner FOR A25

Select owner,object_name,object_type,status,created,LAST_DDL_TIME from dba_objects where object_name like 'EMP';

OWNER                     OBJECT_NAME                    OBJECT_TYPE               STATUS  CREATED   LAST_DDL_
------------------------- ------------------------------ ------------------------- ------- --------- ---------
SIOD         EMP           TRIGGER                   INVALID 29-OCT-14 21-NOV-14

set lines 200
col triggering_event for a35
col table_owner for a25

Select owner,trigger_name,triggering_Event,table_owner,table_name,status from dba_triggers where trigger_name='EMP';

OWNER                          TRIGGER_NAME                   TRIGGERING_EVENT                    TABLE_OWNER               TABLE_NAME                     STATUS
------------------------------ ------------------------------ ----------------------------------- ------------------------- ------------------------------ --------
SIOD             EMP           INSERT OR UPDATE                    SIOD       TAB_1     ENABLED

=====================================================================================================================
From above we can see that TRIGGER is in ENABLED at the database level but it is in INVALID status.
=====================================================================================================================

#########################################
# Solution Available
#########################################

a) Fix the errors in trigger compilation.
b) We can use DBOPTIONS SUPPRESSTRIGGER to prevent Trigger from its execution.
c) Disable the trigger if not needed.

=====================================================================================================================

##########################
## Solution ( a )
##########################

a) Fix the errors in trigger compilation.

Alter trigger SIOD.EMP compile;

SQL> Alter trigger SIOD.EMP compile;
Warning: Trigger altered with compilation errors.

SQL> sho err
Errors for TRIGGER SIOD.EMP:

LINE/COL ERROR
-------- -----------------------------------------------------------------
2/2      PLS-00049: bad bind variable 'NEW.START_DATE'

=====================================================================================================================
Compilation of Trigger is not working as there is some definition error in the trigger.
=====================================================================================================================

##########################
## Solution ( b )
##########################

b) We can use DBOPTIONS SUPPRESSTRIGGER to prevent Trigger from its execution.

GGSCI (host01.example.com) 8> view param RTab

replicat RTab
USERID gold@oralin, PASSWORD ****
ASSUMETARGETDEFS
GROUPTRANSOPS 2000
EOFDELAYCSECS 20

-- To suppress the trigger over the target table.
DBOPTIONS SUPPRESSTRIGGERS
DBOPTIONS DEFERREFCONST

#### Modified the Replicat Parameter file and included "DBOPTIONS SUPPRESSTRIGGERS" keyword and restarted the Replicat which again failed with the below error.

#### oggerr.log

2014-11-21 06:45:12  INFO    OGG-00995  Oracle GoldenGate Delivery for Oracle, RTab.prm:  REPLICAT RTAB starting.
2014-11-21 06:45:12  INFO    OGG-03035  Oracle GoldenGate Delivery for Oracle, RTab.prm:  Operating system character set identified as UTF-8. Locale: en_US, LC_ALL:.
2014-11-21 06:45:12  INFO    OGG-03501  Oracle GoldenGate Delivery for Oracle, RTab.prm:  WARNING: NLS_LANG environment variable is invalid or not set. Using operating system character set value of AL32UTF8.
2014-11-21 06:45:12  ERROR   OGG-01746  Oracle GoldenGate Delivery for Oracle, RTab.prm:  Support for parameter SUPPRESSTRIGGERS is not available in the RDBMS version you are using.
2014-11-21 06:45:12  ERROR   OGG-01668  Oracle GoldenGate Delivery for Oracle, RTab.prm:  PROCESS ABENDING.

=====================================================================================================================
Tried placing the suppresstriggers options below userid also which didn't work out.
=====================================================================================================================

##########################
## Solution ( c )
##########################

c) Disable the trigger if not needed.

Alter trigger SIOD.EMP disable;

SQL> Alter trigger SIOD.EMP disable;

Trigger altered.

Select owner,trigger_name,triggering_Event,table_owner,table_name,status from dba_triggers where trigger_name='EMP';

OWNER                          TRIGGER_NAME                   TRIGGERING_EVENT                    TABLE_OWNER               TABLE_NAME                     STATUS
------------------------------ ------------------------------ ----------------------------------- ------------------------- ------------------------------ --------
SIOD             EMP         INSERT OR UPDATE                    SIOD       TAB_1     DISABLED

=====================================================================================================================
As the other options are not working we disabled the trigger as it is not needed and restarted the replicat which came up without any issues...
=====================================================================================================================
 Comments Are Always welcome
=====================================================================================================================



Saturday, January 18, 2014

OGG-01668 PROCESS ABENDING, OGG-01930 Datastore error in 'dirbdb', OGG-01172 Discard file


One of our Oracle Golden Gate Replicat Process Got Abended when we tried to start Replicat Process got OGG-01930 error. We have done lot of search in google for the error and found out the actual error is different and its very simple.

Take a look into below steps for a simple fix.

##########################
## Error
##########################

2014-01-15 22:16:13  ERROR   OGG-01668  PROCESS ABENDING.
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.

### Full Error

2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  ERROR   OGG-01172  Discard file (./dirrpt/DB2.dsc) exceeded max bytes (1000000000).
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  ERROR   OGG-01668  PROCESS ABENDING.
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.

##########################
#  Error Occurred
##########################

Replicat Process got ABENDED in Golden Gate Version 11.2.1.0.1

##########################
## Command Executed
##########################

start replicat DB2

**************************************** Step By Step Analysis ******************************************************

#########################################
# 1) Current Status of the Replicat
#########################################

GGSCI (host01) 9> info all
Program     Status      Group       Lag at Chkpt  Time Since Chkpt
MANAGER     RUNNING
ABE      RUNNING
REPLICAT    ABENDED     GG2         00:00:03      00:35:42
REPLICAT    RUNNING     GG3         00:00:00      00:00:01
REPLICAT    RUNNING     GGSE        00:00:00      00:00:08



=====================================================================================================================

#########################################
# 2) Reason for Failure
#########################################

GGSCI (host01) 10> view report DB2

2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  ERROR   OGG-01172  Discard file (./dirrpt/DB2.dsc) exceeded max bytes (1000000000).
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
2014-01-15 22:16:13  ERROR   OGG-01668  PROCESS ABENDING.
2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.

#########################################
# 3) Reproduce the Error
#########################################

Restarting the Abended replicat, throws below error and again goes back to ABENDED state.

2014-01-15 22:16:13  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.

=====================================================================================================================
If we can see from Step 2, We have received Error OGG-01172 that discard file has reached its max bytes
=====================================================================================================================

#########################################
# 4) Check Replicat Parameter Details
#########################################

GGSCI (host01) 11> view param DB2

DISCARDFILE ./dirrpt/DB2.dsc, APPEND, MEGABYTES 1000
DISCARDROLLOVER on friday

GGSCI (host01) 12>

=====================================================================================================================
 From above we can see that Discard file Limit is set to 1 GB and its also set to Rollover on Every Friday
=====================================================================================================================

#########################################
# 5) Discard File Size
#########################################

cd /u01/app/goldgate/11.2/dirrpt/

-rw-rw-rw- 1 oracle oinstall 1000000103 Jan 15 22:16 DB2.dsc

[oracle@host01 dirrpt]$ du -sh DB2.dsc
955M    DB2.dsc
[oracle@host01 dirrpt]$

=====================================================================================================================
 Maximum Size of Discard File set is 1000 MB and discard file has reached 1000000103 bytes..
=====================================================================================================================

##########################
## Solution
##########################

Discard file has reached its maximum size set. So,

1) We can increase size of the discardfile to a higher value.

2) If above option is not feasible due to disk space shortage, best option is to delete the existing discard file as it contains all discarded records

since start of replicat process and all of them are not needed. You'll need only latest discarded records if you want to perform RCA for these records.

=====================================================================================================================
 Increasing Discard File Limit
=====================================================================================================================

edit param DB2

GGSCI (host01) 13> edit param DB2

2014-01-15 22:25:06  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.



GGSCI (host01) 14> view param DB2

DISCARDFILE ./dirrpt/DB2.dsc, APPEND, MEGABYTES 1500
DISCARDROLLOVER on friday

GGSCI (host01) 16> info DB2
REPLICAT   DB2       Last Started 2014-01-10 13:37   Status ABENDED
Checkpoint Lag       00:00:03 (updated 00:39:33 ago)
Log Read Checkpoint  File ./dirdat/pg000952
                     2014-01-15 21:46:32.000864  RBA 55114491


GGSCI (host01) 17>

GGSCI (host01) 17> start DB2
2014-01-15 22:26:18  WARNING OGG-01930  Datastore error in 'dirbdb': BDB0113 Thread/process 23555/1125325120 failed: BDB1507 Thread died in Berkeley DB library.
Sending START request to MANAGER ...
REPLICAT DB2 starting

GGSCI (host01) 18> info DB2
REPLICAT   DB2       Last Started 2014-01-15 22:26   Status RUNNING
Checkpoint Lag       00:00:03 (updated 00:39:47 ago)
Log Read Checkpoint  File ./dirdat/pg000952


=====================================================================================================================
 Comments Are Always welcome
=====================================================================================================================