Friday, August 30, 2013

ORA-28002: the password will expire within 7 days


##########################
## Error
##########################

ORA-28002: the password will expire within 7 days

### Full Error

oralin_2 @ hostp03:/etc
> sqlplus system/pass_word@oralin_gvl

SQL*Plus: Release 11.2.0.2.0 Production on Fri Aug 30 06:14:48 2013

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

ERROR:
ORA-28002: the password will expire within 7 days

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL>

##########################
## Cause
##########################

Error occured while trying to connect using a user.

##########################
## Command Executed
##########################

conn system/pass

##########################
## Solution
##########################

###

Check User Account Status :

 select username,account_status,expiry_date,profile from dba_users where username='SYSTEM';

USERNAME                       ACCOUNT_STATUS                   EXPIRY_DA PROFILE
------------------------------ -------------------------------- --------- ------------------------------
SYSTEM                         EXPIRED(GRACE)                   06-SEP-13 DEFAULT


Check the Profile limit set for password life time

select * from dba_profiles where RESOURCE_NAME='PASSWORD_LIFE_TIME';

PROFILE                                  RESOURCE_NAME                  RESOURCE LIMIT
---------------------------------------- ------------------------------ -------- -------------------------
DEFAULT                                  PASSWORD_LIFE_TIME             PASSWORD 180

If you dont want to alter the profile limit set, then change the password for the user by noting it values from user$ and then reset it back.

If you wanna change the profile limit then,

Alter profile default limit password_life_time unlimited;

SQL> Alter profile default limit password_life_time unlimited;

Profile altered.

select * from dba_profiles where RESOURCE_NAME='PASSWORD_LIFE_TIME';

PROFILE                                  RESOURCE_NAME                  RESOURCE LIMIT
---------------------------------------- ------------------------------ -------- -------------------------
DEFAULT                                  PASSWORD_LIFE_TIME             PASSWORD UNLIMITED

After changing the profile limit also the user account will be in expired status only,

USERNAME                       ACCOUNT_STATUS                   EXPIRY_DA PROFILE
------------------------------ -------------------------------- --------- ----------------------------------------
SYSTEM                         EXPIRED(GRACE)                   06-SEP-13 DEFAULT

Now change the password you would like or use the same password to set.

Alter user system identified by pass_word;

SQL> Alter user system identified by pass_word;

User altered.

SQL> select username,account_status,expiry_date,profile from dba_users where username='SYSTEM';


USERNAME                       ACCOUNT_STATUS                   EXPIRY_DA PROFILE
------------------------------ -------------------------------- --------- ----------------------------------------
SYSTEM                         OPEN                                       DEFAULT

Thursday, August 29, 2013

ORA-27211: Failed to load Media Management Library


##########################
## Error
##########################

ORA-27211: Failed to load Media Management Library

### Full Error

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 08/29/2013 10:09:21
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 2

##########################
#  Error Occurred
##########################

Error occured while a backup was fired in one of the database of 11.2.0.3.0 version

##########################
## Command Executed
##########################

In RMAN,

backup archivelog logseq 5432 thread 1;

##########################
## Issue Description.
##########################

### 11.2.0.3.0


RMAN> backup archivelog logseq 5432 thread 1;

Starting backup at 29-AUG-13
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 08/29/2013 10:09:21
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 2

This database was failed over from hostp02 to hostp03 due to some issue. Backup was working fine in hostp02 server. backup is not working after failover
to hostp03 server.

While checking found that libobk.so64 library of netbackup is not linked to oracle libraries.

oralin_2 @ hostp03:/u01/app/oracle/product/11.2/db_10/lib
> ls libob*
ls: libob*: No such file or directory

##########################
## Solution
##########################

So Created a symbolic link in $ORACLE_HOME/lib location and tested the backup and it worked fine.

ln -s /usr/openv/netbackup/bin/libobk.so64 libobk.so

oralin_2 @ hostp03:/u01/app/oracle/product/11.2/db_10/lib
> ln -s /usr/openv/netbackup/bin/libobk.so64 libobk.so

oralin_2 @ hostp03:/u01/app/oracle/product/11.2/db_10/lib
> ls -ltr libob*
lrwxrwxrwx 1 oracle oinstall 36 Aug 29 09:32 libobk.so -> /usr/openv/netbackup/bin/libobk.so64
oralin_2 @ hostp03:/u01/app/oracle/product/11.2/db_10/lib

RMAN>  backup archivelog logseq 5432 thread 1;

Starting backup at 29-AUG-13
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=2799 instance=oralin_1 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Veritas NetBackup for Oracle - Release 7.1 (2011020316)

Tuesday, August 27, 2013

Different Methods to find or Check whether the installed Oracle Client Software is 32 bit or 64 bit in Linux / Unix Environments


########################
 Task
########################

To find whether the installed Client Software is 32 bit or 64 bit in Linux / Unix Environments

$ export ORACLE_HOME=/u01/app/oracle/product/11.2.0/client_1

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

=====================================================================================================================
Method 1 :
--------
Using ORACLE_HOME properties File
=====================================================================================================================


$ cat $ORACLE_HOME/inventory/ContentsXML/oraclehomeproperties.xml | grep -i architecture

     

From above output, we can find its 64 bit, 32 means then the client software is 32 bit.

=====================================================================================================================
Method 2 :
--------
Using File OS command
=====================================================================================================================

$ file $ORACLE_HOME/bin/sqlplus

/u01/app/oracle/product/11.2.0/client_1/bin/sqlplus: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped

From above output, we can find x86-64, so this is 64 bit version.

=====================================================================================================================
Method 3 :
--------
From lib directories created during installation.
=====================================================================================================================

$ ls -l $ORACLE_HOME | grep lib

drwxr-xr-x  2 oracle oinstall 4096 Jun 26 14:26 jlib
drwxr-xr-x  3 oracle oinstall 4096 Jun 26 14:26 lib

If the two directories $ORACLE_HOME/lib32 and $ORACLE_HOME/lib exist, then it is 64 bit client.

If you have only $ORACLE_HOME/lib you need to use method 1 as there are client versions (11.2) where $ORACLE_HOME/lib32 directory does not exist on 64-bit client installations.

=====================================================================================================================
Method 4 :
--------
Using Perl Version
=====================================================================================================================

$ perl -v

This is perl, v5.8.8 built for x86_64-linux-thread-multi

Note: Perl is installed according to the bit version of the ORACLE_HOME but not the OS bit version.

=====================================================================================================================

ORA-20600: The specified target is in the process of being deleted.



##########################
## Error
##########################

ORA-20600: The specified target is in the process of being deleted.

### Full Error

Saving instance oralin_1 to repository and agent https://host01:3872/emd/main...java.sql.SQLException: ORA-20600: The specified target is in the process of being deleted.(target name = oralin_1)(target type = oracle_database)(target guid = 31427C7C89C2C89148EB7B775E90E20D) ORA-06512: at "SYSMAN.TARGETS_INSERT_TRIGGER", line 46 ORA-04088: error during execution of trigger 'SYSMAN.TARGETS_INSERT_TRIGGER' ORA-06512: at "SYSMAN.EM_TARGET", line 2283 ORA-06512: at "SYSMAN.MGMT_TARGET", line 2720 ORA-06512: at line 1
... finished.
Saving instance oralin_2 to repository and agent https://host02:3872/emd/main... ... finished.
Properties for instance oralin have been updated.

##########################
#  Error Occurred
##########################

When trying to add “oralin_1” instance in OEM. Above errors has occurred.

##########################
## Command Executed
##########################

Add instance of a cluster in OEM

##########################
## Issue Description.
##########################

Couple of minutes back, “oralin_1” instance was deleted through OEM and the same was tried to add back.

Checking the instance registration details in OEM Repository database,

Set lines 200
col target_name for a30
select target_name, target_type, target_guid from mgmt_targets where target_name like '%oralin%' and target_type='oracle_database';

TARGET_NAME                    TARGET_TYPE                                                      TARGET_GUID
------------------------------ ---------------------------------------------------------------- --------------------------------
oralin_2                       oracle_database                                                  83F1FC27B89BAEFB9F4077620D700303


##########################
## Solution
##########################

Only the 2nd instance details were available. So tried to register the same after 2 minutes and it worked like charm.

TARGET_NAME                    TARGET_TYPE                                                      TARGET_GUID
------------------------------ ---------------------------------------------------------------- --------------------------------
oralin_2                       oracle_database                                                  83F1FC27B89BAEFB9F4077620D700303
oralin_1                       oracle_database                                                  31427C7C89C2C89148EB7B775E90E20D

Temporary Tablespace Group : ORA-01652: unable to extend temp segment by 256 in tablespace AU_TEMP


Our Application team has encountered the below error while running the month end jobs. So below are the recommendation which fixed the issue.

##########################
## Error
##########################

ORA-01652: unable to extend temp segment by 256 in tablespace AU_TEMP

### Full Error

ERROR at line 1:
ORA-12801: error signaled in parallel query server P000, instance host01.example.com:aua01p2 (2)
ORA-01652: unable to extend temp segment by 256 in tablespace AU_TEMP
ORA-06512: at "au_ADHOC.au_EOM_REPORTS_NEW", line 3334
ORA-06512: at "au_ADHOC.au_EOM_REPORTS_NEW", line 3904
ORA-06512: at line 1

##########################
#  Error Occurred
##########################

Error occured while application month end jobs are running in 10.2.0.3.0 version

##########################
## Command Executed
##########################

Application jobs

##########################
## Issue Description.
##########################

### 10.2.0.3.0

We have a temporary tablespace of size 63 GB, due to month end multiple jobs were running at the same time and it errors out with unable to extend error.

##########################
## Solution
##########################

From oracle 10g onwards, we can create a temporary tablespace group such that if one tablespace in that group ran out of space then the other available tablespace will be used.

A new group will be created when a tablespace is assigned to the group.

SQL> Select * from dba_tablespace_groups;

no rows selected

Alter tablespace AU_TEMP tablespace group au_TMP_GROUP;

Alter tablespace USER_TEMP tablespace group au_TMP_GROUP;

SQL> Select * from dba_tablespace_groups;

GROUP_NAME                     TABLESPACE_NAME
------------------------------ -------------------------
au_TMP_GROUP                  USER_TEMP
au_TMP_GROUP                  AU_TEMP

And then change the tablespace of the user who is executing the job.

SET LINES 200
COL USERNAME FOR A25
COL ACCOUNT_STATUS FOR A18
COL DEFAULT_TABLESPACE FOR A20
COL PROFILE FOR A20
col password for a25
col temporary_tablespace for a15

select username,account_status,password,PROFILE,DEFAULT_TABLESPACE,temporary_tablespace,created from dba_users
where username='au_ADHOC' order by username;

USERNAME                  ACCOUNT_STATUS     PASSWORD                  PROFILE              DEFAULT_TABLESPACE   TEMPORARY_TABLE CREATED
------------------------- ------------------ ------------------------- -------------------- -------------------- --------------- ---------
au_ADHOC                 OPEN               DEFAULT              au_ADHOCD1M         AU_TEMP     09-DEC-04

#### Assigning the user the newly created tablespace group.

Alter user au_adhoc temporary tablespace au_tmp_group;

SQL> Alter user au_adhoc temporary tablespace au_tmp_group;

User altered.

SQL> select username,account_status,password,PROFILE,DEFAULT_TABLESPACE,temporary_tablespace,created from dba_users
where username='au_ADHOC' order by username;
  2
USERNAME                  ACCOUNT_STATUS     PASSWORD                  PROFILE              DEFAULT_TABLESPACE   TEMPORARY_TABLE CREATED
------------------------- ------------------ ------------------------- -------------------- -------------------- --------------- ---------
au_ADHOC                 OPEN               DEFAULT              au_ADHOCD1M         au_TMP_GROUP   09-DEC-04

#### To unassign the tablespace from the temp tablespace group.

Alter tablespace USER_TEMP tablespace group '';


SQL> Select * from dba_tablespace_groups;

no rows selected

Friday, August 23, 2013

ORA-00600: internal error code, arguments: [kccscf_1], [9], [72704], [65535], [], [], [], []


##########################
## Error
##########################

ORA-00600: internal error code, arguments: [kccscf_1], [9], [72704], [65535], [], [], [], []

### Full Error

check rman_dup_proddb_uatdb_08062013.log for RMAN Duplication Log.

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 08/06/2013 23:26:29
RMAN-06136: ORACLE error from auxiliary database: ORA-01503: CREATE CONTROLFILE failed
ORA-00600: internal error code, arguments: [kccscf_1], [9], [72704], [65535], [], [], [], []

##########################
#  Error Occurred
##########################

Error occured while RMAN Duplication is performed, once duplication is done and when oracle tried to create controlfile in 10.2.0.3.0 version of database

##########################
## Command Executed
##########################

In sqlplus,

Create Controlfile Statment,

##########################
## Issue Description.
##########################

### 10.2.0.3.0


Error occurred RMAN duplication of create controlfile statement in 10.2.0.3.0 version.

Problem is related to parameter MAXLOGHISTORY specified in create controlfile  command.  It's complaining that maximum should  be 65535. whereas in the create controlfile statement its 72704. due to which its failing.

This seems to be a Bug

Bug:
-------

This is unpublished bug 4877360
Abstract: APPSST SRV 10G :ORA-600: INTERNAL ERROR CODE, ARGUMENTS: [KCCSCF_1], [9], [65732], [65535]

##########################
## Solution
##########################

A possible workaround is to manually recreate the control file, and change the maxloghistory set to 65535 or lower in the script.

The bug is fixed in 10.2.0.4 and 11.1.0.6 versions.



ORA-16033: parameter LOG_ARCHIVE_DEST_1 destination cannot be the same as


##########################
## Error
##########################

ORA-16033: parameter LOG_ARCHIVE_DEST_1 destination cannot be the same as

### Full Error

Alter system set log_archive_dest_1="location=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=stdby";

*
ERROR at line 1:
ORA-02097: parameter cannot be modified because specified value is invalid
ORA-16033: parameter LOG_ARCHIVE_DEST_1 destination cannot be the same as
parameter LOG_ARCHIVE_DEST_4 destination

##########################
#  Error Occurred
##########################

Error occured while trying to modify log_archive_dest_1 parameter to resolve archive gaps issue in our standby database

##########################
## Command Executed
##########################

In Sqlplus,

Alter system set log_archive_dest_1="location=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=stdby";


##########################
## Issue Description.
##########################

###

Error occurred because, there is already a destination  defined with the same physical location. so we need to define a new value.

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_4                   string      LOCATION=USE_DB_RECOVERY_FILE_
                                                 DEST VALID_FOR=(ALL_LOGFILES,A
                                                 LL_ROLES) DB_UNIQUE_NAME=stdby

##########################
## Solution
##########################

Either change the value for LOG_ARCHIVE_DEST_4 to a different path or reset LOG_ARCHIVE_DEST_1 to a different value.

Alter system set log_archive_dest_4="location=/u01/app/oracle/11upgrade/stdby/rman_standby VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=stdby";

Tuesday, June 25, 2013

ORA-32018: parameter cannot be modified in memory on another instance


##########################
## Error
##########################

ORA-32018: parameter cannot be modified in memory on another instance

### Full Error

SQL> Alter system set large_pool_size=1.3g;
Alter system set large_pool_size=1.3g
*
ERROR at line 1:
ORA-32018: parameter cannot be modified in memory on another instance

##########################
#  Error Occurred
##########################

Error occured while trying to modify the large_pool_size parameter in RAC database,

##########################
## Command Executed
##########################

Alter system set large_pool_size=1.3g;

(or)

Alter system set large_pool_size=1.3g sid='*';

##########################
## Issue Description.
##########################

###

Parameter cant be changed in another instance, needs to be changed by logging to each instance.

##########################
## Solution
##########################

Mention the SID of that particular instance to be changed. Do the same in other instances by changing their instance names.

Alter system set large_pool_size=1300m sid='rac11gn1';


Wednesday, April 10, 2013

ORA-00245: control file backup failed; target is likely on a local file system



Full database backup of 2 node RAC database failed when its trying to run autobackup of controlfile.

##########################
#   Errors 
##########################


Starting Control File and SPFILE Autobackup at 2013-09-04:21:25:34
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of Control File and SPFILE Autobackup command on ORA_SBT_TAPE_1 channel at 09/04/2013 21:25:36
ORA-00245: control file backup failed; target is likely on a local file system


##########################
#   Command Used  
##########################

Controlfile Autobackup.

##########################
#   Informations  
##########################


From 11gR2 onwards, the controlfile backup happens without holding the controlfile enqueue. For non-RAC database, this doesn't change anything. But for RAC database, due to the changes made to the controlfile backup mechanism in 11gR2, any instance in the cluster may write to the snapshot controlfile. Due to this snapshot controlfile need to be visible to all instances. 


The snapshot controlfile MUST be accessible by all nodes of a RAC database, if the snapshot controlfile does not reside on a shared device error will be raised at the time of RMAN backup while taking snapshot of controlfile.


##########################
 Solution 
##########################

 configure snapshot controlfile name to '+ORALIN_RECO/snapcf_oralin.f';  

show snapshot controlfile name;

RMAN> show snapshot controlfile name;

using target database control file instead of recovery catalog
RMAN configuration parameters for database with db_unique_name oralin are:
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2/db_1/dbs/snapcf_oralin1.f'; # default


configure snapshot controlfile name to '+ORALIN_RECO/snapcf_oralin.f';

RMAN> configure snapshot controlfile name to '+ORALIN_RECO/snapcf_oralin.f';

using target database control file instead of recovery catalog
new RMAN configuration parameters:
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+ORALIN_RECO/snapcf_oralin.f';
new RMAN configuration parameters are successfully stored.

After setting snapshot controlfile to Shared location (diskgroup) all the backups are successful.

Thursday, March 21, 2013

ORA-00955: name is already used by an existing object



While doing the Production deployment with the scripts given by application team, encountered the error.

This error which initially looked like strange but then came to know that Oracle works Like this..

##########################
#   Errors 
##########################

ORA-00955: name is already used by an existing object

##########################
#   Command Used  
##########################


SQL> ALTER TABLE tracking ADD CONSTRAINT tracking_pk PRIMARY KEY (menu_id,asset_version);
ALTER TABLE tracking ADD CONSTRAINT tracking_pk PRIMARY KEY (menu_id,asset_version)
*
ERROR at line 1:
ORA-00955: name is already used by an existing object


##########################
#   Informations  

##########################


Initially i thought that a constraint exists with the same name TRACKING_PK 
But there is no constraint created with the same name in this schema. 

So started to Dig more on this...

##########################
 Solution 
##########################

  Drop the indexes which were previously created as a part of Primary Key Constraint. 

The constraint which we are creating is a Primary Constriant. 

When we create a Primary Constraint, Oracle Creates 2 objects.

1) Constraint
2) Index

These 2 objects controls the uniqueness in the table.

So i checked DBA_INDEXES and found that a index with the same name TRACKING_PK still exists in the database which is not allowing to create a constraint with the same name TRACKING_PK

Select owner,index_name,index_type,table_owner,table_name from dba_indexes where table_name='tracking' and owner='AU_USER';


OWNER                          INDEX_NAME                     TABLE_OWNER                    TABLE_NAME 
------------------------------ ------------------------------------------------------------ 
AU_USER       tracking_PK                      AU_USER       tracking   

So Dropped the index and added the constraint and it worked.

Drop index "AU_USER"."tracking_PK";



SQL> ALTER TABLE tracking ADD CONSTRAINT tracking_pk PRIMARY KEY (menu_id,asset_version);

Table altered.


Thursday, March 14, 2013

Oradim -edit -sid gives Unable to start service, OS Error 1056



Every Time i restart the system, i need to bring up the Oracle instance service either from services.msc or using netstart.

So i have decided to make it auto start whenever i restart the system.

##########################
#   Errors 
##########################

Unable to start service, OS Error 1056

##########################
#   Command Used  
##########################

oradim -edit -sid orawin -startmode auto -srvcstart system


##########################
#   Informations  
##########################

I have used this command before in 10g but it didnt give any errors, But when i tried it in 11.2.0.1.0 version of database, getting this error. This seems to be Bug 9584383 and its expected if a service is edited when its running.

But in my case, i have tried by stopping the service also still  the error comes. So whether a service is running or not running, when we try to modify the startmode of a service we will get an error.

##########################
 Solution 
##########################

  We don't have to worry about the error because eventhough we get an error while modifying the service startmode, the command does it purpose, Instance Service startmode gets changed when we issue the command. 

This bug is fixed in.
  • 12.1 (Future Release)
  • 11.2.0.2 (Server Patch Set)
  • 11.2.0.1 Patch 2 on Windows Platforms
Change the Database Service Startmode from Manual to Automatic.

oradim -edit -sid orawin -startmode auto -srvcstart system




Change the Database Service Startmode from Automatic to Manual.

oradim -edit -sid orawin -startmode manual -srvcstart demand



Below Link will be useful if you want start the service using commands like oradim and net start.




Friday, March 8, 2013

ORA-00997: illegal use of LONG datatype

Our Application team has created objects in the USERS tablespace instead of creating the objects in their dedicated tablespace.

So we need to move the objects from USERS tablespace to their default tablespace.

##########################
#   Errors 
##########################


SQL> alter table AU.TRACTION move tablespace AU_DATA;
alter table AU.TRACTION move tablespace AU_DATA
*
ERROR at line 1:
ORA-00997: illegal use of LONG datatype


##########################
#   Command Used  
##########################

alter table AU.TRACTION move tablespace AU_DATA;

##########################
#   Informations  
##########################

Table which i tried to move contains LONG datatype. Tables with Data Types with LONG or LONG RAW cannot be moved.

##########################
 Solution 
##########################

  Export & Import The Table. 

SQL> desc AU.TRACTION

 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 NAME                                      NOT NULL VARCHAR2(256)
 DATA                                      NOT NULL LONG


Alter table move will not work. We need to Export & Import the tables. 

Friday, March 1, 2013

ORA-1613 signalled during: alter database enable public thread 2...

Enabling the thread 2 of a RAC 2 node database fails with the below error.

##########################
#   Errors 
##########################

Wed Feb 27 07:34:57 2013

ORA-1613 signalled during: alter database enable public thread 2...


ORA-1613 signalled during: alter database enable public thread 2...

##########################
#   Command Used  
##########################

Alter Database Enable Public Thread 2;

##########################
#   Informations  
##########################

There are no redo logs which belongs to Thread 2, all the redo logs are created in Thread 1.

##########################
 Solution 
##########################

Added New Redo log groups to thread 2. 


alter database add logfile thread 2 group 7
('/u01/oradata/mydb/redo_02_07.log') size 1024M;

And then enabled the thread.

Alter Database Enable Public Thread 2;


Check Whether Thread is Enabled Or Not :


SQL> select thread#, enabled from v$thread;

   THREAD# ENABLED
---------- --------
         1 PUBLIC
         2 PUBLIC


ORA-01618: redo thread 2 is not enabled - cannot mount & ORA-1613 Error.

RAC Duplication was performed in our database and then we changed the cluster_database=TRUE, and tried to startup the RAC database using SRVCTL command.

First instance Comes Up without any issues but when 2nd instance was trying to come up it fails in the mount stage with the errors. 

##########################
#   Errors 
##########################

Alert Log Info :

ORA-1618 signalled during: ALTER DATABASE   MOUNT...

ORA-01618: redo thread 2 is not enabled - cannot mount

##########################
#   Command Used  
##########################

srvctl start database -d mydb

##########################
#   Informations  
##########################

Initially what i thought is that Redo Thread 2 is not Enable, so i tried to enable it with the below command, which gave me ORA-1613 Error. 

http://stepintooracledba.blogspot.com/2013/03/ora-1613-signalled-during-alter.html

alter database enable public thread 2;

With further investigation found whats the issue.


##########################
 Solution 
##########################

  Error says that Redo Thread 2 Cannot be Enabled. So to Enable Redo Thread we need REDO LOGS, When i checked the Redologs in our database,  i Can see we have 12 Redo Groups all belongs to Thread 1, 

But Actually we have allocated 6 Redo logs to Thread 1 and 6 Redo Logs to Thread 2.  

So Added Redo Log Groups to thread 2 and enabled the thread and it got completed without any issues.

Command Used To Add Redo Logs to thread 2 :


alter database add logfile thread 2 group 7
('/u01/oradata/mydb/redo_02_07.log') size 1024M;

Now Opening the database with srvctl doesnt give any error and both the instances were UP & Running.

ORA-38856: cannot mark instance UNNAMED_INSTANCE_2 (redo thread 2) as enabled

When Performing a RAC 2 node Production Database Restore with disk based backup, by duplicating to a single instance database, Restoration went successful.

Once Restore has completed, i have tried to open the database with RESETLOGS option. But end with the below error, So thought of sharing this little information.

##########################
#   Errors 
##########################

RESETLOGS after incomplete recovery UNTIL CHANGE 13115861554203
ORA-38856 signalled during: alter database open resetlogs...

ORA-38856: cannot mark instance UNNAMED_INSTANCE_2 (redo thread 2) as enabled

##########################
#   Command Used  
##########################

Alter Database open ResetLogs;

##########################
#   Informations  
##########################


This seems to be Bug 4355382 and its expected while doing RAC Restore/Recovery.

##########################
 Solution 
##########################

  ADD _no_recovery_through_resetlogs Parameter and set it to TRUE. 


I have added _no_recovery_through_resetlogs=TRUE parameter to our PFILE and brought up the database to Mount Stage.


Now opened the database with RESETLOGS options and it worked.

This parameter tells oracle not to do any recovery while doing this resetlogs operation.

After Opening the database, the parameter can be removed from the pfile.



expdp estimate_only with & Without compression

Application team wants us to refresh a table which has LOB data's. So we wanna check the size of the export dump file before running it to allocate storage.

Using Datapump its a very easy task but want to check whether we can estimate the size of the Compressed Export Dump file. 

From 11g onwards, advanced compresssion was introduced which will compress DATA's also when compared to the option of compressing only METADATA which was available in 10g.

##########################
#   Task 
##########################

Compare the Datapump Export Dumpfile size using Compressed and Normal one.

Lets see the Difference between using Normal & Compressed

##########################
#   Normal Export Estimation  
##########################

Command to Estimate export dumpfile.


expdp / tables=u1.T1393 estimate_only=y

mydb_2 @ myserver:/dcunix
> expdp \'/ as sysdba\' tables=u1.T1393 estimate_only=y

Export: Release 11.2.0.3.0 - Production on Fri Mar 1 02:32:30 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
Starting "SYS"."SYS_EXPORT_TABLE_01":  "/******** AS SYSDBA" tables=u1.T1393 estimate_only=y
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
.  estimated "u1"."T1393"                           190.4 GB
Total estimation using BLOCKS method: 190.4 GB
Job "SYS"."SYS_EXPORT_TABLE_01" successfully completed at 02:33:06

So Estimated Size using Normal One is 190.4 GB. 



##########################
#   Compressed Export Estimation  
##########################

Command to Estimate Compressed export dumpfile.


expdp / tables=u1.T1393  tables=u1.T1393 compression=all estimate_only=y


mydb_2 @ myserver:/dcunix
> expdp \'/ as sysdba\' tables=u1.T1393 compression=all estimate_only=y

Export: Release 11.2.0.3.0 - Production on Fri Mar 1 02:34:29 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
Starting "SYS"."SYS_EXPORT_TABLE_01":  "/******** AS SYSDBA" tables=u1.T1393 compression=all estimate_only=y
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
.  estimated "u1"."T1393"                           190.4 GB
Total estimation using BLOCKS method: 190.4 GB
Job "SYS"."SYS_EXPORT_TABLE_01" successfully completed at 02:34:40


So Estimated Size using Compressed One is also 190.4 GB. 


##########################
 End of Story 
##########################

 We cant estimate the Size of the compressed Export Dump File. Hope Oracle May introduce this in Upcoming Versions. 


In 10g, we can compress only METADATA's but from 11g onwards we can compress DATA's also.  
 Available Options in 11g, Compression Parameters are,
NONE
METADATA_ONLY
DATA_ONLY
ALL
This Advanced Compression in 11g requires license so make sure before using it.


Wednesday, February 27, 2013

Restore Archive logs from Disk Based Backups gives RMAN-06026: some targets not found - aborting restore RMAN-06102: no channel to restore a backup or copy of log thread 1 seq


One of our DG has gone Out of SYNC. Primary database is in MAXIMUM AVAILABILITY mode and we have specified standby location as an alternate destination.

Now our Primary archive diskgroup got 100% full and one of our team member has ran backup of archive logs to disk (Our Backup Tapes are not Working at that Moment) with the delete input option which has backed up all the Archives and deleted from Primary. 

Now Standby database is looking for archive log which is not in Diskgroup but in RMAN disk backup. 

##########################
#   Task 
##########################

Restore Archive logs from Disk Based Backups.

So our team member has started to make it sync by trying to restore the archives from the backup and got the below errors.

##########################
#   Command Used  
##########################

Command to Restore Archive logs to different location.


run
{
allocate channel c2 device type disk;
set archivelog destination to '/opt/oracle/backup/oralin/arch_bkp/';
restore archivelog from logseq 105408 until logseq 105413;
}

When running the restore got the below errors, 

##########################
#   Errors  
##########################


RMAN> run
{
allocate channel c2 device type disk;
set archivelog destination to '/opt/oracle/backup/oralin/arch_bkp/';
restore archivelog from logseq 105408 until logseq 105413;

}
2> 3> 4> 5> 6> 7>
released channel: ORA_SBT_TAPE_1
released channel: ORA_SBT_TAPE_2
released channel: ORA_SBT_TAPE_3
released channel: ORA_SBT_TAPE_4
released channel: ORA_DISK_1
allocated channel: c1
channel c1: sid=862 devtype=DISK
allocated channel: c2
channel c2: sid=1309 devtype=DISK
executing command: SET ARCHIVELOG DESTINATION
Starting restore at 27-FEB-13
released channel: c1
released channel: c2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 02/27/2013 22:28:14
RMAN-06026: some targets not found - aborting restore
RMAN-06102: no channel to restore a backup or copy of log thread 1 seq 105413 lowscn 13117372976201
RMAN-06102: no channel to restore a backup or copy of log thread 1 seq 105412 lowscn 13117372752915
RMAN-06102: no channel to restore a backup or copy of log thread 1 seq 105411 lowscn 13117372598071
RMAN-06102: no channel to restore a backup or copy of log thread 1 seq 105410 lowscn 13117372461415
RMAN-06102: no channel to restore a backup or copy of log thread 1 seq 105408 lowscn 13117372091300
RMAN>

We spent more time on investigating this error and to resolve the above error. But later on further investigation i found the problem made by the team member and the solution is very simple.




##########################
#   Pre checks 
##########################
  1.  Check whether backup of Archive Logs available. 
  2.  Is the Backup Stored in TAPE or in DISK. 
##########################
 Solution
##########################

 Check whether backup of Archive Logs available. 

RMAN> list backup of archivelog from logseq 105408 until logseq 105413 thread 1;
List of Backup Sets
===================
BS Key  Size       Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
56800656 577.25M    SBT_TAPE    00:07:21     27-FEB-13
        BP Key: 56800662   Status: AVAILABLE  Compressed: NO  Tag: oralin_FULL_0
        Handle: bk_11040_1_808444905   Media: O00100
  List of Archived Logs in backup set 56800656
  Thrd Seq     Low SCN    Low Time  Next SCN   Next Time
  ---- ------- ---------- --------- ---------- ---------
  1    105408  13117372091300 26-FEB-13 13117372283649 26-FEB-13
  1    105409  13117372283649 26-FEB-13 13117372461415 26-FEB-13
  1    105410  13117372461415 26-FEB-13 13117372598071 26-FEB-13
  1    105411  13117372598071 26-FEB-13 13117372752915 26-FEB-13
  1    105412  13117372752915 26-FEB-13 13117372976201 26-FEB-13
  1    105413  13117372976201 26-FEB-13 13117373012970 27-FEB-13

From the above output if you can see the Backup is stored in SBT_TAPE, but our team member has allocated disk Channel because he took backup of archives to disk.

But the archives which needed to restore are already backed up in TAPE and its his understanding that the archives are stored in DISK Backup.

So After removing the Disk channel allocation command, the restore was successful.

RMAN>  run
{
set archivelog destination to '/opt/oracle/backup/oralin/arch_bkp/';
restore archivelog logseq 105409 thread 1;
}
2> 3> 4> 5>

executing command: SET ARCHIVELOG DESTINATION

Starting restore at 27-FEB-13
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: sid=862 devtype=SBT_TAPE
channel ORA_SBT_TAPE_1: Veritas NetBackup for Oracle - Release 7.1 (2011020313)
allocated channel: ORA_SBT_TAPE_2
channel ORA_SBT_TAPE_2: sid=1309 devtype=SBT_TAPE
channel ORA_SBT_TAPE_2: Veritas NetBackup for Oracle - Release 7.1 (2011020313)
allocated channel: ORA_SBT_TAPE_3
channel ORA_SBT_TAPE_3: sid=617 devtype=SBT_TAPE
channel ORA_SBT_TAPE_3: Veritas NetBackup for Oracle - Release 7.1 (2011020313)
allocated channel: ORA_SBT_TAPE_4
channel ORA_SBT_TAPE_4: sid=576 devtype=SBT_TAPE
channel ORA_SBT_TAPE_4: Veritas NetBackup for Oracle - Release 7.1 (2011020313)
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=359 devtype=DISK
channel ORA_SBT_TAPE_1: starting archive log restore to user-specified destination
archive log destination=/opt/oracle/backup/oralin/arch_bkp/
channel ORA_SBT_TAPE_1: restoring archive log
archive log thread=1 sequence=105409
channel ORA_SBT_TAPE_1: reading from backup piece bk_11063_1_808494442
channel ORA_SBT_TAPE_1: restored backup piece 1
piece handle=bk_11063_1_808494442 tag=TAG20130227T134717
channel ORA_SBT_TAPE_1: restore complete, elapsed time: 00:02:37
Finished restore at 27-FEB-13

So the moral of this story is listen what your team member says but always start with your own investigation path. :)

Wednesday, February 20, 2013

Check DATAPUMP Dump File is corrupted or not


Some times we may be in a situation, to check whether the dump file exported long time back is VALID or not.

Or Application Team is saying that the dumpfile provided by us is corrupted.

Easiet way to check whether a DATAPUMP dump file is corrupted or not.

##########################
#  Task 
##########################

Check Whether a DATAPUMP Export DUMPFILE is corrupted or valid

##########################
#   Use  
##########################

 Use SQLFILE Parameter to Detect Corruption 

##########################
## Solution
##########################

 impdp system/*** directory=dump_dir dumpfile=expdp.dmp logfile=corruption_check.log sqlfile=corruption_check.sql 
                                                                                                                                         
What does this sqlfile do? 

Here we have used sqlfile option, which will not import any data to the database.

This will write all DDL statements (which will be executed if a import is performed) into the file which we mentioned in the command. 

So this will read the entire DATAPUMP Export dump file and will report if a corruption is detected.


Monday, February 18, 2013

Find Time Taken(Elapsed) by Objects Types During DATAPUMP Export



Some times we might have thought to know how much time the datapump utility took to export different Object Types.

We have a undocumented parameter in DATAPUMP to check how much it took to export different Object Types.

##########################
#  Task 
##########################

Find Time Taken(Elapsed) by Objects Types During DATAPUMP Export

##########################
#   Parameter  
##########################

 Metrics = Y 

##########################
## Solution
##########################

 Using Metrics Parameter in export, it will list out how much time elapsed to take export of specific object and this information will be recorded in the logfile used in the export command. 


Shrink Temporary Tablespace 11g New Features


In 11g, enhancements has been made to Temporary tablespace.

In Previous Versions, if tempfile has grown big and needs to be resized, the procedure we used to follow is to create a new temporary tablespace and drop the bigger one.

In 11g onwards, Shrinking of temp files can be done and also we have new data dictionary table to find the Free space in the temporary tablespace.

##########################
#  Task 
##########################

Shrinking of Bigger Temp File.

##########################
#   Commands 
##########################

  1.  Alter Tablespace temp shrink tempfile '+DATA/mydb/tempfile/temp.264.798205781' keep 5G; 
  2.  Alter Tablespace temp shrink space; 
New Data Dictionary Table to Check Temporary Tablespace Free Space.

Select * from dba_temp_free_space;

##########################
## Solution
##########################