Buffer cache        =  1048448    4.0g  25%  details with -bufcache

In HP-UX, The memory allocated to (dynamic) buffer cache is controlled by two parameters dbc_min_pct and dbc_max_pct. It can vary between dbc_min_pct and dbc_max_pct percent of the total RAM. They default to 5 and 50 respectively. For a system that is running an Oracle database value of 50 for dbc_max_pct is way too high. That means half of the memory is going to be allocated to OS buffer cache. As Oracle has got its own buffer cache so the OS cache is not of much use. As mentioned in the metalink note 726652.1, the value of dbc_max_pct can be safely lowered without impacting the Oracle database performance. In many of the threads (on HP website) people have suggested the value of 10 for db_max_pct. Not sure if it is more like a thumb rule but in the same metalink note (726652.1) it is mentioned that if %rcache in sar -b is above 90, that means your OS buffer cache is adequately sized.

After setting the value of dbc_max_pct to 15 (It will be changed to 10, finally), around 1.6 GB more memory was freed. Also there was no impact on the database or OS performance. Here are few of the metalink notes and threads on HP-UX website that talk about these parameters in detail:

Oracle Shadow Processes Are Taking Too Much Memory (Doc ID 434535.1)

How OS Buffer Cache Size Affects Db Performance (Doc ID 726652.1)

Commonly Misconfigured HP-UX Kernel Parameters (Doc ID 68105.1)

http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1306231311459+28353475&threadId=1266914

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=727618

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=467288

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=750342

Scenario was: Oracle is installed from “oracle” user and all runs well. There is a new OS user “test1″ that also needs to use sqlplus. So granted the necessary permissions on ORACLE_HOME to test1. Tried to connect sqlplus scott/tiger@DB and yes it works. But while trying sqlplus scott/tiger it throws:

$ sqlplus scott/tiger

SQL*Plus: Release 10.2.0.5.0 - Production on Wed May 18 09:32:35 2011

Copyright (c) 1982, 2010, Oracle.  All Rights Reserved.

ERROR:
ORA-12547: TNS:lost contact

Enter user-name: ^C
$

Did a lot of troubleshooting including checking tnsnames.ora, sqlnet.ora, listener.ora and so on. Nothing was hitting my mind so finally raised an SR. And it has to do with the permissions of the $ORACLE_HOME/bin/oracle binary. The permissions of oracle executable should be rwsr-s–x or 6751 but they were not. See below:

$ id
uid=241(test1) gid=202(users) groups=1(staff),13(dba)
$

$ cd $ORACLE_HOME/bin
$ ls -ltr oracle
-rwxr-xr-x    1 oracle   dba       136803483 Mar 16 20:32 oracle
$

$ chmod 6751 oracle
$ ls -ltr oracle
-rwsr-s--x    1 oracle   dba       136803483 Mar 16 20:32 oracle
$

$ sqlplus scott/tiger

SQL*Plus: Release 10.2.0.5.0 - Production on Wed May 18 10:23:27 2011

Copyright (c) 1982, 2010, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> show user
USER is "SCOTT"
SQL>

SQL> @wait

EVENT                                                              COUNT(*)
---------------------------------------------------------------- ----------
wait for possible quiesce finish                                          1
Streams AQ: qmn coordinator idle wait                                     1
Streams AQ: qmn slave idle wait                                           1
Streams AQ: waiting for time management or cleanup tasks                  1
SQL*Net message to client                                                 1
smon timer                                                                1
pmon timer                                                                1
jobq slave wait                                                           4
rdbms ipc message                                                        11
SQL*Net message from client                                              27
resmgr:become active                                                    322

11 rows selected.

SQL>

Tanel’s snapper showed something like:

SQL> @snapper ash 5 1 all
Sampling with interval 5 seconds, 1 times...

-- Session Snapper v3.11 by Tanel Poder @ E2SN ( http://tech.e2sn.com )

-----------------------------------------------------------------------
Active% | SQL_ID          | EVENT                     | WAIT_CLASS
-----------------------------------------------------------------------
 26322% | 4ffu7nb93c2c9   | resmgr:become active      | Scheduler
  1900% | 2wn958z7gzh57   | resmgr:become active      | Scheduler
  1400% | 9d9bg2r538nd2   | resmgr:become active      | Scheduler
   600% | 4d3k70q6y344k   | resmgr:become active      | Scheduler
   500% | d6vwqbw6r2ffk   | resmgr:become active      | Scheduler
   500% | 4tsrz92mmshbw   | resmgr:become active      | Scheduler
   200% | 37td1bbvc1a69   | resmgr:become active      | Scheduler
   100% | ftdjfxws0s8q9   | resmgr:become active      | Scheduler
   100% | 41apc1bjqrfbv   | resmgr:become active      | Scheduler
   100% | af9d8aqtkvn02   | resmgr:become active      | Scheduler

--  End of ASH snap 1, end=2011-02-10 11:06:40, seconds=5, samples_taken=23

PL/SQL procedure successfully completed.

SQL>

If we check the description of the wait event, it says:

The session is waiting for a resource manager active session slot. This event occurs when the resource manager is enabled and the number of active sessions in the session’s current consumer group exceeds the current resource plan’s active session limit for the consumer group. To reduce the occurrence of this wait event, increase the active session limit for the session’s current consumer group.

But if we check the resource_limit settings:

SQL> show parameter resource

NAME_COL_PLUS_SHOW_PARAM   TYPE		VALUE_COL_PLUS_SHOW_PARAM
----------------------- -----------     --------------------------
resource_limit            boolean		FALSE
resource_manager_plan     string

SQL>

What ? Resource manager is not enabled. But why all the sessions are waiting for resmgr:become active and nobody is able to login ?

A bit of googling lead me to this page from where I got the clue.

Generally, this wait situation occurs when you execute certain EMCA operations such as the operation for creating the EM repository. As a result of these operations, the systems implicity switches to QUIESCE mode. Therefore, all database connections (except users SYS and SYSTEM) must wait for “resmgr:become active”. In this case, refer to Note 1044758 and execute the following command if necessary:

ALTER SYSTEM UNQUIESCE;

I asked around in the DBA team and one of the guys was trying to configure EM for the database due to which system switched tto QUIESCE mode and all the sessions were waiting on resmgr:become active.

After canceling the operation, the wait event was gone and everything was working normally.

• Few users in the database were assigned the CREATE SESSION privilege through a password protected role (That role was the default role for that user). 10.2.0.5 onwards, password protected roles can’t be set as default roles. The alternate is to either disable the password for the role or assign CREATE SESSION directly to the user, not through a role.
• After the upgrade, few procedures became invalid and while compiling started giving ORA-00918: COLUMN AMBIGUOUSLY DEFINED. The issue was bug 2846640 which is fixed in 10.2. Actually, in few of the queries using ANSI syntax, the developer didn’t qualify the column names with table names. It worked fine in 9i but due to the bug getting fixed in 10g, it started giving ORA-00918. The simple solution is to prefix the column name with the table name.
• Few of the application schema owner users complained that they were not able to modify the procedures/packages in their own schemas. The schemas were not assigned CREATE PROCEDURE privilege but as per documentation, they should be able to modify the existing procedures/packages owned by them. This again is a documentation bug. It worked fine in 9i but in 10g onwards you need to have either a CREATE PROCEDURE or ALTER ANY PROCEDURE privilege (a risky one) to be able to edit the PL/SQL units in your own schema.

These were few of the issues encountered, rest of the upgrade was super smooth !

Happy upgrading !

While running impdp import in 11g, you hit:

ORA-39083: Object type INDEX failed to create with error:
ORA-14102: only one LOGGING or NOLOGGING clause may be specified

It is related to bug 9015411 where DBMS_METADATA.GET_DDL creates incorrect create index statement by dumping both LOGGING and NO LOGGING clauses. Due to this the CREATE INDEX statement, while running impdp fails with the above error. It applies to 11.2.0.1 (Metalink doc id 1066635.1)

Fix is to install the patch, if it is available for your platform. Another workaround is given in this OTN thread i.e. strip the create index statement of storage related information by using TRANSFORM=SEGMENT_ATTRIBUTES:N:INDEX & TRANSFORM=SEGMENT_ATTRIBUTES:N:CONSTRAINT

exp system/manager@DB owner=ABC file=ABC.dmp log =ABC.log

Export: Release 8.1.7.1.0 - Production on Fri Nov 12 04:21:05 2010

(c) Copyright 2000 Oracle Corporation.  All rights reserved.

EXP-00056: ORACLE error 28002 encountered
ORA-28002: the password will expire within 11 days
EXP-00056: ORACLE error 24309 encountered
ORA-24309: already connected to a server
EXP-00000: Export terminated unsuccessfully

I googled and metalink’ed (oops…is there anything like that ? ) a bit and found that it was bug 1654141 where user accounts in grace period cannot perform export. It is fixed in Oracle 9i (version 9.0.0.0 as per metalink). The obvious work around is to change the password and then try again. Thought about posting it here so that Google can give little better results if someone in trouble comes searching for it .

exec dbms_mview.refresh('SCHEMA1.MVIEW1','C'); 

i wrote

exec dbms_mview.refresh('SCHEMA1','MVIEW1','C');

And it gave me:

ERROR at line 1:
ORA-30019: Illegal rollback Segment operation in Automatic Undo mode
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 794
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 851
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 832
ORA-06512: at line 1

which has nothing to do with the real error. Take care !

ERROR at line 1:
ORA-03001: unimplemented feature
ORA-06512: at "SYS.DBMS_STATS", line 13336
ORA-06512: at "SYS.DBMS_STATS", line 13682
ORA-06512: at "SYS.DBMS_STATS", line 13760
ORA-06512: at "SYS.DBMS_STATS", line 13719
ORA-06512: at line 1

Little bit of searching on Metalink revealed that i had hit Bug no 6011068 which points to the base Bug 576661 which is related to function based indexes. There were 2 function based indexes in the schema. Before talking about the workaround let us re-produce the test case. Here i am doing it on my laptop (10.2.0.1 on Windows XP 32 bit)

SCOTT@TESTING >create table test1 as select * from emp;

Table created.

SCOTT@TESTING >create index ind1 on test1(comm,1);

Index created.

SCOTT@TESTING >

SYSTEM@TESTING>exec dbms_stats.gather_schema_stats('SCOTT');
BEGIN dbms_stats.gather_schema_stats('SCOTT'); END;

*
ERROR at line 1:
ORA-03001: unimplemented feature
ORA-06512: at "SYS.DBMS_STATS", line 13210
ORA-06512: at "SYS.DBMS_STATS", line 13556
ORA-06512: at "SYS.DBMS_STATS", line 13634
ORA-06512: at "SYS.DBMS_STATS", line 13593
ORA-06512: at line 1

SYSTEM@TESTING>

As suggested in the metalink article let us set event 3001 before running the GATHER_SCHEMA_STATS command.

SYSTEM@TESTING >alter session set tracefile_identifier=stats1;

Session altered.

SYSTEM@TESTING >alter session set events '3001 trace name ERRORSTACK level 3';

Session altered.

SYSTEM@TESTING >exec dbms_stats.gather_schema_stats('SCOTT');
BEGIN dbms_stats.gather_schema_stats('SCOTT'); END;

*
ERROR at line 1:
ORA-03001: unimplemented feature
ORA-06512: at "SYS.DBMS_STATS", line 13210
ORA-06512: at "SYS.DBMS_STATS", line 13556
ORA-06512: at "SYS.DBMS_STATS", line 13634
ORA-06512: at "SYS.DBMS_STATS", line 13593
ORA-06512: at line 1

SYSTEM@TESTING >

Part of the trace file reads:

ksedmp: internal or fatal error
ORA-03001: unimplemented feature
Current SQL statement for this session:
select /*+ no_parallel_index(t,IND1) dbms_stats cursor_sharing_exact use_weak_name_resl dynamic_sampling(0) no_monitoring no_expand index(t,"IND1") */ count(*) as nrw,count(distinct sys_op_lbid(51966,'L',t.rowid)) as nlb,count(distinct hextoraw(sys_op_descend("COMM")||sys_op_descend(1))) as ndk,sys_op_countchg(substrb(t.rowid,1,15),1) as clf from "SCOTT"."TEST1" t where "COMM" is not null or 1 is not null
----- PL/SQL Call Stack -----
object      line  object
handle    number  name
65AA77D4      9406  package body SYS.DBMS_STATS
65AA77D4      9919  package body SYS.DBMS_STATS

So the problem is being caused by the index ind1 we created on (comm,1). This bug has been fixed in 10.2.0.5 and 11.1.0.7. The available workaround for other versions is to create index using 1 as character instead of number.

SCOTT@TESTING >drop index ind1;

Index dropped.

SCOTT@TESTING >create index ind1 on test1(comm,'1');

Index created.

SCOTT@TESTING >

And now running GATHER_SCHEMA_STATS:

SYSTEM@TESTING >exec dbms_stats.gather_schema_stats('SCOTT');

PL/SQL procedure successfully completed.

SYSTEM@TESTING >

1. User will log in to the application with his username (Lets say USER01) and password (basically every application user is a database user).
2. He is provided with a screen where he can enter details about the job and the code behind the button calls a PL/SQL procedure in the main application schema (lets say APP1) which in turn uses DBMS_SCHEDULER.CREATE_JOB to schedule the new job.
3. The ultimate task of the job is to move data from one table in the first database to a table in the second database using a DB Link.
4. There is a VPD policy applied on all the application users to restrict the view of data. Policy function uses SYS_CONTEXT to fetch some information about the logged in user. The main application user APP1 is exempted from policy and can see the whole data.

Things seem to work fine till the schedule part. But when the job runs it hits ORA-02070: database does not support operator SYS_CONTEXT in this context as SYS_CONTEXT and DB link doesn’t go together.

I did a bit of troubleshooting and came to know that the job gets created with JOB_CREATOR (a field in DBA_SCHEDULER_JOBS) as the user who is logged in (ie USER001). Now when the job runs from USER001, there is a VPD policy which is going to append a where clause to the query and there is a DB link being used, hence ORA-02070.

So the way out would be to schedule and run the job from some user that has no VPD policy applied to it. The best choice would obviously be the main application user; APP1 but as the user logs in with his own username so the job would always be created with JOB_CREATOR as USER001. After a bit of thought provoking an idea hit me:

Create a table in the APP1 schema. Now when the user schedules the job, insert the values of the parameters required to schedule the job in the table. Schedule one master job in APP1 schema which would read this table and in turn call DBMS_SCHEDULER.CREATE_JOB to schedule the job required by the user. Now as there is no policy applied on the APP1 database user so the job is not going to hit ORA-02070. The frequency of the master job can be set as per the requirements. To identify which entries in the table have been processed either keep a flag which can be updated or delete the record from the table after scheduling.

That is how it clicked in my mind at that time. Suggestions about any other better (or worse ) methods are welcome

PS: About the title: Nothing really was coming into my mind so i picked up the all three words and titled it DBMS_SCHEDULER, DBMS_RLS and SYS_CONTEXT