Extended memory VM instances in OCI

While provisioning compute instances in OCI, you may come across scenarios which need more memory and cores than what the standard shapes provide. Extended memory VMs are meant to solve that problem. Such VMs are able to access cores and memory across a single physical socket and it allows them to go beyond the limits of standard shapes. There are no additional charges for using this specific feature. Customer is charged on the basis of total number of cores and total amount of memory used. ...

September 21, 2023 at 3:48 PM · 2 min · 216 words · Amardeep Sidhu

Can’t su to oracle user

Last week, got this issue reported by a DBA that he wasn’t able to su to oracle user from root on a Oracle Base Database VM in OCI. The login of opc user worked fine and he could do sudo su to root but he couldn’t su to oracle. When he did it just came back to root shell. [root@xxx ~]# su - oracle Last login: Fri Jan 12 10:20:38 UTC 2023 [root@xxx ~]# There was nothing relevant in /var/log/messages or /var/log/secure. I tried it for some other user and it worked fine. Then I suspected something with the profile of oracle user and voila ! The .bashrc looked like this ...

January 20, 2023 at 4:55 PM · 1 min · 160 words · Amardeep Sidhu

Adding a new cell to Exadata with asm scoped security enabled

A customer is using an Exadata X8M-2 machine with multiple VMs (hence multiple clusters). I was working on adding a new storage cell to the configuration. After creating griddisks on the new cell and updating cellip.ora on all the VMs, I noticed that none of the clusters was able to see the new griddisks. I checked the usual suspects like if asm_diskstring was set properly, private network subnet mask on new cell was same as the old ones. All looked good. I started searching about the issue and stumbled upon some references mentioning ASM scoped security. I checked on one of the existing cells and that actually was the issue. The existing nodes had it enabled while the new one hadn’t. Running this command on an existing cell ...

May 9, 2022 at 1:26 PM · 3 min · 555 words · Amardeep Sidhu

Smokescreen detects traffic from an Exadata VM

A customer who is using an Exadata X8M-2 with multiple VMs had Smokescreen deployed in their company recently and they reported an issue that one of the Smokescreen decoy servers in their DC was seeing traffic from one of the Exadata VMs on a certain port. That was rather confusing as that port was the database listener port on that VM and why would a VM with Oracle RAC deployed try to access any random IP on the listener port. Also it was happening only for this VM. Nothing for so many other VMs. ...

March 21, 2022 at 7:30 PM · 1 min · 182 words · Amardeep Sidhu

Unable to connect to the database with SQLPlus

I was working on configuring a new database for backup to ZDLRA and hit this issue while testing a controlfile backup via Enterprise Manager -> Schedule backup. It could happen in any environment. Unable to connect to the database with SQLPlus, either because the database is down or due to an environment issue such as incorrectly specified... If the database is up, check the database target monitoring properties and verify that the Oracle Home value is correct. The 2nd line clearly tells the problem but since the Cluster Database status in EM was green, so it took me a while to figure it out. Issue turned out to be a missing / in the end of ORACLE_HOME specified in monitoring configuration of the cluster database. The DB home specified was /u01/app/oracle/product/11.2.0.4/dbhome_1 instead of /u01/app/oracle/product/11.2.0.4/dbhome_1 /. ...

January 6, 2022 at 6:41 PM · 2 min · 216 words · Amardeep Sidhu

File system already present at specified mount point /dbfs_direct

It was actually funny. So thought about posting it that sometimes how we can miss the absolute basics. This customer is using a virtualized Exadata with multiple VMs. One VM hosts the database meant to be used for dbfs and another VM connects to this DB over IB to mount dbfs file system using dbfs_client. One day VMs were rebooted and due to some reason the dbfs filesystem didn’t mount on startup. It went on for few days and they couldn’t mount it. One day I got a chance to look at it and the error they were facing was: ...

June 5, 2021 at 8:19 PM · 2 min · 248 words · Amardeep Sidhu

Database performance degradation due to multipath issues

To put it in bit of an Indian context, database is not your daughter-in-law that you can blame it for every performance issue that occurs in the environment. But it does happen. Most of the time it is the database that is blamed for all such issues. Many times, the issues are in some other layer like OS, network or storage. Faced this issue recently at one of the customer sites where performance in one of the databases went down suddenly. It was a 2 node RAC on 12.1.0.2 running on Linux 7 using some kind of Hitachi SSD storage array. There were no changes as per DBA, application, OS and storage teams. But something must have changed somewhere. Otherwise why would performance degrade just like that. I & my colleague checked some details and found that something happened in the morning a day before. Starting from that point in time, the execution time for all the commonly run queries shot up. Generally speaking, when all the queries are doing bad and you are sure that nothing has been changed on the database side, the reasons could be outside the database. But being a DBA, it is not easy to prove that. We took AWRs from good and bad times and the wait events section looked like this: ...

March 22, 2021 at 12:54 PM · 3 min · 578 words · Amardeep Sidhu

ZDLRA patching

To be honest, Fernando Simon has already documented all the steps needed in ZDLRA patching . So this post is more like a reference post for me and it points to the links on his blog. One thing he could change though are the post titles. He also agrees ;) https://twitter.com/amardeep_sidhu/status/1370304085245661192 ZDLRA patching is broadly divided into two parts. First part is where you patch the RA library and Grid & DB homes. Second part includes compute node & storage cell image patch and patches for IB/RoCE switches. Second part is exactly similar to Exadata except that it is bit restricted in terms of image versions that you can use. Only the versions that are certified for ZDLRA can be used. Also the RA library version and the Exadata image version should be compatible with each other. So if you are planning to patch only one part; RA library or the image, make sure that both the components stay compatible. The MOS note that has all these details is 1927416.1. This note should be the first place to go when you are planning to patch a ZDLRA. The steps for upgrade/patch, image patching are given in MOS note 2028931.1. There is another note 2639262.1 that discusses some of the known issues that you may face while doing the patching. It is important to review all three notes before you plan to patch. ...

March 12, 2021 at 10:30 PM · 2 min · 422 words · Amardeep Sidhu

[FATAL] [INS-44000] Passwordless SSH connectivity is not setup

Faced this while running installer for setting up a 2 node RAC setup (version 19.8) on an Oracle SuperCluster. The error reported in the log is: [FATAL] [INS-44000] Passwordless SSH connectivity is not setup from the local node node1 to the following nodes: [node2] [INS-06006] Passwordless SSH connectivity not set up between the following node(s): [node2] From the error it appears that the ssh is not setup between two nodes but actually that is not the case. Here the error message is bit misleading. It turned out to be an issue with scp with openssh version 8.x. Running the setup with -debug option gives the clue: ...

March 10, 2021 at 6:17 PM · 1 min · 212 words · Amardeep Sidhu

Doing an Exadata mixed cells config with OEDA

Earlier versions of OEDA didn’t allow you to have mixed cells in the configuration i.e. High Capacity (HC) and Extreme Flash (EF). The way to deal with that configuration was that deploy the system with either HC or EF cells and then manually configure the remaining cells. I am not sure when did it change but the newer versions allow you have mixed type of cells in a single OEDA configuration. Once you select the hardware, there is an additional option called Enable Additional Storage, where you can select the other type of cells. The minimum number of cells has to be three to use this option. Also the cells that are at the bottom of the rack physically should be selected as main storage and the other cells should be added as additional storage as that is how OEDA builds the configuration files. ...

October 27, 2020 at 6:53 PM · 2 min · 285 words · Amardeep Sidhu