Cellcli can lie to you…

Yes that is true, i said it, cellcli can lie to you there are some special cases were the output of cellcli is not the reality and you should double check it’s output with your standard OS tools. So this the output if dcli calling cellcli on an Exadata rack from an client:

[root@dm01db01 ~]# dcli -g cell_group -l root cellcli -e list cell attributes cellsrvStatus,msStatus,rsStatus
dm01cel01: running       running         running
dm01cel02: running       running         running
dm01cel03:  (more...)

SCAN VIP Troubleshooting

We had a client that was running into a strange issue on their Exadata where new connections coming in through the SCAN were failing.  After doing some troubleshooting, it was discovered that it was related to one of the SCAN listeners not properly accepting requests from new sessions.  The VIP and listener were running, and everything looked normal.

We had the following SCAN setup:

SCAN VIP # VIP IP
1 172.25.2.70
2 (more...)

Troubleshooting Oracle DBFS mount issues

On Exadata the local drives on the compute nodes are not big enough to allow larger exports and often dbfs is configured. In my case I had a 1.2 TB dbfs file system mounted under /dbfs_direct/.

While I was doing some exports yesterday I found that my dbfs wasn’t mounted, running quick crsctl command to bring it online failed:

[oracle@exadb01 ~]$ crsctl start resource dbfs_mount -n exadb01
 CRS-2672: Attempting to start 'dbfs_mount' on 'exadb01'
  (more...)

Kdump to NFS in UEK (Solution)

I’ve previously written about a problem I encountered when kdump is configured to write to an NFS location with UEK (in Exadata software version 11.2.3.2.1). I’m please to report that the root cause of the problem has been identified and there is a very simple workaround.

There were some frustrating times working this particular SR, the most notable being a response that was effectively, “It works for me (and so I’ll (more...)

Using SSD for a temp tablespace on Exadata

I seem to be getting a lot of surprising performance results lately on our X-2 quarter rack Exadata system, which is good – the result you don’t expect is the one that teaches you something new.

This time, I was looking at using a temporary tablespace based on flash disks (more...)

Exadata Storage Server Version 12.1.1.1.0 Released

Oracle has released the much-anticipated version of cellsrv compatible with Oracle Database 12.1.0.1 (patch #16980054). Before thinking about upgrading, read MOS note #1571789.1 carefully.  Unless you are planning to run database 12c on your Exadata, it would be advisable to continue down the 11.2 branch (more...)

Oracle Announces Exadata X4-2

Today, Oracle announced the Exadata X4-2 model.  The X4 has some considerable improvements, namely:

  • 12-core Intel Xeon e5-2697 CPUs, up from the 8-core models found in the X3-2 (hello, database licenses!)
  • 256GB RAM per database server, upgradeable to 512GB
  • 96GB RAM per storage server
  • 800GB Sun Flash F80 cards (more...)

Exadata MAA Presentation – Oracle Day 2013 Istanbul

Exadata Maximum Availability Architecture from Yunus Emre Baransel

Kdump to NFS Broken in UEK

There were problems that affected UEK and NFS when 11.2.3.2.1 was initially released (as covered by Andy Colvin). As mentioned in the comments of Andy’s post: Oracle released an updated ISO with fixes for this problem (patch 16432033).

There were also problems with kdump not (more...)

Info about Exadata’s imageinfo and rpm’s

Recently we noticed that imageinfo was displaying the following information at a customers site. Imageinfo was not displaying data i suspected, so a quick little blogpost about what was going on there.

[root@dm01db01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group -l root "imageinfo | grep 'Image version'"
dm01db01: Image version: 11.2. (more...)

Redo on SSD: effect of redo size (Exadata)

Of all the claims I make about SSD for Oracle databases, the one that generates the most debate is that placing redo logs on SSD is not likely to be effective.  I’ve published data to that effect in particular see  Using SSD for redo on Exadata - pt 2 (more...)

Exadata Write-back cache and free buffer waits

Prior to storage server software version 11.2.3.2.0 (associated with Exadata X3), Exadata Smart Flash Cache was a “write-through” cache, meaning that write operations are applied both to the cache and to the underlying disk devices, but are not signalled as complete until the IO to the (more...)

RDBMS and their bundle-mates

Relational DBMS used to be fairly straightforward product suites, which boiled down to:

  • A big SQL interpreter.
  • A bunch of administrative and operational tools.
  • Some very optional add-ons, often including an application development tool.

Now, however, most RDBMS are sold as part of something bigger.

Exadata – a new learning cuvrve

Perhaps I might be touch late on adopting/exploring/reacting/coping up with the exadata, but, it was always my dream and passion to work with the technologies since the announcement. As we knew Exadata is not something we can download and configure that easily on PC like those of traditional database or (more...)

About cellcli calibrate

With cellcli you can run the calibrate command, this will measure the performance of your harddisks and flashcards. It will eventually report to you the throughput and amount of IOPS that was measured during the calibrate, it will also report to you which luns are not performing up to par.

(more...)

Why my EM12c is giving Metric evaluation error for Exadata cell targets?

As part of my Cloud Control journey I encountered a strange problem where I got the following error for few Exadata Storage Server (cell) targets:

Metric evaluation error start - oracle.sysman.emSDK.agent.fetchlet.exception.FetchletException: em_error=Failed to execute_exadata_response.pl ssh -q -o ConnectTimeout=60 -o BatchMode=yes -o StrictHostKeyChecking=no -o  (more...)

My #OOW13 Session: Are Indexes Unnecessary in Exadata

Thanks to all those who came to my session "Are Indexes Unnecessary in Exadata" at Oracle Open World 2013. Considering it was late afternoon of the last day of the conference, especially after the appreciation concert, it was a pleasure to see a packed room. Judging by the interaction I (more...)

Stuck Timezone Upgrades and Smart Scans

I heard a strange report from a client late last week - they weren't getting any smart scans on their Exadata - a data warehouse, no less (and we checked to see if cell_offload_processing was set to true).  Queries would run normally, but for some reason, the "cell physical IO (more...)

Virtualization on Exadata?

A while back, I noticed this session in the Open World 2013 content catalog.  I didn't think much of it at the time, other than that it had a catchy title.  The other day, I downloaded the latest Exadata deployment utility via the OneCommand tool.  I noticed something a little (more...)

Analyzing page allocation failures on Exadata

I have seen several clients who are struggling to decode page allocation failures on Exadata, in this post i will try to explain how to read the backtrace. The following is an anonymized client case where page allocation failures are leading up to a node reboot.

Jan 1 11:58:02 dm01db01  (more...)