Recovering ODA VMs From Lost ACFS Snapshots

This is a continuation of my previous post regarding dropped ACFS snapshots.  In this scenario, a user logged in to a virtualized ODA system and deleted the underlying ACFS snapshots for multiple virtual machines on the host.  Oracle advises how to back up and restore guest VMs on an ODA in MOS note #1633166.1.

The basic process to get a clean backup of the VM is to shut it down, take a (more...)

Where’d My VMs Go? Adventures in Simple Linux Forensics

**While the story is true, any identifying details have been changed to protect both the guilty and the innocent**

I woke up to a message the other day from a colleague on a project where I occasionally provide assistance.  He was a bit panicked – they use an Oracle Database Appliance as a utility server for their environment.  The ODA is virtualized, and provides the database backend for their RMAN catalog, OEM repository, (more...)

Exadata and WebLogic CVE-2019-2725

I can still remember the surprise when Frits Hoogland pointed out that Oracle had replaced the old OC4J instance on Exadata storage servers with a WebLogic service. I wasn’t quite sure what to make of this, but it’s become apparent as things move cloudward that it would enable an entire set of REST operations for Exadata on both storage and compute tiers. This becomes interesting due to the recent 0-day vulnerability for WebLogic, CVE-2019-2725. According (more...)

Checking Flash Cache Mode in Exadata Cloud

While looking at an exachk report for an Exadata rack running in Oracle Cloud Infrastructure, I saw an interesting finding:

Since this is an Oracle cloud environment, we don’t have any access to view the storage servers – they are managed and maintained by Oracle.  For several reasons, I believed this to be a false alert, but wanted to see if I could determine what the flashcachemode setting is.

On some flavors of Exadata (more...)

Changing IP Addresses on InfiniBand Network with 12.2 GI on Exadata

I was tasked a while back with modifying the IP addresses assigned to the InfiniBand network on an X6-2 rack.  We were configuring a ZFS appliance to share between 2 Exadata clusters, and we wanted to keep them on separate network ranges.  In this case, we were moving the interfaces from 192.168.10.x/22 (the Exadata default network range) to 192.168.14.x/22. The process is documented very well by Oracle (more...)

Exadata Storage Server Patching in 18.1.1 and Above

One of the cool new features that was introduced with 18.1.0.0.0 was a subtle change in the Exadata storage server patching process.  While not required, this new functionality removes the need to use patchmgr to drive Exadata storage server patches.  In Juan Loaiza’s Open World session on new Exadata features, this slide jumped out to me:

While this slide may look like marketing fluff, this new feature turns out (more...)

Thoughts About the Exadata X7-2 Launch

Last week’s Oracle Open World conference included many different announcements – one that impacts my work was the release of a new generation of Exadata machines – this time, the X7-2. If you’ve followed Exadata development over the years, there has been a typical set of features that come in with each release:

  • Increase in CPU core count
  • Increase in disk storage (typically 2x)
  • Increase in flash storage (typically 2x)
  • New software features

The screenshot (more...)

Exadata Upgrade to 12.2.0.1 – The Missing Step

I decided this week to be a little brave and upgrade one of the Enkitec Exadata racks to 12.2.0.1.  I installed the 12.2.1.0.0 Exadata image a few weeks ago, and have been waiting for a chance to upgrade clusterware to 12.2.  Thankfully, Oracle provides a very good note for this, but I did hit one large snag that should be documented.

The process for upgrading GI to 12.2 (more...)

Parallelizing datapatch execution

When applying quarterly PSUs to an Oracle database, there are generally two steps – one step that applies updates to the binaries inside the Oracle home, and one step that updates objects inside the database catalog via registry$sqlpatch. The first step has long been performed with OPatch, but beginning with Oracle 12.1.0.1, the second task has a new script – datapatch. This script automates the post-patch actions, removing the need to run (more...)

Exadata Cell Patching – USB I/O Errors

I was playing around with the Exadata X2-2 in the Enkitec lab this weekend, and hit an interesting issue when patching the storage servers.  We were taking the system up to version 12.1.2.3.3 for testing purposes.  I fired off the patchmgr script, and one of the storage servers failed when beginning the first phase of the patching cycle:

[root@enkdb03 patch_12.1.2.3.3.161208]# ./patchmgr -cells cell_group -patch -ignore_alerts
 (more...)