Exadata motherboard replacement

This photo story is about a motherboard replacement in an Exadata X2-2 database server (Sun Fire X4170 M2).

While setting up a half rack Exadata system, we discovered a problem with the motherboard in database server 4, so we got ourselves a new one. Even for an internal system, I had to log an SR, get our hardware team to review the diagnostics and have them confirm that a new motherboard is required. I had a full customer experience with me being both the customer and Support Engineer! Anyway, a replacement motherboard arrived the next day.

We stopped the clusterware (more...)

ASM in Exadata

ASM is a critical component of the Exadata software stack. It is also a bit different - compared to non-Exadata environments. It still manages your disk groups, but builds those with grid disks. It still takes care of disk errors, but also handles predictive disk failures. It doesn't like external redundancy and ACFS, but it makes the disk group smart scan capable. Let's have a closer look.

Grid disks

In Exadata the ASM disks live on storage cells and are presented to compute nodes (where ASM instances run) via Oracle proprietary iDB protocol. Each storage cell has 12 hard disks (more...)

ASM files number 12 and 254

The staleness directory (ASM file number 12) contains metadata to map the slots in the staleness registry to particular disks and ASM clients. The staleness registry (ASM file number 254) tracks allocation units that become stale while the disks are offline. This applies to normal and high redundancy disk groups with the attribute COMPATIBLE.RDBMS set to 11.1 or higher. The staleness metadata is created when needed, and grows to accommodate additional offline disks.

When a disk goes offline, each RDBMS instance gets a slot in the staleness registry for that disk. This slot has a bit for each allocation unit (more...)

ASM files number 10 and 11

ASM metadata file number 10 is ASM user directory and ASM file number 11 is ASM group directory. These are supporting structures for ASM file access control feature.

ASM file access control can be used to restrict file access to specific ASM clients (typically databases), based on the operating system effective user identification number of a database home owner.

This information is externalized via V$ASM_USER, V$ASM_USERGROUP and V$ASM_USERGROUP_MEMBER views.

ASM users and groups

To make use of ASM file access control feature, we need to have the operating system users and groups in place. We would then add them to (more...)

ASM file number 9

The attributes directory - ASM file number 9 - contains metadata about disk group attributes. The attributes directory exists only in disk groups where COMPATIBLE.ASM is 11.1 or higher.

Disk group attributes were introduced in ASM version 11.1[1] and can be used to fine tune the disk group properties. It is worth noting that some attributes can be set only at the time the disk group is created, while other attributes can be set at any time. Some attributes might be stored in the disk header, for example the AU_SIZE, while some other attributes, for (more...)

ASM file number 8

The disk Used Space Directory (USD) – ASM file number 8 - maintains the number of allocation units (AU) used per zone, per disk in a disk group. The USD is split into a set of Used Space Entries (USE). Each USE will maintain a counter for the number of used AUs per disk, per zone. A disk zone can be either HOT or COLD.

This structure is version 11.2 specific and is relevant to the Intelligent Data Placement feature. The USD will be present in a newly created disk group in version 11.2 or when the ASM (more...)