The problem: Operations with high latency on a filesystem and/or a storage volume can sometimes be attributed to just a few disks 'misbehaving', possibly because they are suffering mechanical failures and/or are going to break completely in the near future.
I/Os of high latency on just a few disks can then appear (more...)
Topic: this post is about some simple tools and techniques that can be used to drill down high-latency I/O events using SystemTap probes.
Motivations: The drill down of I/O latency is an important technique for troubleshooting and benchmarking storage. Average latency values can hide details of what is happening on the storage. Think for example of storage systems with flash and spindles, each serving I/O at different latency. Moreover averaging the measured values over time can hide details in case of varying (more...)
Topic: This post is about collecting and visualizing I/O latency histograms for NetApp filers in C-mode.
Topic: event histogram metric, a script and some related comments on collecting and displaying wait event latency histograms for Oracle performance troubleshooting.
Why: Latency histograms (and by extension wait event histograms) provide very useful information when troubleshooting performance for systems exhibiting response time with multi-mode distribution. In such cases average wait values are often not sufficient to understand the behavior of the system under study and histograms provide a finer level of details. A (more...)
CHEP 2015, the 21st international conference on computing in high energy physics, has taken place in April 2015 in Okinawa, Japan. Here below you can find the links to the pdf of the talks, posters and related preprints of two of the contributions by members the CERN database group. These are on the topics of our tests with scale-out databases (in particular on the Hadoop platform) and on the evolution of the replication technologies (more...)
Introduction: When studying storage performance, the latency drill down is a very important data source. Measuring the average I/O latency is often not enough, latency histograms are proven to more suitable for investigating modern storage systems. This is because for many storage systems the response time has multiple modes: think for example of the common case of storage (more...)
Why PerfSheet4: PerfSheet4 is aimed at querying and displaying time-series data from AWR repository tables. This is very a rich source of information to analyze database workloads and trends in the context of performance analysis or (more...)
- Modern Linux Tools for Oracle Troubleshooting (*)
- A Closer Look at CALIBRATE_IO (*)
- The Importance of Having an Appropriate Data Segmentation (Partitioning) (link is external)
(*) slides can be downloaded from: http://cern.ch/canali
Introduction: Performing input/output to and from the storage media, i.e. physical I/O, is one of the critical tasks underlying all database engines. When accessing data (more...)
Userspace probing is a very powerful technique that can be used to complement the available (more...)
I have first started investigating this topic when deploying 10g RAC databases in 2005. From time to time (more...)
Context and goal: Stack sampling and visualization with flame graphs are powerful techniques to probe the activity of OS processes and in particular for advanced investigations and troubleshooting of Oracle (more...)
Motivations: The techniques and tools described here can be used for performance investigations to complement wait-event based information gathered from the Oracle engine, such as information available with ASH and sql monitoring. They (more...)
Spoiler: If you have reached this article in search of a tool for quantitative analysis of storage performance and in particular for measuring random read I/O in Oracle, I'd rather advise you to use tools that allow generating test workloads in a controlled manner, in a way that can be understood and measured and in particular with latency details together with IOPS measurements. For example (more...)
OraLatencyMap is a SQL*Plus tool, with a core written in PL/SQL, aimed at studying Oracle random I/O by displaying the latency drill-down of the wait event 'db file sequential read' using heatmaps. The tool can also be used to collect and display event latency histograms for any other Oracle wait (more...)