IPython Notebooks for Querying Apache Impala

Topic: in this post you can find examples of how to get started with using IPython/Jupyter notebooks for querying Apache Impala.

Apache Impala is an open source massively parallel processing (MPP) SQL Query Engine for Apache Hadoop. This post explores the use of IPython for querying Impala and generates from the notes of a few tests I ran recently on our systems. For completeness please that that several additional options exist to query Impala, some (more...)

IPython Notebooks for Apache Impala, by Example

Topic: in this post you can find examples of how to get started with querying Apache Impala using IPython/Jupyter notebooks.

SystemTap Guru Mode and Oracle SQL Parsing

Introduction and motivations

SystemTap and dynamic tracing tools in general give administrators great control on their systems with the relatively little additional effort to learn the new tools. In this post you will see of how SystemTap that can be used to modify data on the fly at runtime. The outcome is a form of "live patching". Examples are provided on how to apply these ideas to Oracle SQL parsing functionality. This type of "guru mode" use (more...)

PerfSheet.js: Oracle AWR Data Visualization in the Browser with JavaScript Pivot Charts

PerfSheet.js is a tool aimed at DBAs and Oracle performance analysts. It provides a simplified interface to extract and visualize AWR time series data in the browser using javascript.

The reason for a tool like PerfSheet.js is to make the analysis of AWR data easier by providing a graphical interactive interface and by automating several repetitive steps of data extraction and chart preparation. Pivot charts provide a flexible and easy to use way to navigate (more...)

PerfSheet.js: Oracle AWR Data Visualization in the Browser with JavaScript Pivot Charts

PerfSheet.js is a tool aimed at DBAs and Oracle performance analysts. It provides a simplified interface to extract and visualize AWR time series data in the browser using javascript.

Linux Perf Probes for Oracle Tracing

Topic: this post is about Linux perf and uprobes for tracing and profiling Oracle workloads for advanced troubleshooting.

Context

The recent progress and maturity of some of the Linux dynamic tracing tools has raised interest in applying these techniques to Oracle troubleshooting and performance investigations. See Brendan Gregg's web pages for summary and future developments on dynamic traces for Linux. Some recent work on applying these tools and techniques to Oracle can be found (more...)

Linux Perf Probes for Oracle Tracing

 Topic: this post is about Linux perf and uprobes for tracing and profiling Oracle workloads for advanced troubleshooting.

 

Context

Extended Stack Profiling – Ideas, Tools and Comments

Topic: This post provides a short summary and pointers to previous work on Extended Stack Profiling for troubleshooting and performance investigations.

Understanding the workload is an important part of troubleshooting activities. We seek answers to questions like: what is the system doing, where is the time spent, which code paths are most used, what are the wait events, etc. Sometimes the relevant diagnostic data is easy to find, other times we need to dig (more...)

Slides of the CERN Talks at UKOUG Tech15

This is a short blog post with the links to download the slides of the presentations with CERN speakers at UKOUG Tech15:


  • Migrating from Streams to GoldenGate12c, Zbigniew Baranowski, slides (pptx)
  • Stack Traces & Flame Graphs for Oracle Troubleshooting, Luca Canali, slides (pptx, pdf)


  • Many thanks to UKOUG for delivering once more a technical conference of very high quality and to all the participants who make the (more...)

    Oracle Wait Events Investigated with Extended Stack Profiling and Flame Graphs

    Topic: this post is about investigating Oracle wait events using stack profiles and flame graphs extended with OS-process state and Oracle wait event details.

    Context: The case of the DB Time > CPU Time + Wait Time

    Oracle instrumentation provides wait event and CPU time accounting, a powerful and readily accessible data source for performance troubleshooting. An Oracle session at a given point in time is either on CPU, for example when processing data from cache, (more...)

    Linux Kernel Stack Profiling and Flame Graphs Applied to Oracle Investigations

    Topic: This blog post is about kernel stack profiling and visualization with flame graphs with examples.

    Stack profiling and flame graphs are very useful techniques and tools for troubleshooting and investigating workloads at the OS-level and understand which code path take most of the execution time. You can find extensive material and examples o flame graphs in Brendan Gregg's website and blog. A few additional examples of using stack tracing and flame graphs for (more...)

    Add Color to your SQL

    Topic: this post is about some simple techniques to add color to SQL scripts and their terminal output using ANSI escape codes.

    Colors can be used to improve the output of command line tools. This is common practice, for example with the (bash) shell. Colors can also be very useful to improve the quality of the output of SQL scripts. In my experience this is not used frequently, probably because of the the need of (more...)

    Diagnose High-Latency I/O Operations using SystemTap

    Topic: this post is about some simple tools and techniques that can be used to drill down high-latency I/O events using SystemTap probes.

    The problem: Operations with high latency on a filesystem and/or a storage volume can sometimes be attributed to just a few disks 'misbehaving', possibly because they are suffering mechanical failures and/or are going to break completely in the near future.
    I/Os of high latency on just a few disks can then appear (more...)

    Heat Map Visualization of Latency Histograms for NetApp C-Mode

    Topic: This post is about collecting and visualizing I/O latency histograms for NetApp filers in C-mode.

    Motivations: The drill down of I/O latency is an important technique for troubleshooting and benchmarking storage. Average latency values can hide details of what is happening on the storage. Think for example of storage systems with flash and spindles, each serving I/O at different latency. Moreover averaging the measured values over time can hide details in case of varying (more...)

    Event Histogram Metric and Oracle 12c

    Topic: event histogram metric, a script and some related comments on collecting and displaying wait event latency histograms for Oracle performance troubleshooting.

    Why: Latency histograms (and by extension wait event histograms) provide very useful information when troubleshooting performance for systems exhibiting response time with multi-mode distribution. In such cases average wait values are often not sufficient to understand the behavior of the system under study and histograms provide a finer level of details. A (more...)

    Heat Map Visualization for SystemTap

    Topic: PyLatencyMap v1.2 and how it can be used to produce heat map visualization of SystemTap histograms.

    Introduction: When studying storage performance, the latency drill down is a very important data source. Measuring the average I/O latency is often not enough, latency histograms are proven to more suitable for investigating modern storage systems. This is because for many storage systems the response time has multiple modes: think for example of the common case of storage (more...)

    Latest updates to PerfSheet4, a tool for Oracle AWR data mining and visualization

    Topic: This post is about the latest updates to PerfSheet4 v3.7 (February 2015). PerfSheet4 is a tool aimed at DBAs and Oracle performance analysts. It provides a simplified interface to extract and visualize AWR time series data using Excel pivot charts.


    Why PerfSheet4: PerfSheet4 is aimed at querying and displaying time-series data from AWR repository tables. This is very a rich source of information to analyze database workloads and trends in the context of performance analysis or (more...)

    Talks at UKOUG TECH 2014 with CERN speakers

    This is a short blog post with a list of the talks at the UKOUG TECH 2014 conference in Liverpool with CERN speakers. Come and say hi, we hope to see many of you there!



    (*) slides can be downloaded from: http://cern.ch/canali

    Life of an Oracle I/O: tracing logical and physical I/O with systemtap

    Topic: This post is about tracing logical and physical reads in Oracle using systemtap. You will find a few examples illustrating the main mechanisms used by Oracle to do physical and logical I/O and you will learn how to build systemtap scripts to further explore and troubleshoot Oracle I/O.

    Introduction: Performing input/output to and from the storage media, i.e. physical I/O, is one of the critical tasks underlying all database engines. When accessing data (more...)

    SystemTap into Oracle for Fun and Profit

    Topic: This post is about using SystemTap for investigating and troubleshooting Oracle RDBMS. In particular you will learn how to probe Oracle processes and their userspace functions. These techniques aim to be useful as well as fun to learn for those keen into peeking under the hood of the technology and improve their effectiveness in troubleshooting and performance investigations.


    Introduction


    Userspace probing is a very powerful technique that can be used to complement the available (more...)