OraNA :: Oracle News Aggregator

ODTUG Kaleidoscope 2010, June 27 - July 1st Washington DC is a great conference for Oracle developers and architects, offering the best content by renowned experts. We will have the privilege of delivering two Optimizer sessions this year, 'Explaining the Explain plan' and 'Finally Plan Stability during Database Upgrade with SQL Plan Management'. In the Explain the Explain plan session we will discuss each aspect of an execution plan (from selectivity to parallel execution), explain what information you should be getting from the plan, and how it affects the execution. While in the SQL Plan Management session, we will give detailed instructions on how to capture your existing execution plans before you upgrade to 11g, as well as an in-depth discussion on what to expect from the Optimizer after you upgrade to 11g.
ODTUG is a great conference where you can learn lots in a fun and casual atmosphere. Looking forward to seeing some of you there!

Here's another question that was submitted during the OpenWorld Optimizer Roundtable. It's a common question that we've discussed a little bit in a couple other posts, but we wanted to summarize everything in one place.

First, let’s quickly review what the different values for the cursor_sharing parameter mean. We discussed this behavior in some detail in an earlier post about cursor_sharing. Below is a summary of the behavior of the different values in different cases (copied from the earlier post):

CURSOR_SHARING VALUE	SPACE USED IN SHARED POOL	QUERY PERFORMANCE
EXACT (No literal replacement)	Worst possible case - each stmt issued has its own parent cursor	Best possible case as each stmt has its own plan generated for it based on the value of the literal value present in the stmt
FORCE	Best possible case as only one parent and child cursor for each distinct stmt	Potentially the worst case as only one plan will be used for each distinct stmt and all occurrences of that stmt will use that plan
SIMILAR without histogram present	Best possible case as only one parent and child cursor for each distinct stmt	Potentially the worst case as only one plan will be used for each distinct stmt and all occurrences of that stmt will use that plan
SIMILAR with histogram present	Not quite as much space used as with EXACT but close. Instead of each stmt having its own parent cursor they will have their own child cursor (which uses less space)	Best possible case as each stmt has its own plan generated for it based on the value of the literal value present in the stmt

Adaptive cursor sharing (ACS) is another feature we've blogged about before, which allows the optimizer to generate a set of plans that are optimal for different sets of bind values. A common question is how the two interact, and whether users should consider changing the value of cursor_sharing when upgrading to 11g to take advantage of ACS. The simplest way to think about the interaction between the two features for a given query is to first consider whether literal replacement will take place for a query. Consider a query containing a literal:

select * from employees where job = 'Clerk'

As we see from the table above, the treatment of this query by literal replacement will depend on the value of the cursor_sharing parameter and whether there is a histogram on the job column. Here are the interesting cases:

1. Cursor_sharing = exact. No literal replacement will take place, and the optimizer will see the query as is.


2. Cursor_sharing = force. Whether there is a histogram or not, literal replacement will take place, and the optimizer will optimize the query as if it were:
   
   select * from employees where job = :b
   
   Bind peeking will take place, so that the value "Clerk" is used to generate cardinality estimates for the query. Subsequent executions of this query differing only in the literal value will share the same plan.
   


3. Cursor_sharing = similar. There are two different cases for this:
   
   
   
   1. There is a histogram on the job column. In this case, literal replacement will not take place. The presence of a histogram indicates that the column is skewed, and the optimal plan may depend on the literal value. Hence, the optimizer sees the query as:
      
      select * from employees where job = 'Clerk'
      
      and subsequent executions with a different literal will not necessarily use the same plan.
   
   
   2. There is no histogram on the job column. This indicates that the column is not skewed, and the optimizer is likely to choose the same plan no matter the literal, so literal replacement takes place.
   
   

Now that we know when literal replacement will take place, and what the query looks like to the optimizer, we can consider adaptive cursor sharing. If literal replacement takes place, and the query that the optimizer optimizes contains a bind, then adaptive cursor sharing can take place. To adaptive cursor sharing, a bind variable is a bind variable, whether it comes from the user query or is inserted by literal replacement. On the other hand, if the query contains only literals (no binds), adaptive cursor sharing will not take place. In our example above, adaptive cursor sharing can be considered for cases 2 and 3.2. For case 3.2, it is likely that the optimizer will choose the same plan for different values of the literal. In case 2, if there is a histogram, then the optimizer may choose different plans depending on how popular the literal value is.

This example shows that if you use histograms, and want the optimizer to choose an optimal plan for different literal values using ACS, then you should set cursor_sharing to force. If it is set to similar, then literal replacement will not take place, and a child cursor will be created for each value of the literal. Setting cursor_sharing to similar effectively disables ACS for these kinds of queries. By setting cursor_sharing to force and letting adaptive cursor sharing kick in, the optimizer can choose optimal plans for different values, but if the same plan is appropriate for several values, they will share a single child cursor. Historically, cursor_sharing=similar has been recommended as a middle ground between no literal replacement (which causes a lot of cursors to be generated) and forced literal replacement (which causes a potentially sub-optimal plan to be shared for all literals). We now recommend using adaptive cursor sharing along with cursor_sharing=force instead.

So far we have only discussed cursor sharing in the presence of histograms. There are other cases where the optimizer's choice of plan can depend on the specific literal that appears in the query, for instance when binds appear in range predicates or when a bind value falls outside of a column's range (according to the optimizer statistics). Binds appearing in these kinds of predicates are also considered by adaptive cursor sharing, whereas they are not considered by cursor_sharing=similar.

(11gR2 only) Slice and dice physical machines into small, much manageable, much performant pieces, is only one side behind virtualization. The other is: storage virtualization. In Oracle VM you can either add a virtual disk (dedicated to that VM) or a shared disk (that you can share across VMs). After the virtual disks are visible to the [...]

At the first meeting of the newly-reconstituted Vancouver Oracle Users Group this past week, we were treated to three great presentations by Caleb Small and Dan Morgan. They've made the content of their presentations available on the VanOUG web site (these links go to PDFs, if that sort of thing bugs you, consider yourself warned):

• 11gR2 High Availability Best Practices (Caleb Small)
• 11g New Features You Won't Hear About From Oracle (Dan Morgan)
• Edition Based Redefinition Presentation Slides (Dan Morgan)

I'm not going to go into a full recap of the presentations, but they were all full of really cool information. This post is an attempt to collect some of my mental notes, mostly cast in the context of one of my favorite topics, Oracle Applications.

11gR2 HA Best Practices

Caleb's presentation was very thorough and well-constructed. Dan gave him grief for boring the audience, but I think there was just so much new content to absorb that people were too busy processing to ask many questions on the fly. Here are some one-liners from my notebook (anything that looks like an opinion is my commentary/interpretation, not Caleb's):

• Lots more "moving parts" in 11gR2 Grid Infrastructure, clear "separation of duties" across three privileged OS accounts.
• Service startup order is a little different now
• Cluster status utilities show a lot more information, but need to learn to not rely upon crsstat as much
• Proper networking configuration of 11gR2 GI not for the faint of heart.
• Increased memory requirements will make this tougher to virtualize; I'm going to need a bigger laptop.
• ACFS looks interesting; I wonder if it will be a valid option for an (shared application tier filesystem) for Oracle Applications. (Turns out the answer is "not currently planned," based on this exchange I had w/ Steven Chan on his blog later in the week).

11g New Features

There's a lot of really neat stuff going on in this presentation. I'd like to call out small nugget that, while far from the most important, is still pretty interesting on the surface: "deferred segment creation." When a table is created, no extents are actually allocated until rows are inserted. Seems like an odd feature, but one touted benefit is for large ERP systems like SAP and Oracle Applications, where lots of tables are created that may never be used, depending on what products are implemented. Those thousands of initial extents can certainly add up to real storage, and a more cluttered data dictionary. I can't speak to SAP implementations, but I don't see it as a huge win for EBS customers, given that:

1. This feature is available only when tables are created, which means the benefit will only really be available when Oracle starts shipping Oracle Applications install media with an 11gR2 database. Anyone upgrading to the 11gR2 database will still be stuck with those empty extents.
2. Given the overall footprint of an EBS database, the storage savings isn't such a big deal. For example, here's the potential savings from eliminating "empty" tables from an R12 Vision database:

     SYSTEM@R12VIS(11.1.0.7)>select sum(bytes)/1024/1024 potential_savings
   2  from dba_segments s
   3  where exists (select table_name
   4                  from dba_tables
   5                 where num_rows = 0
   6                   and table_name = s.segment_name
   7               )
   8  /
   

   POTENTIAL_SAVINGS
   -----------------
   3850.36719
   

   3.5(ish) GB out of 200GB is okay, I guess, but not a huge deal for a system that's only going to keep growing. FWIW, I'm going to wave my hands and pretend that the fact that a Vision database has way more populated tables than a "fresh-install" EBS database is balanced by the fact that my quick query doesn't account for the possibility that table stats are stale and some of those tables are actually populated.

Of course, it's possible that I'm missing the point. It wouldn't be the first time! Maybe it really comes down more to a less-cluttered data dictionary. I mean, it can't be about tablespace fragmentation, since we're not supposed to care about that anymore, right?

Edition Based Redefinition (EBR)

This seemed like an interesting feature when I first heard about it last autumn, but I'll confess that I didn't quite comprehend the power of EBR until seeing Dan's demo (parts 1, 2, and 3 are on his Morgan's Library site, with part 4 still in the works). Setting aside the obvious benefits for home-grown applications, the potential benefits in an Oracle Applications environment are huge. Consider:

1. There's already an option to create a staged Applications System to shorten patch downtime windows, allowing administrators to run the "copy" and "generate" portions of large EBS patches prior to applying the patch to production. With EBR, it could be possible to stage the "database" portion of a patch as well, and switch to a new default edition at patch time. You'd probably still want to do the database staging at a quiet time in the database, of course, but daring souls could accomplish "almost-no-downtime" patching if EBR were worked into the Oracle Applications patching framework. Wicked.
2. EBR might even make it possible to truly have EBS patches that could be rolled back. The current patching process already backs up files that are replaced. Thoughtful application of cross-edition triggers might make it possible to revert to a previous edition without loss of data if a patch needs to be backed out. Granted, the process would have to be demonstrated to be pretty bullet-proof before I'd try it in production, but it could save restoring test and dev systems from backup in the event that a patch doesn't work out as expected.

Just as I might be missing the point about deferred segment creation's advantages in EBS, I might be guilty of over-extended enthusiasm with respect to edition based redefinition. Or maybe I've decided to turn this into a science fiction blog. It's sure to be far more complicated to implement EBR in an Oracle Applications context than I'm implying above, and this is only speculation on my part, not anything that's actually promised by Oracle. Still, a nerd can dream...

Thanks again to Caleb and Dan for the great presentations, and for your continued support in getting the user group launched!

I found this nice post from Laurent Schneider the other day and wanted  to comment, but my comments were a bit more of a digression and discussion, so I’ve blogged it and put a link on the comments of the post by Laurent.

I’d always used the ROW_NUMBER method myself until I read this and then figured I’d try the KEEP DENSE_RANK method, which works, as Laurent describes. One thing that didn’t sit well with me in the post from Laurent was that he said “the second one should be more performant” – I prefer hard facts, so I decided to test it a bit and my results are below.

In the simple example that Laurent gave, it’s difficult to tell which is quickest, since the table in question only has a handful of rows and therefore any benchmarking is more susceptible to other influences, clouding the results. I figured I’d build a larger table and try it on that.

Before I did that though, I did get the plans from the two statements Laurent ran on the EMP table and both show the same resource costings:

Firstly, for the ROW_NUMBER method:

select ename
,      deptno
,      sal
from   (select ename
 ,      deptno
 ,      sal
 ,      row_number() over (partition by deptno order by sal desc,empno) r
 from   emp
 )
where  r=1;

Plan hash value: 3291446077                                                                                                             

---------------------------------------------------------------------------------
| Id  | Operation                | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------
|   0 | SELECT STATEMENT         |      |    14 |   644 |     4  (25)| 00:00:01 |
|*  1 |  VIEW                    |      |    14 |   644 |     4  (25)| 00:00:01 |
|*  2 |   WINDOW SORT PUSHED RANK|      |    14 |   644 |     4  (25)| 00:00:01 |
|   3 |    TABLE ACCESS FULL     | EMP  |    14 |   644 |     3   (0)| 00:00:01 |
---------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

 1 - filter("R"=1)
 2 - filter(ROW_NUMBER() OVER ( PARTITION BY "DEPTNO" ORDER BY
 INTERNAL_FUNCTION("SAL") DESC ,"EMPNO")<=1)

Note
-----
 - dynamic sampling used for this statement

Now, the KEEP DENSE_RANK method:

select max(ename) keep (dense_rank first order by sal desc,empno) ename
,      deptno
,      max(sal) sal
from   emp
group by deptno;

Plan hash value: 15469362

---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |    14 |   644 |     4  (25)| 00:00:01 |
|   1 |  SORT GROUP BY     |      |    14 |   644 |     4  (25)| 00:00:01 |
|   2 |   TABLE ACCESS FULL| EMP  |    14 |   644 |     3   (0)| 00:00:01 |
---------------------------------------------------------------------------

Note
-----
   - dynamic sampling used for this statement

So, the plans are very similar, especially in terms of having the same resource usage…which means they should be similar in terms of performance…running them, as Laurent did, shows around 1s response times, which, as I say, doesn’t conclusively prove which method is quickest.

OK, on to a bigger example then…

I basically created a similar table to EMP, called JEFF_EMP and added a few more columns (for later) and then put ten million rows in it, taking around 1.3GB on my system…plenty to churn through.

DROP TABLE jeff_emp PURGE
/
CREATE TABLE jeff_emp(deptno     NUMBER
                     ,ename      VARCHAR2(100)
                     ,first_name VARCHAR2(50)
                     ,initials   VARCHAR2(30)
                     ,surname    VARCHAR2(50)
                     ,sal        NUMBER
                     ,empno      NUMBER
                     )
/
INSERT INTO jeff_emp(deptno,ename,first_name,initials,surname,sal,empno)
SELECT (MOD(ROWNUM,3) + 1) * 10
,      'FIRSTNAME_'||TO_CHAR(ROWNUM)||'_INITIALS_'||TO_CHAR(ROWNUM)||'_SURNAME_'||TO_CHAR(ROWNUM)
,      'FIRSTNAME_'||TO_CHAR(ROWNUM)
,      'INITIALS_'||TO_CHAR(ROWNUM)
,      'SURNAME_'||TO_CHAR(ROWNUM)
,      ROWNUM * 100
,      ROWNUM
FROM   (SELECT LEVEL l FROM dual CONNECT BY LEVEL < 10000001)
ORDER BY l
/
COMMIT
/
EXEC dbms_stats.gather_table_stats(ownname => USER, tabname => 'JEFF_EMP',estimate_percent=>10);

Now, here is the plan for the ROW_NUMBER method:

---------------------------------------------------------------------------------------------
| Id  | Operation                | Name     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT         |          |    10M|   868M|       |   204K  (1)| 00:40:49 |
|*  1 |  VIEW                    |          |    10M|   868M|       |   204K  (1)| 00:40:49 |
|*  2 |   WINDOW SORT PUSHED RANK|          |    10M|   629M|  1533M|   204K  (1)| 00:40:49 |
|   3 |    TABLE ACCESS FULL     | JEFF_EMP |    10M|   629M|       | 46605   (1)| 00:09:20 |
---------------------------------------------------------------------------------------------

…and the results:

ENAME                                                                                                    DEPTNO        SAL
---------------------------------------------------------------------------------------------------- ---------- ----------
FIRSTNAME_9999999_INITIALS_9999999_SURNAME_9999999                                                           10  999999900
FIRSTNAME_10000000_INITIALS_10000000_SURNAME_10000000                                                        20 1000000000
FIRSTNAME_9999998_INITIALS_9999998_SURNAME_9999998                                                           30  999999800

Elapsed: 00:00:24.47

…and the KEEP DENSE_RANK method plan:

-------------------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     3 |   198 | 47109   (2)| 00:09:26 |
|   1 |  SORT GROUP BY     |          |     3 |   198 | 47109   (2)| 00:09:26 |
|   2 |   TABLE ACCESS FULL| JEFF_EMP |    10M|   629M| 46605   (1)| 00:09:20 |
-------------------------------------------------------------------------------

…and it’s results:

ENAME                                                                                                    DEPTNO        SAL
---------------------------------------------------------------------------------------------------- ---------- ----------
FIRSTNAME_9999999_INITIALS_9999999_SURNAME_9999999                                                           10  999999900
FIRSTNAME_10000000_INITIALS_10000000_SURNAME_10000000                                                        20 1000000000
FIRSTNAME_9999998_INITIALS_9999998_SURNAME_9999998                                                           30  999999800

Elapsed: 00:00:07.76

So, reasonably clear results, indicating that the KEEP DENSE_RANK is about a third of the time to run, compared to the ROW_NUMBER method. You can also see from the plans that the ROW_NUMBER method involves use of TEMP, whereas the KEEP DENSE_RANK doesn’t, hence the slowdown.

So, Laurent was correct in his assertion that it should be more performant…but it’s nice to see the results based on a more meaningful set of data.

Now, there was one other thing that concerned me, and that was whether if you added more columns into the SQL, would it change the performance fo either method to any significant degree, so I started using the extra name columns like this:

SELECT ename
,      first_name
,      initials
,      surname
,      deptno
,      sal
FROM   (SELECT ename
        ,      first_name
        ,      initials
        ,      surname
        ,      deptno
        ,      sal
        ,      ROW_NUMBER() OVER (PARTITION BY deptno ORDER BY sal DESC,empno) r
        FROM   jeff_emp
       )
WHERE  r = 1
/

…which has a plan of:

---------------------------------------------------------------------------------------------
| Id  | Operation                | Name     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT         |          |    10M|  1546M|       |   307K  (1)| 01:01:36 |
|*  1 |  VIEW                    |          |    10M|  1546M|       |   307K  (1)| 01:01:36 |
|*  2 |   WINDOW SORT PUSHED RANK|          |    10M|  1107M|  2606M|   307K  (1)| 01:01:36 |
|   3 |    TABLE ACCESS FULL     | JEFF_EMP |    10M|  1107M|       | 46605   (1)| 00:09:20 |
---------------------------------------------------------------------------------------------

…and results:

ENAME                                                                                                FIRST_NAME                                         INITIALS                       SURNAME                      DEPTNO         SAL
---------------------------------------------------------------------------------------------------- -------------------------------------------------- ------------------------------ -------------------------------------------------- ---------- ----------
FIRSTNAME_9999999_INITIALS_9999999_SURNAME_9999999                                                   FIRSTNAME_9999999                                  INITIALS_9999999               SURNAME_9999999          10  999999900
FIRSTNAME_10000000_INITIALS_10000000_SURNAME_10000000                                                FIRSTNAME_10000000                                 INITIALS_10000000              SURNAME_10000000         20 1000000000
FIRSTNAME_9999998_INITIALS_9999998_SURNAME_9999998                                                   FIRSTNAME_9999998                                  INITIALS_9999998               SURNAME_9999998          30  999999800

Elapsed: 00:00:25.76

For the KEEP DENSE_RANK I get:

SELECT MAX(ename) KEEP (DENSE_RANK FIRST ORDER BY sal DESC,empno) ename
,      MAX(first_name) KEEP (DENSE_RANK FIRST ORDER BY sal DESC,empno) first_name
,      MAX(initials) KEEP (DENSE_RANK FIRST ORDER BY sal DESC,empno) initials
,      MAX(surname) KEEP (DENSE_RANK FIRST ORDER BY sal DESC,empno) surname
,      deptno
,      MAX(sal) sal
FROM   jeff_emp
GROUP BY deptno
/

Which has the following plan:

-------------------------------------------------------------------------------
| Id  | Operation          | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |          |     3 |   348 | 47109   (2)| 00:09:26 |
|   1 |  SORT GROUP BY     |          |     3 |   348 | 47109   (2)| 00:09:26 |
|   2 |   TABLE ACCESS FULL| JEFF_EMP |    10M|  1107M| 46605   (1)| 00:09:20 |
-------------------------------------------------------------------------------

…and results:

ENAME                                                                                                FIRST_NAME                                         INITIALS                       SURNAME                      DEPTNO         SAL
---------------------------------------------------------------------------------------------------- -------------------------------------------------- ------------------------------ -------------------------------------------------- ---------- ----------
FIRSTNAME_9999999_INITIALS_9999999_SURNAME_9999999                                                   FIRSTNAME_9999999                                  INITIALS_9999999               SURNAME_9999999          10  999999900
FIRSTNAME_10000000_INITIALS_10000000_SURNAME_10000000                                                FIRSTNAME_10000000                                 INITIALS_10000000              SURNAME_10000000         20 1000000000
FIRSTNAME_9999998_INITIALS_9999998_SURNAME_9999998                                                   FIRSTNAME_9999998                                  INITIALS_9999998               SURNAME_9999998          30  999999800

Elapsed: 00:00:14.56

So, the differential in performance has reduced significantly, with the KEEP DENSE_RANK around double it’s original time, whilst the ROW_NUMBER method has only increased marginally. I’ve not tested with adding additional columns, but I’m guessing (I know…I could and should test it!) it will get worse, to the extent that, eventually, the KEEP DENSE_RANK will become the worse performer. If that’s the case, then essentially, these two methods have different scalability dynamics and one should bear this in mind when considering which to choose, depending on how many DENSE_RANK’d columns you’d need to deliver your results.

Hope this helps.

In some of my previous posts (particularly the ones about analysing the data by running aggregate queries) I’ve mentioned the “sample” clause from time to time, so I thought I’d better write a short note about it for those not familiar with it. To demonstrate the feature, my first step was to run a test script [...]

This is the 182nd edition of Log Buffer, the weekly review of database blogs. Make sure to read the whole edition so you do not miss where to submit your SQL limerick!

This week started out with me posting about International Women’s Day, and has me personally attending Confoo (Montreal) which is an excellent conference I hope to return to next year. I learned a lot from confoo, especially the blending nosql and sql session I attended.

This week was also the Hotsos Symposium. Doug’s Oracle Blog has a series of posts about Hotsos. If all this talk about conferences has gotten you excited, Joshua Drake notes that 14 days and the hotel is almost full for postgresql conference east which is March 25th-28th in Philadelphia. And the Oracle database insider notes that the Oracle OpenWorld call for papers is now open.

According to Susan Visser this week (ending tomorrow) is also read an e-book week. So if you have not already done so, read an e-book! She links a coupon for an e-book in the post.

Craig Mullins notes that the mainframe is a good career choice in Mainframes: The Safe IT Career Choice. He notes that the mainframe is still not dead:

People having been predicting the death of the mainframe since the advent of client/server in the late 1980s. That is more than 20 years! Think of all the things that have died in that timespan while the mainframe keeps on chugging away: IBM’s PC business, Circuit City, Koogle peanut butter, public pay phones, Johnny Cash… the list is endless.

In other career-related news, Antonio Cangiano is looking for [2] top-notch student hackers for a 16-month internship at IBM in Toronto starting in May. All the details, including how to apply, are in Cangiano’s blog post.

Willie Favero wants to know how you “solve the batch dilemma” for issues like “shrinking your batch window, designing your batch to play nicely with … OLTP” in how’s your batch workload doing? Perhaps Favero should read the updated batch best practices posted by Anthony Shorten.

Bryan Smith surveys a more personal question by asking if you go both ways and “manage both DB2 for Linux, UNIX, and Windows and DB2 for z/OS” in don’t ask, don’t tell, bi-platform DBAs. This week’s Log Buffer editor admits to being a tri-platform DBA — she has tried many platforms, and in fact, many databases (MySQL, Oracle, DB2, SQL Server, Sybase, Postgres and Ingres)!

Hari Prasanna Srinivasan promotes a patching survey in Oracle really wants to hear from you! Patching Survey.

Henrik Loeser explains what a deadlock and a hot spot are by using a real life analogy taken from a police report in deadlock and hot spot in real life.

Jamie Thomson asks why do you abbreviate schema names?. Shlomi Noach tries to solve the issue that “there is no consistent convention as for how to write [about table aliases in] an SQL query” in proper sql table alias use conventions. Noach also gives us a tip: faster than truncate.

Leons Petrazickis reminds us that “rulesets are chains” and it is important to have your rulesets in the proper order in iptables firewall pitfall.

Anyone interested in the history of MySQL AB will be informed after reading Dries Buytaert’s article.
Gavin Towey shares his software that helps centrally manage 120 MySQL servers in qsh.pl: distributed query tool For those who want to learn more about column-oriented databases, particularly in MySQL, Robin Schumacher of the InfiniDB blog announces that there is a MySQL University session recording on MySQL column databases now available. MySQL join-fu expert Jay Pipes has moved his blog to www.joinfu.com and starts with An SQL Puzzle and of course a follow up on the sql puzzle.

Ivan Zoratti is happy that finally, slides posted for the MySQL DW breakfast. Venu Anuganti gives you tips on one of the most common MySQL frustrations: optimizing subqueries in how to improve subqueries derived tables performance. Justin Swanhart posts the way in which he Gets Linux performance information from your MySQL database without shell access and emulates a ‘top’ CPU summary using /proc/stat and MySQL using the same method.

The Oracle Apps blog has an introduction to Oracle user productivity kit (UPK). Even though in this editor’s opinion the article is very sales-pitchy, it has valuable information, and does indeed live up to its promise:

UPK is a software tool that can capture all the steps in a system process. It records every keystroke, every click of the mouse, each menu option chosen and each button pressed. All this is done in the UPK Recorder by going through the transaction and pressing “printscreen” after every user action. From this, without any further effort from the developer, UPK builds a number of valuable outputs.

Allen White gives a great tip on how to optimize queries in keep your data clean.

Mike Dietrich reminds you to remove “old” parameters and events from your init.ora when upgrading, “as keeping them will definitely slow down the database performance in the new release.” He shows evidence of slowness when this is not done. Dietrich also shows how you can be gathering workload statistics “to give the optimizer some good knowledge about how powerful your IO-system might be”, especially “a few days after upgrading to the new release…while a real workload is running.”

Brian Aker shows the exciting features coming soon in Drizzle in Drizzle, Cherry, Roadmap for our Next Release.

Maybe you are thinking of migrating, not upgrading…..The O’Reilly Radar shows how to asses an Oracle to MySQL migration in MySQL migration and risk management. Actually, that article interviews Ronald Bradford on the subject — Bradford has been prolific lately, updating free my.cnf advice series and “Don’t Assume”: MySQL for the Oracle DBA series. Nick Quarmby also talks about migrating Oracle, but not to a new database, just to a new platform, in his primer on migrating Oracle Applications to new platforms. And the big news comes from Carlos of dataprix that Twitter will migrate from MySQL to Cassandra DB.

Paul S. Randal explains his way of benchmarking: 1 Tb table population on SQL Server.

Pete Finnigan shares his slides from a webinar on how to secure oracle, and Denis Pilipchuk shares his approaches for discovering security vulnerabilities in software applications.

Jeff Davis shares his thoughts about scalability and the relational model. Robert Treat responds actually, the relational model doesn’t scale and Baron Schwartz counters with NoSQL doesn’t mean non-relational.

Buck Woody explains “whenever you want to know something about SQL Server’s configuration, whether that’s the Instance itself or a database, you have a few options” — and of course what those options are — in system variables, stored procedures or functions for meta data.

This week’s T-SQL Tuesday topic was I/O. There are many links to great blog posts in the comments; three random posts I chose to highlight: Michael Zilberstein talks about IO capacity planning, while Kalen Delaney talks about using STATISTICS IO in I/O, you know, and Merrill Aldrich chimes in with information on real world SSD’s. Aldrich also begs folks not to waste resources and make more work for developers and DBAs in dear ISV, you’re keeping me awake nights with your VARCHAR() dates.

And we end with a bit of fin: Paul Nielsen wants us all to have a bit of fun; he has posted an SQL limerick and asks readers to create there own in there once was in Dublin a query.

Said farewell today to the guys I met in the lobby of the Omni Mandelay Hotel in Irving, Texas. The final bits of the Hotsos 2010 Symposium. It has been a good run.

By the way, what did you mean Doug, with “a big laptop”, regarding your new Sony VAIO ???

It isn’t that big…

If you ever experience these symptoms, ask your self whether you have Grid Control (or OEM) running.Single Sign On failsOID Processes suddenly have stopped (which explains the above)ODS database account is locked, which seems to cause the problemUnlocking the ODS account resolves the problem, but it gets locked after a short period.The cause is the fact you changed the ODS password. Now, you may

March 12, 2010

I thought that we would try something a bit more difficult today.  In previous articles I showed how to generate and read 10046 trace files using various methods, and I also showed a couple of different VBS scripts that could interact with an Oracle database.  With some effort we could even read through a 10046 to pull out bits of information, much like TKPROF, but it probably does not make much sense to reproduce what TKPROF already accomplishes.  I thought instead what I would do is to create a VBS script that generates a 10046 trace file at level 4, while executing a couple of SQL statements.  A second VBS script will read the raw 10046 trace file and convert that trace file back into a VBS script, complete with bind variables.  The code for the second VBS script is based on some of the code in my Toy project for performance tuning – something that I originally created just to see if it could be done.

First, we need a table to use as the data source for the first VBS script – this is the test table used in this blog article:

CREATE TABLE EMPLOYEE_RECORD_TEST AS
SELECT
  DECODE(TRUNC(DBMS_RANDOM.VALUE(0,5)),
          0,'MIKE',
          1,'ROB',
          2,'SAM',
          3,'JOE',
          4,'ERIC') EMPLOYEE_ID,
  TRUNC(SYSDATE)-ROUND(DBMS_RANDOM.VALUE(0,1000)) SHIFT_DATE,
  DECODE(TRUNC(DBMS_RANDOM.VALUE(0,10)),
          0,'VAC',
          1,'HOL',
          2,'BEREAVE',
          3,'JURY',
          4,'ABS',
          5,'EXCUSE',
          6,'MIL',
          'OTHER') INDIRECT_ID
FROM
  DUAL
CONNECT BY
  LEVEL<=1000;

With the test table built, we execute the following simple VBS script (using either CSCRIPT or WSCRIPT on a Windows client):

Const adCmdText = 1
Const adNumeric = 131
Const adDate = 7
Const adDBDate = 133
Const adDBTimeStamp = 135
Const adDBTime = 134
Const adVarChar = 200
Const adParamInput = 1

Dim strSQL
Dim strUsername
Dim strPassword
Dim strDatabase

Dim dbDatabase
Dim snpDataEmployees
Dim comDataEmployees
Dim snpDataAttend
Dim comDataAttend
Dim snpDataEmpRecord
Dim comDataEmpRecord

Set dbDatabase = CreateObject("ADODB.Connection")
Set snpDataEmployees = CreateObject("ADODB.Recordset")
Set comDataEmployees = CreateObject("ADODB.Command")
Set snpDataAttend = CreateObject("ADODB.Recordset")
Set comDataAttend = CreateObject("ADODB.Command")
Set snpDataEmpRecord = CreateObject("ADODB.Recordset")
Set comDataEmpRecord = CreateObject("ADODB.Command")

strUsername = "MyUsername"
strPassword = "MyPassword"
strDatabase = "MyDB"

dbDatabase.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
dbDatabase.Open
'Should verify that the connection attempt was successful, but I will leave that for someone else to code

dbDatabase.Execute "ALTER SESSION SET TRACEFILE_IDENTIFIER = 'VBS2TRACE2VBS'"
dbDatabase.Execute "ALTER SESSION SET EVENTS '10046 TRACE NAME CONTEXT FOREVER, LEVEL 4'"

strSQL = "INSERT INTO EMPLOYEE_RECORD_TEST(" & VBCrLf
strSQL = strSQL & "  EMPLOYEE_ID," & VBCrLf
strSQL = strSQL & "  SHIFT_DATE," & VBCrLf
strSQL = strSQL & "  INDIRECT_ID)" & VBCrLf
strSQL = strSQL & "VALUES(" & VBCrLf
strSQL = strSQL & "  ?," & VBCrLf
strSQL = strSQL & "  ?," & VBCrLf
strSQL = strSQL & "  ?)"

With comDataEmpRecord
    'Set up the command properties
    .CommandText = strSQL
    .CommandType = adCmdText
    .CommandTimeout = 30
    .ActiveConnection = dbDatabase

    .Parameters.Append .CreateParameter("employee_id", adVarChar, adParamInput, 15, "TEST")
    .Parameters.Append .CreateParameter("shift_date", adDate, adParamInput, 8, Date)
    .Parameters.Append .CreateParameter("indirect_id", adVarchar, adParamInput, 15, "HOL")
End With

'Rollback Test
dbDatabase.BeginTrans

comDataEmpRecord.Execute

dbDatabase.RollbackTrans

strSQL = "SELECT DISTINCT" & vbCrLf
strSQL = strSQL & "  EMPLOYEE_ID" & vbCrLf
strSQL = strSQL & "FROM" & vbCrLf
strSQL = strSQL & "  EMPLOYEE_RECORD_TEST" & vbCrLf
strSQL = strSQL & "WHERE" & vbCrLf
strSQL = strSQL & "  SHIFT_DATE>= ?" & vbCrLf
strSQL = strSQL & "  AND INDIRECT_ID= ?" & vbCrLf
strSQL = strSQL & "ORDER BY" & vbCrLf
strSQL = strSQL & "  EMPLOYEE_ID"

With comDataEmployees
    'Set up the command properties
    .CommandText = strSQL
    .CommandType = adCmdText
    .CommandTimeout = 30
    .ActiveConnection = dbDatabase

    .Parameters.Append .CreateParameter("shift_date", adDate, adParamInput, 8, DateAdd("d", -90, Date))
    .Parameters.Append .CreateParameter("indirect_id", adVarChar, adParamInput, 15, "VAC")
End With

strSQL = "SELECT" & vbCrLf
strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'2',1,0)) MON_COUNT," & vbCrLf
strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'3',1,0)) TUE_COUNT," & vbCrLf
strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'4',1,0)) WED_COUNT," & vbCrLf
strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'5',1,0)) THU_COUNT," & vbCrLf
strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'6',1,0)) FRI_COUNT" & vbCrLf
strSQL = strSQL & "FROM" & vbCrLf
strSQL = strSQL & "  EMPLOYEE_RECORD_TEST" & vbCrLf
strSQL = strSQL & "WHERE" & vbCrLf
strSQL = strSQL & "  EMPLOYEE_ID= ?" & vbCrLf
strSQL = strSQL & "  AND INDIRECT_ID= ?"

With comDataAttend
    'Set up the command properties
    .CommandText = strSQL
    .CommandType = adCmdText
    .CommandTimeout = 30
    .ActiveConnection = dbDatabase

    .Parameters.Append .CreateParameter("employee_id", adVarChar, adParamInput, 15, "HOOPER")
    .Parameters.Append .CreateParameter("indirect_id", adVarChar, adParamInput, 15, "EXCUSE")
End With

Set snpDataEmployees = comDataEmployees.Execute

If Not (snpDataEmployees Is Nothing) Then
    Do While Not snpDataEmployees.EOF
        comDataAttend("employee_id") = snpDataEmployees("employee_id")
        comDataAttend("indirect_id") = "EXCUSE"
        Set snpDataAttend = comDataAttend.Execute
        If Not snpDataAttend.EOF Then
            'Do Something with the data
        End If
        snpDataAttend.Close

        comDataAttend("indirect_id") = "ABS"
        Set snpDataAttend = comDataAttend.Execute
        If Not snpDataAttend.EOF Then
            'Do Something with the data
        End If
        snpDataAttend.Close

        snpDataEmployees.MoveNext
    Loop

    snpDataEmployees.Close
End If

dbDatabase.Close
Set snpDataEmployees = Nothing
Set snpDataAttend = Nothing
Set comDataEmployees = Nothing
Set comDataAttend = Nothing
Set snpDataEmpRecord = Nothing
Set comDataEmpRecord = Nothing
Set dbDatabase = Nothing

(TestScript.vbs - save as TestScript.vbs)

In the above, replace MyDB with a valid database name from the tnsnames.ora file, MyUsername with a valid username, and MyPassword with the password for the user.  The script starts by starting a transaction (the default behavior is an implicit commit), a row is inserted into the test table, and then a ROLLBACK is performed.  The script then submits a SQL statement that retrieves a list of 5 employees from the test table.  For each of the (up to) 5 employees a second SQL statement is executed with two different bind variable sets to determine the number of each week day the employee has been out of work on an excused (EXCUSE) or unexcused (ABS) absence.  If I were writing a real program to accomplish this task I would combine the three SELECT statements into a single SELECT statement, but I want to demonstrate how the second VBS script handles multiple SQL statements that are open at the same time.

Running the above script generated a trace file when executed against Oracle Database 11.2.0.1: or112_ora_5482_VBS2TRACE2VBS.trc  (save as C:\or112_ora_5482_VBS2TRACE2VBS.trc – Windows users can view the file with Wordpad and convert the file into a plain text file that can be opened with Notepad).  The goal is to take the trace file and transform it back into a VBS script, ignoring SQL statements that appear in the trace file at a depth greater than 0.

The output of the VBS script that reads the 10046 trace file and generates a VBS file should look something like this:

'Source File:C:\or112_ora_5482_VBS2TRACE2VBS.trc

HyperactiveTrace

Sub HyperactiveTrace()
    Const adCmdText = 1
    Const adCmdStoredProc = 4
    Const adParamInput = 1
    Const adVarNumeric = 139
    Const adBigInt = 20
    Const adDecimal = 14
    Const adDouble = 5
    Const adInteger = 3
    Const adLongVarBinary = 205
    Const adNumeric = 131
    Const adSingle = 4
    Const adSmallInt = 2
    Const adTinyInt = 16
    Const adUnsignedBigInt = 21
    Const adUnsignedInt = 19
    Const adUnsignedSmallInt = 18
    Const adUnsignedTinyInt = 17
    Const adDate = 7
    Const adDBDate = 133
    Const adDBTimeStamp = 135
    Const adDBTime = 134
    Const adVarChar = 200
    Const adChar = 129
    Const adUseClient = 3

    Dim i
    Dim strSQL
    Dim strUsername
    Dim strPassword
    Dim strDatabase

    Dim dbDatabase
    Set dbDatabase = CreateObject("ADODB.Connection")
    Dim snpData1
    Dim comData1
    Set snpData1 = CreateObject("ADODB.Recordset")
    Set comData1 = CreateObject("ADODB.Command")
    Dim snpData2
    Dim comData2
    Set snpData2 = CreateObject("ADODB.Recordset")
    Set comData2 = CreateObject("ADODB.Command")
    Dim snpData3
    Dim comData3
    Set snpData3 = CreateObject("ADODB.Recordset")
    Set comData3 = CreateObject("ADODB.Command")
    Dim snpData4
    Dim comData4
    Set snpData4 = CreateObject("ADODB.Recordset")
    Set comData4 = CreateObject("ADODB.Command")
    Dim snpData5
    Dim comData5
    Set snpData5 = CreateObject("ADODB.Recordset")
    Set comData5 = CreateObject("ADODB.Command")
    Dim snpData6
    Dim comData6
    Set snpData6 = CreateObject("ADODB.Recordset")
    Set comData6 = CreateObject("ADODB.Command")
    Dim snpData7
    Dim comData7
    Set snpData7 = CreateObject("ADODB.Recordset")
    Set comData7 = CreateObject("ADODB.Command")
    Dim snpData8
    Dim comData8
    Set snpData8 = CreateObject("ADODB.Recordset")
    Set comData8 = CreateObject("ADODB.Command")
    Dim snpData9
    Dim comData9
    Set snpData9 = CreateObject("ADODB.Recordset")
    Set comData9 = CreateObject("ADODB.Command")
    Dim snpData10
    Dim comData10
    Set snpData10 = CreateObject("ADODB.Recordset")
    Set comData10 = CreateObject("ADODB.Command")
    Dim snpData11
    Dim comData11
    Set snpData11 = CreateObject("ADODB.Recordset")
    Set comData11 = CreateObject("ADODB.Command")
    Dim snpData12
    Dim comData12
    Set snpData12 = CreateObject("ADODB.Recordset")
    Set comData12 = CreateObject("ADODB.Command")
    Dim snpData13
    Dim comData13
    Set snpData13 = CreateObject("ADODB.Recordset")
    Set comData13 = CreateObject("ADODB.Command")
    Dim snpData14
    Dim comData14
    Set snpData14 = CreateObject("ADODB.Recordset")
    Set comData14 = CreateObject("ADODB.Command")
    Dim snpData15
    Dim comData15
    Set snpData15 = CreateObject("ADODB.Recordset")
    Set comData15 = CreateObject("ADODB.Command")
    Dim snpData16
    Dim comData16
    Set snpData16 = CreateObject("ADODB.Recordset")
    Set comData16 = CreateObject("ADODB.Command")
    Dim snpData17
    Dim comData17
    Set snpData17 = CreateObject("ADODB.Recordset")
    Set comData17 = CreateObject("ADODB.Command")
    Dim snpData18
    Dim comData18
    Set snpData18 = CreateObject("ADODB.Recordset")
    Set comData18 = CreateObject("ADODB.Command")
    Dim snpData19
    Dim comData19
    Set snpData19 = CreateObject("ADODB.Recordset")
    Set comData19 = CreateObject("ADODB.Command")
    Dim snpData20
    Dim comData20
    Set snpData20 = CreateObject("ADODB.Recordset")
    Set comData20 = CreateObject("ADODB.Command")

    On Error Resume Next

    strUsername = "MyUsername"
    strPassword = "MyPassword"
    strDatabase = "MyDB"
    dbDatabase.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbDatabase.Open
    'Should verify that the connection attempt was successful, but I will leave that for someone else to code

    'dbDatabase.BeginTrans

    'Transaction Committed and NO Records were Affected, Need to determine transaction start
    dbDatabase.CommitTrans

    'dbDatabase.BeginTrans

    Set comData3 = CreateObject("ADODB.Command")

    strSQL = "INSERT INTO EMPLOYEE_RECORD_TEST(" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_ID," & vbCrLf
    strSQL = strSQL & "  SHIFT_DATE," & vbCrLf
    strSQL = strSQL & "  INDIRECT_ID)" & vbCrLf
    strSQL = strSQL & "VALUES(" & vbCrLf
    strSQL = strSQL & "  ?," & vbCrLf
    strSQL = strSQL & "  ?," & vbCrLf
    strSQL = strSQL & "  ?)"

    With comData3
        'Set up the command properties
        .CommandText = strSQL
        .CommandType = adCmdText
        .CommandTimeout = 30
        .ActiveConnection = dbDatabase
        'Bind variables will be defined below, if there are no bind variable, uncomment the next line and add the looping construct
    End With
    'comData3.Execute

    With comData3
        .Parameters.Append .CreateParameter("B1", adChar, adParamInput, 4, "TEST")
        .Parameters.Append .CreateParameter("B2", adDate, adParamInput, 7, "3/11/2010 0:0:0")
        .Parameters.Append .CreateParameter("B3", adChar, adParamInput, 3, "HOL")
    End With

    comData3("B1") = "TEST"
    comData3("B2") = cDate("3/11/2010 0:0:0")
    comData3("B3") = "HOL"

    comData3.Execute

    'Transaction Rolled Back and Records Should have been Affected, Need to determine transaction start
    dbDatabase.RollbackTrans

    'dbDatabase.BeginTrans

    'Cursor 2 Closing
    If snpData2.State = 1 Then
        snpData2.Close
    End If
    Set comData2 = Nothing

    Set comData2 = CreateObject("ADODB.Command")

    strSQL = "SELECT DISTINCT" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_ID" & vbCrLf
    strSQL = strSQL & "FROM" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_RECORD_TEST" & vbCrLf
    strSQL = strSQL & "WHERE" & vbCrLf
    strSQL = strSQL & "  SHIFT_DATE>= ?" & vbCrLf
    strSQL = strSQL & "  AND INDIRECT_ID= ?" & vbCrLf
    strSQL = strSQL & "ORDER BY" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_ID"

    With comData2
        'Set up the command properties
        .CommandText = strSQL
        .CommandType = adCmdText
        .CommandTimeout = 30
        .ActiveConnection = dbDatabase
        'Bind variables will be defined below, if there are no bind variable, uncomment the next line and add the looping construct
    End With
    'Set snpData2 = comData2.Execute

    'Cursor 4 Closing
    If snpData4.State = 1 Then
        snpData4.Close
    End If
    Set comData4 = Nothing

    'Cursor 4 Closing
    If snpData4.State = 1 Then
        snpData4.Close
    End If
    Set comData4 = Nothing

    'Cursor 4 Closing
    If snpData4.State = 1 Then
        snpData4.Close
    End If
    Set comData4 = Nothing

    With comData2
        .Parameters.Append .CreateParameter("B1", adDate, adParamInput, 7, "12/11/2009 0:0:0")
        .Parameters.Append .CreateParameter("B2", adChar, adParamInput, 3, "VAC")
    End With

    comData2("B1") = cDate("12/11/2009 0:0:0")
    comData2("B2") = "VAC"

    Set snpData2 = comData2.Execute

    If Not (snpData2 Is Nothing) Then
        Do While Not snpData2.EOF

            snpData2.MoveNext
        Loop
    End If

    'Cursor 4 Closing
    If snpData4.State = 1 Then
        snpData4.Close
    End If
    Set comData4 = Nothing

    Set comData4 = CreateObject("ADODB.Command")

    strSQL = "SELECT" & vbCrLf
    strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'2',1,0)) MON_COUNT," & vbCrLf
    strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'3',1,0)) TUE_COUNT," & vbCrLf
    strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'4',1,0)) WED_COUNT," & vbCrLf
    strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'5',1,0)) THU_COUNT," & vbCrLf
    strSQL = strSQL & "  SUM(DECODE(TO_CHAR(SHIFT_DATE,'D'),'6',1,0)) FRI_COUNT" & vbCrLf
    strSQL = strSQL & "FROM" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_RECORD_TEST" & vbCrLf
    strSQL = strSQL & "WHERE" & vbCrLf
    strSQL = strSQL & "  EMPLOYEE_ID= ?" & vbCrLf
    strSQL = strSQL & "  AND INDIRECT_ID= ?"

    With comData4
        'Set up the command properties
        .CommandText = strSQL
        .CommandType = adCmdText
        .CommandTimeout = 30
        .ActiveConnection = dbDatabase
        'Bind variables will be defined below, if there are no bind variable, uncomment the next line and add the looping construct
    End With
    'Set snpData4 = comData4.Execute

    With comData4
        .Parameters.Append .CreateParameter("B1", adChar, adParamInput, 4, "ERIC")
        .Parameters.Append .CreateParameter("B2", adChar, adParamInput, 6, "EXCUSE")
    End With

    comData4("B1") = "ERIC"
    comData4("B2") = "EXCUSE"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    'Cursor 5 Closing
    If snpData5.State = 1 Then
        snpData5.Close
    End If
    Set comData5 = Nothing

    comData4("B1") = "ERIC"
    comData4("B2") = "ABS"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    'Cursor 5 Closing
    If snpData5.State = 1 Then
        snpData5.Close
    End If
    Set comData5 = Nothing

    comData4("B1") = "JOE"
    comData4("B2") = "EXCUSE"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    comData4("B1") = "JOE"
    comData4("B2") = "ABS"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    comData4("B1") = "MIKE"
    comData4("B2") = "EXCUSE"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    comData4("B1") = "MIKE"
    comData4("B2") = "ABS"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    comData4("B1") = "SAM"
    comData4("B2") = "EXCUSE"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    comData4("B1") = "SAM"
    comData4("B2") = "ABS"

    Set snpData4 = comData4.Execute

    If Not (snpData4 Is Nothing) Then
        Do While Not snpData4.EOF

            snpData4.MoveNext
        Loop
    End If

    'Transaction Committed and NO Records were Affected, Need to determine transaction start
    dbDatabase.CommitTrans

    'dbDatabase.BeginTrans

    'Cursor 5 Closing
    If snpData5.State = 1 Then
        snpData5.Close
    End If
    Set comData5 = Nothing

    '*************************************************************
    'Maximum Recordset Number Used is 5 - Adjust the Code at the Start Accordingly
    '*************************************************************

    If snpData1.State = 1 Then
        snpData1.Close
    End If
    Set snpData1 = Nothing

    If snpData2.State = 1 Then
        snpData2.Close
    End If
    Set snpData2 = Nothing

    If snpData3.State = 1 Then
        snpData3.Close
    End If
    Set snpData3 = Nothing

    If snpData4.State = 1 Then
        snpData4.Close
    End If
    Set snpData4 = Nothing

    If snpData5.State = 1 Then
        snpData5.Close
    End If
    Set snpData5 = Nothing

    Set comData1 = Nothing
    Set comData2 = Nothing
    Set comData3 = Nothing
    Set comData4 = Nothing
    Set comData5 = Nothing

    dbDatabase.Close
    Set dbDatabase = Nothing
End Sub

(TraceToVBSOutput.vbs – save as TraceToVBSOutput.vbs)

If you compare the original TestScript.vbs with the above output, we see that the two scripts are similar, but with a couple of distinct differences:

• It is not necessarily easy to determine when a transaction starts, but it is possible to determine when a transaction ends.  The script that reads the trace file inserts ‘dbDatabase.BeginTrans where it believes that a transaction should start – remove the ‘ if that is the correct starting point for the transaction.
• Looping structures with nested SQL statements (the retrieval of the employee list from the EMPLOYEE_RECORD_TEST table and the probing of matching rows for each of those employees) cannot be reproduced automatically – you will have to recognize when one SQL statement is feeding the bind variable values of a second SQL statement.
• The VBS script assumes that up to 20 cursors will be open at any one time, but will automatically handle many more than 20 simultaneously open cursors.  The resulting VBS file should be cleaned up to remove the unneeded comData and snpData objects.
• There are spurious snpDatan.Close statements – see the suggestions for improvements.
• SQL statements submitted without bind variables will not have code written to execute those statements in the generated VBS file - see the suggestions for improvements.
• Bind variables that are submitted as VARCHAR (adVarchar) are written to the trace file as if the bind variables were declared as CHAR (adChar) – while this does not appear to cause a problem, it might appear to be an unexpected change when comparing the test script with the automatically generated script.

Suggestions for improvement:

• Recognize the EXEC line in the 10046 trace and use that to actually indicate that a SQL statement should execute in the generated script, rather than executing the SQL statement in response to the submission of bind variables.
• Allow submitting the source trace file name and the destination (generated) VBS filename on the command line.
• Allow submitting the username, password, and database name on the command line or in a web-based user interface.
• Correct the script so that it does not attempt to close recordsets when those recordsets were never opened at dep=0 – this is caused by the script seeing a recursive SQL statement that is preparing to open with that cursor number.

The VBS script that converts 10046 trace files to VBS script files may be downloaded here: TraceToVBS.vbs (save as TraceToVBS.vbs).  There may be bugs in the script, but it should be close enough to provide some degree of educational benefit.

Related Blog Articles:
10046 Extended SQL Trace Interpretation
Automated DBMS_XPLAN, Trace, and Send to Excel
Database Inpector Gadget
Simple VBS Script to Retrieve Data from Oracle
Toy Project for Performance Tuning 2

My last day already, so I hereby leave you with some impressions from Tanel’s Training Day. Maybe until next year.

If you also want to learn from him, then here’s your chance in Holland. Scripts and tools used by Tanel can be found here.

A few years ago, my wife and I went to New York for a holiday. We got a cab from the airport into Manhattan. It was an expensive way to see, at great length, some of the more uninteresting automobile transit routes through New York. We arrived at our hotel a great deal later than we anticipated. And with a lot of our paper dollars no longer in our possession.

I’ve also taken cabs through London, usually at the weekend to get back to Liverpool Street Station. The trip is generally quick, painless and not too expensive, no matter what bit of London is currently being dug up. Those black-cab drivers know their stuff.

Of course, the CABs I refer to in the title of this Friday Philosophy are not private cars for hire. In this context CAB is Change Advisory Board. A term that can make grown developers weep. If you do not know, the Change Advisory Board is a group of people who look at the changes that are planed for a computer system and decide if they are fit for release. My personal experience of them has been similar to my experience of the taxi variety, though sadly more of the New York than London experience.

You might expect me to now sink into a diatribe {ie extended rant} about how I hate CABs. Well, I don’t. CABs can be a part of a valuable and highly worthwhile process control mechanism. Just as proper QA is core to any mature software development process, so CABs are important in getting informed, talented stakeholders to review proposed changes. They check for overall system impact, clashes with other proposed changes that individual development streams may be unaware of, use their own {hopefully deep and wide} experience to consider the changes and to verify Due Diligence has been invoked {that last one is a bit of a minefield and where, I believe, many CABs fail}.

Sadly, though this is often the aim, the end result is too often a bunch of uninformed and technically naive politicos trying to wield power, using the CAB meeting as an extended game of management chess.

I’ve seen CABs trade changes. “I’ll let you have X if I can have Y and Z”. I’ve seen CABs turn down changes because the form had spelling mistakes in it. I’ve seen CABs object to a change that will save the company 5 million pounds a day because it lacked one signature.

That last one just stopped me in my tracks {I’m not exaggerating either, if anything I am underplaying the cost impact of that decision. I saw the figures and I wasted a couple of days of my life checking, 5 million pounds a day was the least I felt I could prove.} We are talking about enough money every day to pay the salary of everyone on the CAB for several years. And they blocked it because the DBA team had not signed off the change. No effort was made to address the lack of the signature in any way, the change was just refused.

The DBA Team had not signed off the change because the one and only DBA Team Leader who was allowed to sign off was on holiday for two weeks. They needed that holiday too, for other but I suspect linked reasons.

Now, I knew the DBA Team Lead and he was a good bloke, he knew his stuff and he was not paid 5 million pounds a day. His deputy was paid even less but was no less talented but she was not allowed to sign off the change as she was not the DBA Team Lead.

That was a CAB gone very wrong. The process of the CAB had been allowed to over-rule good business sense. It was also overruling general and technical sense, but that really is secondary to what keeps the business making a profit.

I’ve seen the opposite of course, technical teams that just apply whatever changes they feel are fit, with no oversight or CAB. To be honest, this less controlled process seem to mess up less often than a poor CAB process as the technicians know they are the ones who will spend the weekend fixing a mess if one occurs. But that mess up will occur eventually, if control is lacking, and the bigger and more complex the IT environment, the greater the chance of the mess up.

So, I feel CABs are good, no make that Great, if you have the right people on them and you have a sensible cascade of authority so one person being away does not block the system. That is quite a bit harder to put in place than a simple “Dave A, John, Andrea, Alex, Raj, Dave P, Mal, Malcolm and Sarah have final signoff” which most CABs effecively become.

But there is one last fault of CABs I want to highlight. They tend to treat all changes in the same way and all changes are not the same. Upgrading the underlying OS is not the same as adding a cardinality hint to one Business Objects report.

If your CAB or change process treat the two above examples the same, then your CAB or change process is broken. Now, in all IT “rules of thumb” there is an exception. In this case, I am truly struggling to think of one. My feeling is that if your change process treats an OS upgrade the same as adding a hint to a report, it is not fit for purpose.

Those are my main issue with CABs. They should be of significant business importance, but nearly always they are implemented with one process to deal with all situations and then get taken over by people with an “Office Politics” agenda as opposed to a “Getting the best job we can reasonably expect done” agenda.

I’m very passionate about this and I have a way I hope can throw this issue into context, an analogy.

Ask yourself this senario.
You go to your doctor with a niggly cough you have had for a week OR you go to your doctor because you almost passed out each day you got out of bed for the last three days.
If your doctor treated you the same for both sets of symptoms, would you be happy with that doctor?

Why are all IT changes handled by most CABs in exactly the same way?

(BTW if you ever almost collapse when you get out of the bed in the morning, do NOT go to work, go instead to your doctor and ask them for a full medical and if he/she does not take blood pressure readings and order a full blood chemisty test, go find a new doctor.)

While applying patch 7303033 on top of a 12.1.0 R12 installation, I got the following error in one of my workers:

FNDLOAD APPS/***** 0 Y UPLOAD @FND:patch/115/import/afscursp.lct @JTF:patch/115/import/US/jtfdiagresp.ldt -

Connecting to APPS......Connected successfully.

Calling FNDLOAD function.

Returned from FNDLOAD function.

Log file: /r12u/apps/apps_st/appl/admin/UPG/log/

There she goes, squirreling into the corner of your carry-on.  Ready for another adventure.  Tucked between a fun floppy hat and the flourescent sun-block, of which she uses plenty, pale creature that she is, Monique is situated to prove to you that a seasoned traveller knows best.  Dressed in a curly mohair (so as not to show the wrinkles that betray one new to the jet set) and sporting a pale chiffon scarf - to double as a disquise in those cities where she is already known for her somewhat silly escapades - she seems to wink at you and beckon you along.  Forget your cares, but by no means your playing cards, and step lightly into whatever dreams your next voyage should hold for you:  let Monique be your guide.  

Her very presence alleviates all the maladies brought on by the jostling of trains, the dipping of ships, the swerving of buses and even by the simple fact that you are far from home.  For she has discovered the secret to successful travel; with the exception of the luscious few minutes in the X-ray machine, where she delights in making funny faces at the operators searching for more dangerous contents.  Monique never looks back.

Charleen Kinser Designs

(With thanks to Carol Dacko for bringing along an interesting dinner companion for my own cuddly friends. Somehow I doubt they'll ever be the same - she's terribly sophisticated for my lot!)

I heard someone bemoaning the lack of Swag at Hotsos - just an event program, couple of magazines and a small clockwork toy sponsored by Oracle. I don't think I ever come to the Symposium expecting Swag - it's just not that kind of event. But you do tend to get good speaker gifts, including the best speaker gift I ever received. This year, it's a digital photo frame.

As for that clockwork toy, the Cuddly Toys think it's *brilliant*!!! At first, their reaction was "What's THAT?!" then when I wound it up and let it dance in front of them, they fell in love.



The only thing is that it doesn't run for long per wind-up, so my right wrist is aching from them continually beggind me to "Do it again, Douglas!"

Then again, their reaction wasn't quite as relaxed as when they met Carol Dacko's friend while we were at the restaurant. Apologies for the quality of the photo, it was dark in there, but at least it proves that Little H and Chris were allowed out for dinner and that not all Cuddly Toys are made the same.



They were just stunned by their new friend. What's THAT?

All will be revealed in the next post ....

First up was Cary Millsap's - Lessons Learned, Version 2010.03 As Cary pointed out, they always try to put the best speakers in the toughest slots - 8:30 in the morning post-party. I think local guys are slightly more reliable too because they might have actually gone home the night before! He started with a quick Hangover Survey (me - check!) and then pressed on talking about how we test system performance.

He showed a video of Boeing stress-testing the wing of the 787 and, as he pointed out, aircraft manufacturers really know how to stress-test! (Of course whether that reassures you as it does me, or makes you wish no-one would talk about wings disintegrating, as it probably would Mads, is personal.) They showed Boeing test equpiment which is complicated, expensive and non-revenue generating. Those tests are expensive but when people's lives are on the line, what choice is there? Boeing knows that it has to test the analytic models used in the design. He spent a lot of time talking about good test design. A few thoughts that stood out to me ...

- Some stress tests are a waste of time. Will the Boeing 787 land on the moon? If this test fails, what has it proven? If it passes, then it's awesome but it would be a very expensive way to prove it can cope with commercial flights in Earth's atmosphere.

- Why test for more than you will see in Production? Because you don't really know for sure what you'll see in Production.

- At some point, but I can't remember the context, he used a Scottish phrase that he'd heard Billy Connolly shout (although the Big Yin was only fully credited later in the day) ...

    "There's no such thing as bad weather, just the wrong clothes"

... looked over at me and said - "I'd love to hear you say that, with the proper accent". I declined politely.

- Most people try to prove only that their systems will work.

- Most tests of systems that are destined to fail never proved it in advance.

- Test to destruction

    a) Test
    b) Until the system melts
    c) Decide whether your real requirements are likely to be lower or higher than melting point.

There was a small amount of time for questions and once it looked like they were done, I granted Cary's wish (never thought I'd say that), stuck my hand up and repeated The Big Yin's words. It was only after the laughter had stopped that I realised I might have ruined his big closing, but I think he was ok about it

Next was Tanel Poder talking about LGWR, log file sync waits and COMMIT performance and shock, horror, I was actually going to say that this was one of the least rewarding sessions of the week for me. What?!? Tanel? But he's, like, an Oracle God! LOL But there were reasons

- I realise that I know a *lot* about how log file sync and log file parallel write work, how they relate to each other and some of the problems they might help you identify. Because it's a subject I'm *so* familiar with, I didn't learn much.

- His main demo didn't quite show what he wanted it to because it didn't run multiple sessions but, frankly, I'm in no position to talk about demos this week!

By the end, the presentation turned out ok, not least because there was another unexpected appearance from Bob Sneed to talk about the I/O components involved in  redo log management including a suggestion that LGWR be put into a higher scheduling class (but not Real Time!) I'll try to find a link to his slides and let you take a look yourself.

I loved Tanel's Big Log File Sync Tuning Secret, though ...

    COMMIT LESS!

It was particularly relevant to me because I had a Big Log File Sync Tuning Secret as the closing moment of my own presentation. The problem was I couldn't use it after the demos went wrong!

    USE ASYNCHRONOUS COMMITS

But, in my case, that was supposed to be funny, too.

I ran off to try and use the free breakfast voucher that Marco had given me but I was just too late. No food again, then Well, I had a couple of slices of cold meat at lunchtime, but mainly to catch up with Alex G before he had to present and then head back to Ottawa. I managed to skip one session at this stage but, after a quick call home, I decided to go along to Alex's RAC Connection Management presentation after all (a little late). Although I have seen some of this stuff before, I always enjoy watching Alex's demos and was particularly impressed by the fact that he'd managed to write his own RAC connection load balancer! I was waiting for the applause in the room but either people didn't quite get it or there was just a lack of energy post-lunch on the last day. I suspect the latter.

Of course, once I'd said goodbye to Alex properly (don't see him nearly enough), I was a little late for whichever session was going to be my final one of the conference and I was hopelessly torn between Kyle Hailey's modern SQL performance tools presentation (Kyle's done a lot of cool work in the area of Oracle Performance Visualisation) and Chris Antognini's Diagnosing Parallel Executions Performance. In the end I plumped for the latter because I thought it was going to be like something I'd unsuccessfully attempted a couple of years ago and I wanted to see if Chris had a different angle on it and had been more successful. In the end, I probably made the wrong choice because although Chris' presentation was great, it was really all stuff I already knew. Definitely my bad call, though. Hopefully I'll get a chance to catch up with Kyle's presentation at some point in the future too!

After that there was just the usual short farewell and thanks from Gary Goodman of Hotsos. Although the thanks were appreciated, I'm glad they were spread around everybody because the attendees are one of the things that make this conference great and Becky and Rhonda did their usual sterling job of organising everything.

Then it was time for some Fajitas with a few friends (actually, a whopping great number of friends who practically filled the Mexican restaurant!) and a few very sedate beers. (We are old men (and women) now and the night before was a big one!) While we were waiting to go to the Mexican, I had one great surprise left - Alex's flights weren't going to get him home, so he came back from the airport and had to check in overnight! At least I got a chance to talk to him properly when I wasn't hopelessly drunk and didn't try to seduce him this time.

Now I need to stop blogging and get back to listening to Tanel's Training Day (good stuff, too, but more about that later)

I had a very pleasant day yesterday at a SIG meeting of the UK Oracle User Group where I did the presentation about “drawing your SQL” that is the basis of the article I wrote for Simple Talk a little while ago. One of the other sessions had the entertaining title: “What happens if you drop [...]

Yesterday I attended and spoke at the Scotland DBA SIG. It was an enjoyable day with not one but two presentation from Julian Dyke. The second one was an overview of HA options and it in Julian mentioned extended RAC. Now I have worked at several sites which have used or wanted to use extended RAC and I have heard a number of views on what exactly the term extended means.

From MoS Note 220970.1 we get the definitive statement on the maximum distance for extended RAC

The high impact of latency create practical limitations as to where this architecture can be deployed. While there is not fixed distance limitation, the additional latency on round trip on I/O and a one way cache fusion will have an affect on performance as distance increases. For example tests at 100km showed a 3-4 ms impact on I/O and 1 ms impact on cache fusion, thus the farther distance is the greater the impact on performance. This architecture fits best where the 2 datacenters are relatively close (<~25km) and the impact is negligible. Most customers implement under this distance w/ only a handful above and the farthest known example is at 100km. Largest distances than the commonly implemented may want to estimate or measure the performance hit on their application before implementing. Do ensure a proper setup of SAN buffer credits to limit the impact of distance at the I/O layer.

So we now have figures that 100Km is technically possible but probably won’t work well and most sites use less than 25Km. Co-incidentally in the paragraph above that quotation it refers to SE RAC (10g) as being limited to having all nodes located in the same room.  So in two lines of text we have just gone from a 100ft to 100KMs.

Yesterday Julian’s answer when I asked him the question was that he thought extended was defined as a couple of kilometres or above which is reasonable. However I am thinking that a better definition of extended is not to focus on distance of nodes apart but on what connects the nodes. My thoughts are that that if fibre channel or DWDM is involved and the nodes are not in the same building  then that would count as an extended RAC cluster despite potentially only being a 100 yards apart. Another view would be that if some form of array based mirroring is in place and RAC technology is in place then that again would define the system as ‘extended’ .

Now that all fits with a view I have heard from a well-known RAC consultant from Oracle who defines any RAC system where nodes are in different rooms as extended.  I have never subscribed to that theory but I can see that the definition I provided in the previous paragraph would lead to that conclusion.

Now we just need to consolidate on the word ‘extended’ rather than the use of  ’stretched’ which I see occasionally.

I really do hope I get some comments on this piece as I am sure it is worthy of debate and comment and anything that helps to create a shared view on the matter can only be good.

An excellent joint conference for Estonian, Finnish, Latvian and Russian Oracle User Groups in Tallinn, 20. - 21. May 2010.
Speakers also include Tom Kyte, Tanel Põder, Chris J. Date and Steven Feuerstein.

Read the agenda here and register in OUGF home page (250€+VAT registration fee).

In addition, just before the conference, 17.-18. May, Chris Date will perform his "How to Write Correct SQL and Know It: A Relational Approach to SQL" seminar in Helsinki. More info here and more detailed information here.

March 11, 2010

Sometimes I receive seemingly interesting emails showing Oracle errors – leaving me to ponder… certainly, that can’t cause an error, can it?  Here is one that I received a year ago (paraphrased):

The commercially developed application that we are using displayed an error message identifying a SELECT statement as the source of an ORA-00001 error.  What is the source of the Oracle constraint error?  The error message displayed by the application is as follows:

select account_period
from PROJECT_SUMMARY
where project_id = :m_saProjSumProjectID[nProjSumIndex]
and id =:m_saProjSumSavedID[nProjSumIndex]
into :nACCOUNT_PERIOD

ORA-00001: unique constraint (TESTUSER.SYS_C006354) violated

This transaction has resulted in violating an Oracle defined constraint.
Constraints are enforced by the database manager.  This transaction has
been rolled back.

My first thought at the sight of this error was that the commercially developed application was actually displaying one of the SQL statements that was executed after the SQL statement which triggered the primary key violation.  It is easy to let a runtime error slide through for a period of time before the program notices that an error happened – maybe it is just a sign of sloppy programming (I hope not, because this has happened in some of my custom-developed programs too).

How would we start to troubleshoot this error message?  The “SYS_C” portion of the constraint name indicates that the constraint is most likely a system generated constraint name, probably intended to enforce a uniqueness requirement for a primary key column.  Exporting the data from the database using Datapump export (or the legacy exp utility) and importing the data into a new database could cause the number following “SYS_C” to change, and it is likely that constraint SYS_C006354 in my database (that is used by the same application) is very different from that of the person who posed the question to me.

Let’s see if we are able to find the answer by working the problem in a circular fashion.  For example, let’s find the name of the index that is used to enforce the primary key constraint on of one the application’s tables:

SELECT
  INDEX_NAME
FROM
  DBA_INDEXES
WHERE
  INDEX_NAME LIKE 'SYS%'
  AND TABLE_NAME='INVENTORY_TRANS';

INDEX_NAME
-----------
SYS_C005168

Now that we know that the index is named SYS_C005168, we could do something like this:

SELECT
  DC.OWNER,
  DC.CONSTRAINT_NAME,
  DC.CONSTRAINT_TYPE,
  DC.TABLE_NAME,
  DC.STATUS,
  SUBSTR(DCC.COLUMN_NAME,1,30) COLUMN_NAME
FROM
  DBA_CONSTRAINTS DC,
  DBA_CONS_COLUMNS DCC
WHERE
  DC.CONSTRAINT_NAME='SYS_C005168'
  AND DC.OWNER='TESTUSER'
  AND DC.OWNER=DCC.OWNER
  AND DC.CONSTRAINT_NAME=DCC.CONSTRAINT_NAME
ORDER BY
  DCC.POSITION;

OWNER   CONSTRAINT_NAME  CONSTRAINT_TYPE  TABLE_NAME       STATUS   COLUMN_NAME
------  ---------------  ---------------  ---------------  -------  --------------
SYSADM  SYS_C005168      P                INVENTORY_TRANS  ENABLED  TRANSACTION_ID

The above output shows that the primary key constraint SYS_C005168 enforces the uniqueness of the primary key (TRANSACTION_ID) column in the table INVENTORY_TRANS.  We just demonstrated that we now know what we already mostly knew.

In the case of the person who sent the email to me, the table name was not known.  So, we could take the last of the above SQL statements and substitute SYS_C006354 in place of SYS_C005168 to find the table name and primary key column that was violated.  If the SQL statement failed to return usable information the next step might be to enable a 10046 trace at level 4 for one of the affected sessions, and try to reproduce the problem.  A 10046 trace will list the sequence of events that led up to the error message appearing in the client application.