Execution plan rows

The plan

As mentioned in previous post, our example is based on the sample SH schema, with an added table FOOBAR (id number, key varchar2(30)):

SELECT prod_id, key FROM products CROSS JOIN foobar WHERE prod_id in (143,144,id) and id in (1,2,3);

In all reverse engineering, it's good to start with something simple and to know what we to look for. Thus we want to know what the execution plan should look like; and the more unique numbers/ids we can find, the better. It's much easier to look for a number like 0x12fa1893 than for 0x0 or 0x1.

The execution plan, obtained using:
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY(null,null,'ALL')); 
is

I have added the CPU cost from the full detail of the execution plan in  v$sql_plan / x$kqlfxpl.

Looking at the numbers, we also have rows (1 and 2 ... not very unique), bytes (34 / 30 / 8 - not bad) and what is not shown here, we also know object ids of the index and the table: 94765 and 92749 (nice).
We did not use any temp space and no partitions.

Tanel Põder pointed out long time ago that the plan operations consist of operations and options and where to find their numeric ids (x$xplton, x$xpltoo). In our case, we have:

Operations:
0x37 SELECT STATEMENT
0x02 NESTED LOOPS
0x26 TABLE ACCESS
0x17 INDEX
Options:
0x18 FULL
0x0b FULL SCAN

And why do we want to know? Again, we will be looking for these numbers. If we see all these numbers together, we can assume it has something to do with the execution plan and then go on and guess the structure of the plan and so on.

Where do we start

There is a big hurdle to overcome - where do we start? The SGA is huge.
Generally, we can go with:

• The x$ table is filled by a helper function, and Google or My Oracle Support show that it's kqlfgx. Tracing this function can show us what data it is reading.
• Brute force - we know where the SGA is, so we can look for some interesting numbers
• We know that PGA knows about current cursor executed. And such cursor must have an execution plan associated. We also know that the plan is in SGA. This is the easiest and fastest way, so let's use it.

From PGA to the plan

We start with kxscio (see previous post) and we look at data stored there - especially any pointers going to the SGA (private process memory and SGA sit in a very different parts of memory, so one can see easily with own naked eye whether the address is in SGA or PGA/stack). And then we look at the data there and look for any of the "magic" numbers from the plan.

And indeed, it does not take long to find something at 0x2d0 (obviously the exact offset is version and platform dependent, this is 12.1 on Linux 64-bit):

To a trained eye, this looks like a gold mine, although I suppose you really do need a trained eye to appreciate it :-)
Let's try to guess what we are looking at. Please note that in real life, it takes quite some time to get this insight, and you want to run at least a dozen of different execution plans to get some kind of confidence:

For starters, we see plan operations and options: 26 18, 17 0b. There is no 37 - apparently the top operation is not part of the plan. There are 02s, but that's not unique, so wait with that.
Just next to the operations, we see 02 02, 02 03... which are depth and plan row. So there should be 01 01 for the first row, followed by 02... and there it is, near the start! This looks promising, so let's continue.

 Let's look at the CPU costs. These should be 0x997d, 0xbd1, 0x7dab (I just converted the decimal values from the plan). The values in the hex dump are kind of there: 99 7d, 9b d1, 7d ab. And we don't see any zeroes around them... how would Oracle store larger numbers, if there is no free space where to put them (like it would be if these were proper say 32 or 64-bit numbers?). And what about the extra 9 in the second cost?

Similar, looking at the object ids (only rows 2 and 3, as row 1 does not have any object in the plan), we find c1 72 2d, c1 6a 4d. Things now become clear - it's using some kind of variable-length encoding, similar to what UTF-8 uses.

We also see that the first byte is 8f, and it's there 3 times, always 3 bytes before depth + row id. And somewhere after the last row, there is 8e. So now we know where each row starts.
Now let's ponder for a bit: If we were run for example a partitioning query, we would see partition ids here. Or we would see temp space. But we don't see anything like that here... it would be a waste of space. So how do we/Oracle know/s what is stored here?
The answer is in the first two bytes of each row - it's a bitmap denoting what pieces of data are present.

Putting it all together, we can identify row start, the bitmap, depth, row id, operation, option, cost, CPU cost, I/O cost, cardinality, bytes. The bitmap, if we try very hard, also indicates how long each row is - but for visual check, just finding the next 8f is enough. (Note: the bitmap for the first row indicates some extra data present - and the 55 shows length of the data.)
There are some unidentified values - and that's just the life in reverse engineering. Maybe further investigation would identify them, maybe not. We probably don't need them for parsing the plan, so we just leave that as it is.