Dave's Adventures in Business Intelligence » Conferences

Change Management Strategies For Universe Developers

Dave Rathbun — Wed, 28 Jul 2010 13:13:17 +0000

Yesterday I mentioned my first talk that I will be giving at the fall BusinessObjects conference this year. Brian Durning and I are going to compare and contrast Crystal and Web Intelligence. But that’s not all. In the session immediately following that one I will be giving a talk about change management for BusinessObjects universes.

Session Code: 804
Title: Don’t Lose Control: Change Management Strategies for Universe Designers
Day and Time: Tuesday, October 05, 2010, 2:45PM – 3:45PM

My final material is subject to change as I see how long each section gets, but here’s a brief overview of the areas I am considering covering.

Impact of database changes
What happens to my universe when my database changes? How can I address those issues in a way that causes the least impact on my reporting team?
Impact of universe changes
What happens to reports when I change my universe?
How can I compare two versions of a universe to see what is changing?
This is the big one… several years ago I started a script that allows me to check two different versions of a universe and see what is different. I have been using the script for a while now and it does most of what I need, but it isn’t complete. I plan to (hope to) get the code finished by October so I can distribute it at the conference, and of course here on my blog. The code will be released under the GNU GPL (open source license). Details of that license and what someone can (or cannot) do are found on the GNU GPL home page. I will demonstrate the code and how it works under various change scenarios during my presentation… assuming I get it all working the way I want it to, of course.

These were the ideas I had in mind when I submitted the abstract. Time will tell which of them make it into the final presentation. Anything that doesn’t make it will very likely show up here as blog posts.

At some point, anyway. I mean I only started working on my universe change script in 2007…

Dueling Reporting Tools

Dave Rathbun — Mon, 26 Jul 2010 14:53:08 +0000

I submitted several abstracts for consideration for the fall conference this year. The one I was the most excited about got accepted into the Webi/Deski track, and is called, “Discovery Versus Delivery: A Comparison of Client Reporting Tools.” Here is the formal track designation information.

Session Code: 603
Title: Discovery Versus Delivery: A Comparison of Client Reporting Tools
Day and Time: Tuesday, October 05, 2010, 1:30PM – 2:30PM

What is this talk about? A while back I wrote a post about a very short (single page) slide from SAP that shows how to pick which tool for a particular problem. For this presentation, Brian Durning and I will be doing a demonstration of strengths of two of the most important tools from BusinessObjects: Crystal and Web Intelligence. We plan to cover (and demonstrate!) several scenarios that should help show where the strengths of each tool lie. I will be covering Web Intelligence, and Brian Durning will be covering the Crystal side of the house.

It should be fun.

Related Links

Related Blog Post: Delivery Versus Discover

October Conference Season Heats Up

Dave Rathbun — Fri, 23 Jul 2010 19:09:05 +0000

A few weeks back I got an acceptance notice for an abstract I had submitted to TechEd this year. TechEd is in Las Vegas and I will be speaking about universe design to SAP folks that want to learn more about how to use BusinessObjects tools outside of their ERP/BW environment. It should be fun.

Earlier today I got notified that I have also been accepted to speak at the ASUG fall conference in Orlando. This is the conference that replace the Insight conferences put on by BusinessObjects these past few years, which were then followed by the two GBN conferences in Dallas. Since GBN has been folded into ASUG now our fall conference is now under their umbrella. If you are a “classic” BusinessObjects customer who wants to continue to see new and exciting things that you can do with your tools, this is the conference to attend. There will be some great stuff coming from SAP employees about XI 4 that you won’t want to miss, as well as (I am sure) a number of “can’t miss” sessions related to XI 3.x (and perhaps even earlier versions) that will help you make the most of your current environment while we all wait for XI 4 to be generally available.

In a few days I will post more details about my abstracts. I look forward to seeing everyone in Las Vegas or Orlando or perhaps even Both.

Universe Models For Recursive Data Part III: Alias Versus Flattened

Dave Rathbun — Fri, 02 Jul 2010 11:45:03 +0000

This is the third of several posts that will review my presentation “Universe Models For Recursive Data” which was originally presented at the 2009 GBN conference, then at the North Texas / Oklahoma ASUG chapter meeting, and finally at the Mastering BusinessObjects conference in Melbourne. As with my other presentations there is a PDF file that can be downloaded from my conference presentations page. The first post introduced the concepts of recursive (as opposed to hierarchical) data and provided a couple of examples. The second post reviewed some of the different design challenges that I have seen in working with recursive data models. In this post I will introduce four different possible solutions and present a scorecard for each, showing how well it solves the issues presented in the prior post in this series. Links to both prior posts are presented at the end of this entry. I have also included Oracle SQL scripts that can be used to create and populate the tables used in this post.

This post will cover slides 22 through 30 from the presentation and will describe the first two solutions (one with two variations) outlined in the presentation.

Solution Options

The four different solutions that I included in my presentation were: Universe aliases, Flattened structures (column or snowflake), Ancestor Model, and Depth First Tree Traversal. All of them work fine on a clean recursive hierarchy. Each of them partially works for at least some of the other challenges. Some of them present unique challenges (extra disk space requirements or lack of native drilling functionality) that will also be addressed. I am presenting the solutions in increasing order of complexity. This post will cover aliases and flattened structures (both versions). In the next post I plan to cover the ancestor model, and finally I will cover the depth first tree traversal in its own post.

Universe Aliases

This solution is the only one that can be completely self-contained within the universe. No DBA or ETL work is required. There are any number of ways to create an alias. I can:

Right-click on a table and select Insert Alias
Select an existing table in my structure, then select Insert + Alias from the menu
Open my table browser and insert an existing table. An alias will automatically be created for me.
Select an existing table in my structure and click the “Insert Alias” toolbar button

… and there are other ways to get aliases in my universe, especially if I have loops to resolve. The bottom line is that the process is quite simple.

Here’s what an alias looks like after it has been created and joined to an existing table in my structure.

The join can be a bit tricky. In this case, the employee row MGR_ID is joined to the manager row EMP_ID in order to make the relationship work. It might help to look at the raw data again from an earlier post.

See how the recursive relationship is going to work after establishing this join? Field works for Ferrerez, and Ferrerez works for Noakes. Who does Noakes work for? His MGR_ID column is empty (NULL) implying that he does not have a manager. He owns the company.

Pros of Alias Solution

The primary advantage of this solution is that it is completely self-contained in the universe. No DBA or ETL work is required. That’s about it.

Cons of Alias Solution

There are several cons to this solution. It does not represent lateral relationships at all. I have to use outer joins in order to preserve those rows with missing keys (Noakes in this example). Both of these are important, but the most substantial drawback to this solution is that the depth is determined by the number of aliases that the universe designer creates. In the image shown above there is only one link: from manager to direct employee. How can I — in one step — determine my indirect reports? With only one level of alias, I can only report one level of my hierarchy. How many can I report with this structure?

With that structure I now have two outer joins, but I can report on three levels instead of just two.

How many alias levels do I create? Generally when I have seen this solution used (or used it myself) we resort to asking how many levels are required and then creating some number above that. If I need five, I will create seven. If I need seven, I will create ten.

That means, of course, if I have created ten levels and all of a sudden we have twelve I have to update my universe. That’s not a problem (as long as I keep up with things) but it’s certainly not desirable.

Alias Scorecard

Here’s the scorecard for the alias solution for each of the four scenarios I outlined earlier.

Aliases are the easiest solution to implement but they don’t score well. Let’s move on to the next solution.

Flattened Structure – Single Table Columns

The next solution involves running either a SQL script or some form of ETL. I need to take the recursive table relationship and flatten it out much like I did with aliases, but this time in the database itself. The net result is that I will take data going down in rows:

and pivot it into columns in a table.

The net result is all of my recursion is done during the script process and I end up with one table that contains everything (or every person in my case) stored at their specific level in the hierarchy. It easily allows me to drill because it creates a very natural hierarchy.

Pros of Flattened Table Solution

It handles unbalanced hierarchies much better than aliases because missing lower nodes are simply NULL in the table. That’s fine. This solution can also handle ragged hierarchies with a proper “plug node” strategy. If I have a lower level value (Divisional Director) that reports directly to the president (top level) then level 2 (Vice President) will be empty. I need to fill something in so I can drill properly. More important, that plug node has to tell me what the path is or else I cannot drill up properly. Suppose I had a director named Smith who reported directly to Noakes. The first column in my table would include Noakes. The third column would include Smith. The second column (the missing value due to the raggedness of my data) would contain Smith VP Not Assigned or something like that.

Flattened tables cannot handle lateral hierarchies at all because I can’t store two values in a single column.

Cons of Flattened Table Solution

As already mentioned, this solution cannot handle lateral hierarchies at all. It also requires DBA or ETL work if the number of hierarchy levels changes. My column names should reflect the position (node type) in the hierarchy. That’s not a problem unless my hierarchy levels change, then I might want to update my structures.

But by far the most critical issue with this solution is the fact that it requires DBA or ETL work if my levels ever change. Much like aliases when I have seen this solution implemented I generally see extra columns at the end of my table just to allow for future expansion.

Flattened Table Scorecard

Here is my scorecard for the Flattened Table solution.

Flattened Structure – Snowflake Tables

One thing that I noticed about the data for the flattened structure is that I repeat a lot of values. For example, Noakes is the “level 1 mgr” for every person in the company. It might seem to be more efficient to use a structure like this:

This would reduce my overall storage requirements because I would end up with a single row for the highest level table.

However, it also reintroduces the need for outer joins, which the initial flattened structure avoided.

Pros of Flattened Snowflake Solution

Because the tables get smaller as I get further up the tree (ultimately to a single-row table in my simple example) my overall storage requirement should be smaller as well. If I only need the top one or two levels, my queries should be very efficient. Finally, I think it would be easier to maintain as well. If a new level appears, I add a new table to my chain with the proper restrictions on the ETL for proper table population.

Cons of Flattened Snowflake Solution

Each of the solutions defined so far suffers from some form of this issue: I have to define a table (or column) for every possible level of my hierarchy. If I do not know what the total number of levels will be, I can try to anticipate and create extra tables to support future expansion. But that is not the best solution. Because these tables are maintained in the database, I have to talk to my DBA or ETL team when changes are required. Because the tables are joined I have to consider whether to use outer join to preserve depth on unbalanced hierarchies. And finally, the “plug node” strategy I outlined earlier becomes a “plug row” strategy in this case, and that’s substantially more complicated.

Flattened Snowflake Scorecard

Here is the scorecard for the flattened snowflake solution. In my opinion, it’s a slightly worse solution than the flattened table solution simply because of the join issue and the plug row concern.

Next Time

The solutions covered in this post are the least complex and therefore offer the least flexibility. They are easy to set up; in the case of aliases the entire solution can be built within the universe designer application. All of the other solutions require some sort of database scripting. In the next post I will talk about the ancestor model and how we used it at a manufacturing client. It has some definite advantages, and it handles just about all of the different challenges I have outlined. I don’t have to worry about plug nodes, and it handles both ragged and unbalanced hierarchies quite well. However it has an impact on disk usage and it can’t be drilled using the native functionality provided by BusinessObjects. Do the pros outweigh the cons? Come back soon and see for yourself.

Related Links

Supplemental Material
Scripts to create and populate the basic HR table used for this presentation.

Create table

create table employee
(emp_id number(5) not null
,emp_lastname varchar(20)
,emp_firstname varchar(15)
,emp_dob date
,emp_address varchar(40)
,emp_area_code varchar(7)
,emp_town varchar(15)
,emp_phone varchar(18)
,showroom_id number(4)
,emp_start date
,emp_mgr_id number(5)
,emp_sex varchar(1)
,job_id number(4));

alter table employee add constraint emp_pk primary key (emp_id);
create index emp_dept on employee(dept_id);
create index emp_showroom on employee(showroom_id);
create index emp_mgr on employee(emp_mgr_id);

Populate table

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (101, 'Noakes', 'Nicholas', '12-MAR-48', '2356, Melrose Street', '30190', 'San Jose', '12-00-00-01', '01-JAN-91', NULL, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (102, 'Ferrerez', 'Ferdinand', '10-FEB-64', '25 Arcadia Avenue', '75897', 'Los Angeles', '22-55-56-32', '30-MAR-96', 101, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (103, 'Field', 'Felicity', '15-DEC-60', '12 Brasilia Street', '12014', 'Santa Barabara', '14-46-54-22', '26-MAR-95', 102, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (104, 'Fraser', 'Frank', '13-MAR-67', '45 Seaside Avenue', '75016', 'Los Angeles', '22-55-18-33', '13-DEC-91', 101, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (105, 'Snow', 'Sara', '03-OCT-65', 'Square Woodstock', '18000', 'San Jose', '14-34-34-30', '01-MAY-93', 101, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (106, 'Speed', 'Sonya', '03-DEC-70', '5, The Vale', '22000', 'San Jose', '14-32-39-43', '04-JUL-96', 105, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (107, 'Spencer', 'Steve', '01-NOV-64', 'Square Osaka', '33010', 'Los Angeles', '22-24-25-89', '16-APR-91', 105, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (108, 'Helen', 'Harrison', '01-AUG-66', 'Via Firenze', '38200', 'Los Angeles', '22-34-31-11', '13-MAY-94', 101, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (109, 'Thomas', 'Tom', '01-DEC-68', '11 Over Way', '24000', 'San Jose', '22-45-67-45', '20-DEC-95', 101, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (110, 'Thatcher', 'Terry', '03-OCT-50', 'Stars Parkway', '21000', 'San Jose', '12-11-11-09', '06-DEC-92', 109, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (111, 'Davis', 'Diana', '12-AUG-64', 'Rue Opera Sauvage', '92100', 'Los Angeles', '14-54-11-10', '22-SEP-92', 101, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (201, 'Pickworth', 'Paul', '12-FEB-51', '23 Las palmas road', '00316', 'New York', '12-24-26-44', '12-JAN-93', 101, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (202, 'Forest', 'Florence', '10-OCT-32', 'Rue des Lombards', '75100', 'New York', '22-54-11-10', '23-DEC-94', 201, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (203, 'Brown', 'Bella', '12-APR-59', 'Hollywood Blv', '36020', 'New York', '22-36-25-50', '03-FEB-92', 202, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (204, 'Porter', 'Pete', '15-NOV-57', 'Avd Torre De Embarra', '34100', 'New York', '14-44-11-66', '13-APR-92', 201, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (205, 'Irving', 'Ira', '12-FEB-64', '44 Beach avenue', '13000', 'New York', '12-56-55-20', '18-JUN-95', 204, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (206, 'Bailey', 'Ben', '12-JUN-57', '4 Palisades Drive', '75090', 'Long Island', '12-33-51-29', '01-DEC-90', 204, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (207, 'Duckworth', 'Dave', '09-SEP-66', 'Rue du grand temps', '75018', 'New York', '12-85-01-61', '04-NOV-93', 201, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (208, 'Ince', 'Ian', '10-AUG-53', 'Sunset Blvd', '31061', 'New York', '22-52-22-00', '04-DEC-95', 207, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (209, 'Hilary', 'Hibbs', '01-FEB-60', 'Sand Hill Road', '92800', 'New York', '12-54-11-10', '08-JUN-95', 202, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (301, 'Dagmar', 'Davinda', '12-APR-58', '12, The Crescent', 'SL1 1HG', 'Slough', '01628-764234', '24-JUN-95', 101, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (302, 'Presley', 'Percy', '30-OCT-62', '1 Jubilee Close', 'SL5 23F', 'Maidenhead', '01628-834582', '15-JUL-95', 301, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (303, 'Perry', 'Philippa', '24-FEB-71', '23 Rice Hill', 'SL3 12S', 'Maidenhead', '01628-567231', '28-SEP-96', 302, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (304, 'Hubert', 'Henri', '13-DEC-69', '5 Grand Lane', 'SL3 12S', 'Maidenhead', '01628-243535', '17-APR-96', 302, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (305, 'Adamson', 'Anita', '12-OCT-69', '24 Loose Lane', 'SL4 23D', 'Cookham', '01628-782364', '15-FEB-96', 301, 'F');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (306, 'Beaver', 'Bertie', '12-MAR-72', '223 Grange Hill', 'SL2 67E', 'Windsor', '01628-187632', '13-JAN-96', 305, 'M');

insert into employee (emp_id, emp_lastname, emp_firstname, emp_dob, emp_address, emp_area_code, emp_town, emp_phone, emp_start, emp_mgr_id, emp_sex)
values (307, 'Motson', 'Mervin', '22-DEC-74', '67 Blows Down', 'SL5 45G', 'Cookham', '01628-198371', '17-JUN-96', 305, 'M');

Scripts to create and populate the flattened version of the HR table, Oracle syntax

Create flattened table

create table emp_flat
(emp_lvl_1 varchar2(20)
,emp_lvl_2 varchar2(20)
,emp_lvl_3 varchar2(20)
,emp_lvl_4 varchar2(20)
);

Populate flattened table.
Only four levels are supported.
Starting point (Noakes) is hard-coded.

insert into emp_flag (emp_lvl_1, emp_lvl_2, emp_lvl_3, emp_lvl_4)
select a.emp_lastname
,      b.emp_lastname
,      c.emp_lastname
,      d.emp_lastname
from employee a
,    employee b
,    employee c
,    employee d
where a.emp_id = b.emp_mgr_id(+)
and b.emp_id = c.emp_mgr_id(+)
and c.emp_id = d.emp_mgr_id(+)
and a.emp_id = 101;

Create Snowflake Tables

create table emp_level_01
(emp_id number(5)
,emp_lvl_1 varchar2(20));

create table emp_level_02
(emp_id number(5)
,emp_mgr_id number(5)
,emp_lvl_2 varchar2(20));

create table emp_level_03
(emp_id number(5)
,emp_mgr_id number(5)
,emp_lvl_3 varchar2(20));

create table emp_level_04
(emp_id number(5)
,emp_mgr_id number(5)
,emp_lvl_4 varchar2(20));

Populate snowflake tables
Only four levels are built, each starting from the prior table.
Starting point (Noakes) is hard-coded.

insert into emp_level_01 (emp_id, emp_lvl_1)
select emp_id, emp_lastname
from employee
where emp_id = 101;

insert into emp_level_02 (emp_id, emp_mgr_id, emp_lvl_2)
select e.emp_id, e.emp_mgr_id, e.emp_lastname
from employee e, emp_level_01 e1
where e.emp_mgr_id = e1.emp_id;

insert into emp_level_03 (emp_id, emp_mgr_id, emp_lvl_3)
select e.emp_id, e.emp_mgr_id, e.emp_lastname
from employee e, emp_level_02 e2
where e.emp_mgr_id = e2.emp_id;

insert into emp_level_04 (emp_id, emp_mgr_id, emp_lvl_4)
select e.emp_id, e.emp_mgr_id, e.emp_lastname
from employee e, emp_level_03 e3
where e.emp_mgr_id = e3.emp_id;

Universe Models For Recursive Data Part II: Design Challenges

Dave Rathbun — Sat, 26 Jun 2010 00:38:35 +0000

This is the second of several posts that will review my presentation “Universe Models For Recursive Data” which was originally presented at the 2009 GBN conference, then at the North Texas / Oklahoma ASUG chapter meeting, and finally at the Mastering BusinessObjects conference in Melbourne. As with my other presentations there is a PDF file that can be downloaded from my conference presentations page. The first post introduced the concepts of recursive (as opposed to hierarchical) data and provided a couple of examples. In this post I will review some of the different design challenges that I have seen in working with recursive data.

I decided to identify and cover four different examples of recursive data configurations. These included Clean, Unbalanced, Ragged, and Lateral. As I mentioned in the first post, I am going to use some basic human resources (HR) data for my examples. For this post, in order to show samples of each of the four challenges, I am going to represent my recursive data using a tree. The branches of the tree show the relationships between people. The nodes of the tree contain the information about each person. The data might include their name, hire date, and position (title) within the company. In order to properly interact with my recursive data I have to be able to work with both types of information: relationships and node data as well. If you are not sure what I mean, please continue reading, this will make more sense later on.

This post will cover slides 14 through 21 from the presentation and will describe each of the different recursive challenges that I identified.

Clean Hierarchy

In my first example everything is very clean. Each branch of the tree has the same depth. Each branch follows the same path. There are no real challenges encountered in this hierarchy, pictured below.

Imagine that the top of the tree is the company president. The second level (the “B” nodes) represent vice presidents, and the third level (”C” nodes) represents divisional directors. When a hierarchy definition is very rigorous this is the type of tree I expect. For a very simple example let me suggest a product hierarchy instead of an HR chart for the moment. A product hierarchy for a food company might include a Brand Owner, the Brand, the Size, and finally the Flavor. The brand owner could be Beverages-R-Us, the brand could be Super Sports Drinks, the size is two liter bottle, and finally the flavor is Orange. Every product in the system is guaranteed to have all four of these attributes assigned, and they will all be in that exact order.

On the other hand, a human resources hierarchy is rarely as clean. Let me move on to some more interesting examples.

Unbalanced Hierarchy

An unbalanced hierarchy is one where the nodes are at inconsistent depths. Please review the tree shown below.

In the example shown above, there is one node (B1 in this case) that does not have any children while the rest of the nodes at that level (B2) do. If the A node is the company president, and the B nodes are vice presidents, it is entirely possible to have a position (perhaps “VP of Special Projects”) that does not have any additional people that report up to him or her. In that case the tree stops at the VP level and does not go down to the Divisional Director position.

Why is this a challenge? As will be seen later, one of the possible solutions to a recursive data scenario is to pivot the data (flatten) it into different columns. What happens to the missing nodes in this case?

Ragged Hierarchy

In the last example I suggested that there could be a VP of the company that does not have any direct employees. In the case of a Ragged hierarchy it’s slightly different. In this case I might see a Divisional Director who is reporting straight up to the company president without going through a VP.

Note that in the image above I am showing both an unbalanced node (B1) and a ragged node (C2). Let me focus on C2 for a moment. As I already mentioned, there is a relationship from that director position straight up to the president. It does not go through a vice president position. Why is this a challenge? Remember that earlier I mentioned there are two parts that I need to account for: the relationship and the position or node type. In this case the relationship only goes one step, but descends two levels (from president to director). I need to be able to represent both parts properly in whatever data model I come up with.

Lateral Hierarchy

If you have spent any time reviewing company organization charts you may have seen this type of relationship before: I am calling it a lateral (sideways) relationship.

It’s not uncommon to see a lateral relationship from one director to another director (C2 reporting to C1 in this example). This is one of the biggest challenges to most of the design ideas I will be sharing in my next post, because there are two things (people) occupying the same space (node type) on the tree).

Merge / Diverge

As I mentioned toward the beginning of the post that some scenarios are inherently cleaner than others because the relationships all have to exist. Unfortunately, it is quite likely to see a combination of issues. I have even seen challenges where a hierarchy does a merge / diverge relationship such as this:

SAP and other ERP vendors generally allow this sort of hierarchy to be built in order to provide the maximum flexibility to the client company. I have never tried to implement this in BusinessObjects because it simply does not work. There is no clear drill path. Suppose I drill from node B2 to C3, and then from C3 to D2. Now when I drill up, which path do I take? I can drill from D2 up to C3, and then from C3 I can drill up to either node A1 or B2. It’s ambiguous, and therefore our project team decided that we would not attempt to handle this at all. We instituted a business rule (an exception) that would kick out any hierarchy that included this sort of path.

This particular example was dropped from the presentation in the interest of time but I wanted to mention it here.

Combinations

Even without the merge / diverge issue, there are plenty of still challenges. For our project, a typical tree was both ragged and unbalanced. That meant that the solutions we discussed had to be able to handle both. We also had a number of lateral relationships that we needed to address. Our users wanted to be able to enter the tree by node type and drill by level. They wanted to see the entire tree presented as part of a prompt. And they wanted to be able to multi-select from those prompts… for any node at any level.

Next Time

Which solutions work the best? Do any solutions work for all of these different scenarios? My next post in this series will review each of the four solutions I outlined in my presentation and present a scorecard for each.

Related Links

Universe Models for Recursive Data Part I: Introduction

Accepted to Present at SAP TechEd 2010

Dave Rathbun — Wed, 23 Jun 2010 14:14:24 +0000

I got an email yesterday saying that one of my abstracts has been accepted for the 2010 edition of SAP TechEd. I will be talking about universe design over non-ERP data, showing the SAP folks that you can build universes on more than just cubes.

The selection process for the fall BusinessObjects conference in Orlando is continuing through July 2^nd. I submitted several abstracts for that conference as well, including one on universe change management that will feature a (hopefully) completed version of a script to compare universes that I started writing years ago. I am using the script today (thus the abstract) but it’s far from completed, and certainly not ready for public consumption.

As a side note, the “Conferences” category on the right menu for my blog has continued to grow. I experimented with a couple of Wordpress plugins that would allow blog readers to expand / collapse the categories as needed, but so far neither of them have worked to my satisfaction. Either they didn’t work in all of the browsers I tested, or they looked bad, or they were not as functional as I would like. But hopefully I will get something there soon in order to avoid having that category take over the entire side of the screen.

Universe Models For Recursive Data Part I: Introduction

Dave Rathbun — Wed, 16 Jun 2010 18:38:36 +0000

This is the first of several posts that will review my presentation “Universe Models For Recursive Data” which was originally presented at the 2009 GBN conference, then at the North Texas / Oklahoma ASUG chapter meeting, and finally at the Mastering BusinessObjects conference in Melbourne. After presenting it three times it seemed like an appropriate time to (finally) get started writing up the blog posts. As with my other presentations there is a PDF file that can be downloaded from my conference presentations page.

This post will cover slides 6 through 13 as a basic introduction of recursive data and challenges presented to universe designers.

Defining Recursive Data

Sometimes there is confusion about the distinction between hierarchical and recursive data. Hierarchical data does not present a big challenge for BusinessObjects. It can be something related to time (Year, Quarter, Month, Day), geography (Country, Region, State, City), or something more specific like an accounting structure (Business Unit, Account, Sub-Account). What makes this hierarchical structure work easily is that each element is stored in a different place. It could be in a different column in the same table (flattened) or even in different tables (snowflake). As long as I can drill from one column to another in the hierarchy everything works fine.

Self-referencing or recursive data may initially look like a hierarchy. The key difference is that all of the elements are stored in the same place. There are keys that relate one row in a table back to a different row in the same table. That’s how recursive data is different from hierarchical data.

Why is recursion is a problem for BusinessObjects? The language used “behind the curtain” is SQL, and SQL does not natively support recursion. Some database vendors offer extensions (for example the CONNECT BY PRIOR structure in Oracle) but these are not used by BusinessObjects.

How common is recursive data? It is certainly not unusual. Consider any of the following:

Company organizational structure
Object levels: President – Vice President – Director
Object type: Person
Inventory BOM (Bill of Materials)
Object levels: Product – Assembly – Sub-Assembly – Component
Object type: Inventory item
Project Management
Object levels: Project – Task – Sub-Task
Object type: Project entry
Multi-Level Marketing (MLM)
Object levels: Founder – Recruit – Recruit Level 2
Object type: Person

In each of the above examples the type of object (or node) type is the same at any level. For example, a company organization chart is made up of people. Some people are at different levels, and there are therefore relationships from one person to another. In order to show all of the relationships from the top of the company to the bottom (or the bottom to the top) I have to keep going back to the same table. That is recursion.

Because it’s easy to think about a company organizational structure I used that example for the rest of the presentation.

Note: The Motors database is used in the standard Universe Designer training course and will not be presented in its entirety in the download package for this presentation for copyright reasons. However, I will be providing the standard HR table and all of the modified versions used in this presentation.

Example of Recursive Data Using Prestige Motors HR

A picture will help at this point. Here is a screen shot from the Prestige Motors HR universe that I built for this presentation. Notice that there are two tables in the picture, but one is an alias of the other. In other words, I am really using the same table twice.

The table on the left is the Employees table. I have aliased the table and called it Manager. The two tables are joined using the link from EMPLOYEE.EMP_MGR_ID to Manager.EMP_ID. Since this is really the same table twice, this join defines the relationship from any particular person to their immediate manager. It’s a recursive relationship from a person to a person.

Notice that in this case I have defined the join as an outer (optional) join? That’s because the top person in the company does not have a manager, and the relationship would fail in that case. I want to ensure that I return every person and their manager… even if that person does not have a manager. Here is a sample of some of the data to help show why this is important.

I can review the relationships manually if I want. I can look at the data (shown above) and determine that Pickworth works for Noakes. Davis and Ferrerez also work for Noakes. How am I making that determination? Each of those three folks has a manager ID of 101, and 101 is the employee id for Noakes.

Who does Noakes work for? The EMP_MGR_ID column is blank (null) for Noakes, which implies that he is at the top of the company organization chart.

Another way to see where people fall in the organization chart is to look at their level. Here is output from a report that I eventually will want to generate from my recursive data. It is shown in the format of a tree, with each person showing up as a node on the tree.

Noakes is at level 1. Davis, Ferrerez, and Pickworth are all at level 2. But the tree does not stop there. I have employees at level 3 and level 4 as well.

Typical Recursion Questions

This brings me to the set of questions that I need to be able to answer with my recursive data. I need to know:

Who do I work for?
Who works for me?
Who works at my same level and shares the same manager?
Who is my manger’s manager? My manager’s manager’s manager?
What is the total salary of my direct reports (people who work directly for me)?
What is the total salary of my indirect reports (people who work for people who work for me)?

I am sure there are many more questions but these should serve as a starting point. Some of the questions only require one level of the hierarchy (who works for me, or who do I work for). Those are simple enough to answer, and in fact can be answered with the simple alias structure already shown in this post. But in order to traverse the tree for multiple levels I need a solution that is a but more robust.

Next Time

In the next post of this series I want to talk about some of the different challenges encountered when working with recursive data. Once I define the challenges I will be in a position to start talking about solutions. As a preview, here are the four types of hierarchies I will be talking about:

Clean – a hierarchy with clean data, consistent node depths, and consistent node paths
Unbalanced- a hierarchy with inconsistent node depths
Ragged- a hierarchy with inconsistent node paths
Lateral- a hierarchy with sideways node paths

If it is not clear what some of those mean, don’t be too concerned; I will be defining each with examples in the next post.

Finally, here is a preview of the various solutions I will talk about:

Universe aliases
Flattened structures (columns or snowflake tables)
Ancestor / Descendant model
Depth first tree traversal

And a few that I won’t:

Oracle CONNECT BY PRIOR
Stored procedures

Part II of this series will talk in more detail about each of the recursive challenges. After I detail the different challenges the next post will talk about the solutions. My plans for the final post for this series are to review the impact of each solution on the native drilling functionality and then to wrap things up.

Related Links

Wikipedia on Inventory BOM in case you are unfamiliar with the concept of inventory data

Mastering BusinessObjects 2010 Review

Dave Rathbun — Fri, 11 Jun 2010 20:13:24 +0000

I was fortunate to once again be invited down to the Mastering BusinessObjects conference in Australia. I hope this conference continues to do well because I certainly enjoy going. Two weeks prior to this conference I was at ASUG Annual Conference / SAPPHIRE, and the difference is substantial. SAPPHIRE is HUGE, which might explain why their name is in ALL CAPS. The Mastering conference is much smaller (they’re only just getting started, really) and feels much more intimate. I talked to quite a few folks and got to hear some really interesting stories.

And like last year I got a few pictures…

The Airbus is an amazing airplane. It’s huge (double-decker) but incredibly smooth and quiet. I really enjoy flying on this plane.

I got to Melbourne early in the morning after flying all night. Here was the view from my hotel (quite a bit different from Manly last year).

They took a number of pictures of the speakers. Here’s one of me where I am obviously trying to make a point.

One night I was invited to join Tom Nather (and his wonderful wife and daughter) for dinner…

… on this trolley car…

The trolley spent 90 minutes driving around Melbourne while we ate a wonderful meal and were entertained by the restaurant staff. The food was amazing. After the food, we walked back to our hotel and saw the flaming casino.

The jets of fire go off every hour on the hour. We were across the river when I took this picture and we could still feel the heat!

On our continued walk back we saw this restaurant:

On Thursday I had a free day before heading back home. I walked all around downtown area looking for fun shops. I love learning about new musical artists, so I searched for music stores within easy walking distance of my hotel. I found five. Four were local independent stores like Record Collectors Corner:

or the appropriately named Basement Discs:

I was also drawn to what I am calling a “Corvette Truck” because I don’t know of any other way to describe it:

It’s a GM vehicle, but not one that they sell here in the United States. It sure looked fun to drive though. Even if the steering wheel was on the wrong side…

Call For Speakers Extended to June 13

Dave Rathbun — Mon, 07 Jun 2010 13:59:20 +0000

If you procrastinated just enough to miss the deadline, or if you were inspired by some extra stupendous ideas just after the submissions window closed, you are in luck. The call for speakers for the fall BusinessObjects conference has been extended until June 13^th.

To repeat something I mentioned earlier (a tweet from Jamie Oswald)

Did you know that speakers at #SBOUC get free admission. That’s like making $1,000/hr. Go submit an abstract. http://ow.ly/1KefI #ASUG #BOBJ
10:40 AM May 19th

Mastering BusinessObjects 2010 Day 2

Dave Rathbun — Tue, 01 Jun 2010 11:16:28 +0000

Today we closed the Mastering BusinessObjects 2010 conference. I presented my “Return of the Variables” talk, and afterwards I had someone come up to me and saw what I always love to hear: “You just solved one of my problems.”

Tom Nather did a very detailed session on setting up the configuration for single sign-on (SSO). I’ve been away from the server stuff for a while, so it was good to see the process laid out so cleanly.

Fellow SAP Mentor Ingo Hilgefort did a keynote this morning where he outlined current (SP3) and future (XI 4) product features, concepts, and directions. He was (rather predictably I think) asked about Desktop Intelligence, and just like every other time I have heard this question asked, the answer was: “It’s gone.” With folding now in 3.1 and the data manager coming in 4.0 it really seems like it’s time to read the writing on the wall and move on. The only big remaining “gotcha” for Desktop Intelligence users is the lack of scripting support (VBA) in the rich client. In many cases what I have seen scripting used for is now covered by the standard scheduling or other processes.

I had some really good conversations with folks here in Melbourne, and I thoroughly enjoyed my time here at the conference. It’s so much easier (it seems) to connect with people in a smaller setting.

After the conference Tom invited me to go with his family to eat dinner on a train. Who was I to refuse? It was a wonderful dinner, served on a trolley car that drove around the city for an hour and a half while we ate. The food was wonderful, the company even more so.

Tomorrow I am delivering a full-day seminar titled, “From Universe Development to Report Generation: The Untold Story.” It should be fun.