Defining the Problem

I am going to create a couple of very simple queries from the Island Resorts universe to demonstrate this feature. Each query is specifically structured so that it is missing some data found in the other query. For example, here is my first query.

And the data returned by this query.

And here is my second query.

And the data returned by this query.

I am going to call the first query my “left” query and the second query is my “right” query for the rest of this post. I have specifically restricted my left query to 2007-2008 years of data, and my right query to 2006-2007. That means the two queries have one year (2007) in common and each query also has unique data (left has 2008 and right has 2006).

Since both queries are from the same source I do not have to do anything to merge the shared dimensions. Both the Resort and Year are automatically merged as shown here.

And finally here is the resulting block of data shown in my Web Intelligence report. This is the most common presentation for the data. Note that it shows data from all three years of sales results that I selected.

Using the Left Query to Drive The Results

In Desktop Intelligence (more details below) this is as far as I can go. I don’t have any options that let me control the merge process. In Web Intelligence I do. What if I want my first query (left) to drive the results? All I have to do is open up each merged dimension object and use the dimensions that come from the left query. If I do, here’s what the block looks like.

What is going on here? The left query had 2007-2008 data only. I see annual data for both of those years for all three resorts. However, the 2008 rows are missing the Number of guests values. They don’t exist, because the right query only goes up to 2007. In this case I have told Web Intelligence that my “left” data is more important, and to show right data only where it exists.

Using the Right Query to Drive The Results

Next I will turn things around and use the “right” dimensions instead. Here is the block that results from that operation.

Now my revenue column is sparse (has missing values) while the Number of guests column is completely populated. That is what I expect given that I am using the right dimension objects this time.

Those are the three options that I have in Web Intelligence. I can use the merged dimensions (the most typical requirement) or I can use the dimensions from a specific data provider. If I compare the three blocks (merged, left, and right) I can see exactly how Web Intelligence is working. How does Desktop Intelligence work?

Merging Data in Desktop Intelligence

In my example Web Intelligence report I have selected the Resort object in two different queries. After the automatic merging process has run I end up with three different ways to reference the Resort values on a report. Specifically I can reference Resort(left), Resort(right), or Resort. The last option is the merged result. In Desktop Intelligence I am missing that last choice. Is that a problem?

I have a screen shot that shows what the merging process looks like in Desktop Intelligence. One complaint that I have is that I have to click on each dimension object individually to see if it is linked or not. In the screen shot shown below I am only showing that the two year values are linked.

Next I notice that I have no way to refer to the merged result set as I do in Web Intelligence. I have to pick either the left or right value. Does it matter? It turns out that it does not, as shown here. I have selected the right version of the Year object, yet all three values are showing up.

In Desktop Intelligence the results from the left and right queries are merged together. Every value from the left query appears on the right, and every value on the right appears on the left. It is a union of the two data providers. As a result of this union operation, it doesn’t matter which side I pick. I can swap the left and right dimension values and still get the same result. It is always a full outer join.

Is it possible for me to reproduce the way Web Intelligence behaves in Desktop Intelligence? In the example I am using today, the answer is yes. However, I need to look at the measure values instead of the dimensions since I cannot alter the merged dimension behavior. If I want to show the left data, I can create a filter so that only those rows that include a Revenue value will be displayed. Here is how that filter looks.

Dimension values will never be NULL in Desktop Intelligence because of the way the values are shared as a result of the merging process. If I can identify a measure, as I did in this case, I can essentially create a left or right block based on my requirements. The process is not intuitive, nor is it effective in every case.

Extend Merged Dimension Values

So now to the point of this post: what if I would like Web Intelligence to act more like Desktop Intelligence? That’s exactly what the setting “Extend merged dimension values” does for me. Remember in my description of Desktop Intelligence I said that merged dimensions result in a union, where every value from my left query appears in my right dimension and vice versa. That means that I will always see data from either side, no matter which dimension I pick. Here is one way to invoke this setting. I am right-clicking on a report tab and I see this menu.

After selecting Document Properties the following menu appears on the side of my document.

The setting is off by default. What happens if I turn it on?

Here is what my left block looked like before the setting was turned on.

And here is what it looked like afterwards.

And here is the right block.

They look exactly the same. And that’s how Desktop Intelligence would have behaved in this situation. Each dimension has been extended to include the values from the other. My left year includes right values, and my right year includes left values. I have recreated the Desktop Intelligence behavior in Web Intelligence. I am not sure this is a good thing, but I can do it.

Conclusion

If I only use the merged dimension (meaning I use Resort instead of Resort(Left) or Resort(Right)) then this setting doesn’t do anything for me. The results will be the same with the setting on or off. But if I need to pick the specific values from either the left or right, this setting allows me to recreate the behavior found in Desktop Intelligence.

One caveat: this setting is a document setting, not a report setting. That means if I turn it on (or off) it affects the entire document. That may or may not be an issue. It all depends on what I am trying to do.

This post was written in response to a specific topic on BOB. I started to write the answer there but realized it would make much more sense with screen shots and more text.

Filling Out A Calendar

My report requirement states that all months – whether they have sales data or not – must appear in my output. Because Web Intelligence cannot make up data to fill out the missing months, I need to have an alternative source. In the first post in this series I showed how to do that with a personal data provider as I built out the data in a spreadsheet. This time I am going to show how to do the same thing using only a universe. In order to provide a complete (clean) solution I am going to have to make a new object in my universe.

The first step is to create a query that returns some data. I am going to use the same data that I started with on the last post, which is six months of sales revenue data from the eFashion database. I am also limiting the data to the year 2001 in order to match the example from the last post.

Next I will create a second data provider from the calendar table that includes all of my month values. Since this query has no restrictions (conditions) and no measures (no joins to any fact table) it will be guaranteed to return every month. At this point in the last post I detailed how to merge (link) two data providers together. This time that step is not required. That’s the first advantage of using universe data to perform this step: the dimension objects that are shared across my two queries are already merged (linked). When I used personal data, I had to do the merge step myself.

Once I drag the merged month into my report block I will see all 12 month names. As I did last time I will also replace the existing Month Name detail object with the one that comes from my calendar data provider. The results from these operations are shown in the screenshot below. The “Discontinued” value comes from a custom format used in the eFashion universe. The universe was designed with the assumption that if any cell was empty (as opposed to zero) then there were no sales for that item, and it must have been discontinued.

I would like to insert a zero value into those “Discontinued” cells. I could make it look like it has a zero by changing the format. But if I want an average of that column to be impacted by the fact that I had zero sales for six months out of the year, I need to have an actual zero in the cell, not just a custom format. It’s at this point that I have gone as far as I can without talking to my universe designer.

Creating and Using Placeholder Objects

As a universe designer I can help solve this problem. I might create a special class called “Placeholder Objects” that includes new objects used to solve this type of problem. (I have also seen these objects called “dummy” objects because they are only going to work in very specific situations. They’re not “smart enough” to be used by themselves.) When I used personal data to solve this issue I created a new column in my spreadsheet and put a zero in for every month. In my universe I will create just one object with a zero value in the select clause and nothing in the tables. Note that this object will not parse; that is expected.

Since this object does not have a table associated with it I will need to make sure that it’s used with at least one “real” object in a query. As a universe designer I can’t enforce this. But if a report writer tried to use this object by itself the query would not run. I have seen some folks suggest that objects like this should be pointed to the DUAL table (in Oracle) or something similar. I disagree. The only benefit to doing this is that the object will parse. But at the same time I am fixing that issue I would be introducing an entirely new set of issues because the DUAL table isn’t typically going to join to any other table. It’s much more effective to leave this object in a state where it won’t parse than to deal with all of that mess.

Rebuilding the Query

Once I have created my dummy object and exported the updated universe I can fix the problem. In fact I can fix it a couple of different ways. First, I could mimic the solution from the last post by adding the zero object to my calendar-only data provider. Since I have already covered all of those steps I thought I would show a different technique for this post. I am going to use a UNION query.

A union requires that all of the queries (you can have two or more) match. Each query has to include the exact same number of objects, and the object types (character, date, number) have to match as well. For the scenario I am using for this post each query will have three objects. Both queries will include the Month and Month Name. The first query will also include the new Zero object. The second query will include the regular revenue object. Here are the two screen shots showing what this query looks like.

Query Panel Union Part 1

Query Panel Union Part 2

Notice that the first query does not have any conditions at all? The second query includes the condition that I created to drop out months (it only shows months with an “e” in their name) as well as a limit to show data only for 2001 as mentioned earlier. When I run the combined queries I see this output.

At this point the data looks great, and it was obtained with a minimal amount of fuss. All of the months show up as required. There are no merged dimensions here since everything is returned in a single data provider. However, there is one item that needs to be fixed. Check out the column headers… they’re not exactly “user friendly” at this point. What happened? It turns out that column headers from a union query are taken from the first pass of the data provider. As a result my column header says “Zero” instead of “Sales revenue” as I would expect. Fortunately Web Intelligence (the Rich Client in this case) makes this process extremely simple. All I have to do is open the query panel and drag “Query 2″ in front of “Query 1″ in the union area of the query panel and rerun. After rerunning the query I am left with a report block that is missing a measure. This is okay, because the object named “Zero” has been removed from the query, and a new object “Sales revenue” has appeared. All I have to do is drag that new measure onto the block and I see these results.

Notice that I have added an Average calculation in the footer to show that the zero values are being included during the calculation. The result obtained via this process is now essentially identical to that shown in the first post about making up data.

Conclusion

So far I have shown two different techniques that can be used to “make up” data for a report: personal data and universe data. Which is better? As with many questions the answer is “It depends.” Personal data has an advantage in that it can be built on the fly by a report user and does not require any universe maintenance. Universe data has an advantage in that the extra objects are reusable and are available to anyone. With proper training users can build queries that can include extra data that does not exist in the database.

Personal data requires me to know how to merge the results. With universe data the merge step (as long as I use the same objects) is performed automatically.

In this post I also showed two different techniques for getting the “filler data” into my report. With personal data I have to create two data providers and merge the results. With universe objects I have the option of creating a single data provider using a Union technique and my placeholder object.

The placeholder measure object that I created is not limited to this particular report. I can use (and reuse) it for any other report with similar requirements. Since the object is not tied to the calendar (or any other dimension) I can use it anywhere. It can be used to merge data providers or in a union query. I can create similar “dummy” objects for dates or text fields as needed. Ultimately I can decide if I want these special objects to be visible to all users or to a selected subset of “power user” report writers using the standard security process.

Next Time

For Part III of this post I am going to create a more complex example where data might be missing from more than one dimension (instead of just month as I have used in the last two posts.) I am also going to show cross tab and a section output because each of those can cause some interesting challenges as well. Maybe I’ll even do a sectioned cross tab.

Related Posts

• Making Up Data Part I: Personal Data Providers

I have only six months of data, but I have to show all twelve months in my report. How can I show the missing data?

The trick is to understand that Business Objects does not make up data. It just doesn’t have that capability. And you can’t edit the report directly in order to add those extra values to the result set. However, there are a couple of tricks that I can share that will help solve this issue.

Special Note to Blog Readers: This post is fairly long because it has quite a few screen shots to illustrate the process. As a result it will be presented in several pages. When you get to the bottom of a page just click on the next page number from the links provided to continue reading the post. Since this is the first time I have used this feature I thought I would warn you about that. Now on with the solution.

Using Personal Data

One of the easiest ways to make up data for a report is to use a personal data provider. This technique can be used in all versions of Desktop Intelligence and also in the newer “rich client” version of Web Intelligence. While the specific steps are unique to each tool the general concept is the same, so I will only show how to do this using the rich client. Screen shots for this blog post were taken from version 3.0 of the Web Intelligence Rich Client.

The first step is to determine what is missing and therefore what you need to make up as filler information. It’s also important to know that the format of your personal data has to match what is coming from your enterprise data source or the data won’t merge properly. For this scenario I will show some data coming from the eFashion database. I am going to restrict the data to months that include an “e” in their name, just to make sure that I have holes in my data. If you’re wondering, that includes February, June, September, October, November, and December. Why did I pick that? No reason, really, just thought it would be fun.

After applying that condition (along with another to restrict to one specific year) I see the following data on my report.

In the eFashion universe the Month is a dimension object and is a numeric value. The Month Name is a character string containing the month name spelled out (October rather than just Oct) and is configured as a detail object. This is important. When I want to merge two different data providers together I have to perform that process (the merge or link step) using only dimension objects. I cannot use detail objects. That means my personal data provider will have to include the month number as a dimension, and it should also include the month name as a detail.

After building up this query I have six months of data, as described in my original scenario. To meet the report requirements I have to show all twelve months, including those that don’t have sales that meet my current criteria. How can I make this happen?

Making Up Data

I am going to use a spreadsheet to make up my data, but I could just as easily use a delimited text or CSV file created with any text editor. The trick is that I have to structure my personal data so that it matches what I already have from the universe. To solve the scenario I am currently working on that means that I need the month number and the month name in my personal data. Here’s how that looked when I was done.

I saved this data on my computer as a standard spreadsheet file. I could also have used a comma separated values (CSV) or other structured text file just as easily. The next step is to import this data into my document.

Adding Personal Data

In order to add personal data to my document I open the query panel, right click on the current query tab, and select the option to add a new data provider. After I do that I see this dialog box.

After making sure I select “Local data source” I move on to the next step, which is to browse my computer and find the spreadsheet file I saved earlier. Once I find the file I am presented with a few configuration options as shown here.

The filename is there, of course. Since I am using a spreadsheet I can pick which sheet from the workbook contains my data. I can also determine whether to use a named range, to enter the range myself, or to let the client try to figure out where all of the data is for me. For this simple case I mark “All fields” and let the client do just that. Finally, I confirm that the box for “First row contains column names” is checked and click the Next button. Here is what I see at this point in the process.

There are a couple of problems with this result. The client is trying to be helpful, but it’s making the wrong decisions for me. The Month column contained all numeric values so it was imported as a measure. The Month Name contained character data so it was imported as a dimension. Both of these assumptions are wrong. I need the month number to be a dimension so I can link it to the value I already have, and I need the name to be a detail in order to match the existing structure on the report. Fortunately, since this is my own personal data, I am allowed to make changes to the configuration.