Aug 22 2014

Yes, Virginia, You Can Refresh One Data Provider At A Time

Categories: General, Report Techniques, Web Intelligence Dave Rathbun @ 11:45 am

For a while now I have been whining about the fact that there were features dropped from XI 3.1 during the upgrade to BI4, including how the dimension merging process works. Another gap? In XI 3.1 we had the ability from the query panel to run only a selected data provider, leaving the others alone. When working on complex multi-query documents this could be a big help, especially if some of the data providers had longer refresh times.

A few days ago I was grumbling about this yet again and discovered a way to refresh a single data provider! It’s not perfect (nor was it the most obvious workflow for me) but it does work. I had one data provider that was scanning a huge multi-billion row fact table. To supplement this data I needed to run an additional query against an Excel data provider. I had to make several changes to the Excel file in order to help my data match up correctly, and each time I updated the XLS I had to refresh the entire document in order to see the changes… which was annoying and time consuming. Now I have a solution.

Note: Since I’m talking about joining with Excel, clearly I was using the rich client application. However the same technique outlined below works online with multiple universe data providers as well. Continue reading “Yes, Virginia, You Can Refresh One Data Provider At A Time”


Jul 08 2014

Airlines Could Save Millions in Fuel Costs By Providing Everyone An iPad

Categories: General Dave Rathbun @ 11:30 am

There’s an interesting article over at fivethirtyeight.com that discusses various ways that airlines could save money. Like many airline profitability studies this one focuses on things that Southwest Airlines does to remain in the black while other airlines seem to go through bankruptcy every five or ten years. For example, Southwest does not have any in-flight entertainment systems. That saves money in a whole variety of obvious ways (they don’t have to pay for the systems up front, and they don’t have the on-going costs of keeping the systems running) but also in less obvious ways.

Southwest runs about 1.6 million flights every year. At the most basic level, they are in the business of moving cargo (human or otherwise) from point A to point B, therefore one of their primary costs is fuel. The more cargo they carry, the more fuel is required. When I carry my cell phone I hardly know it’s there. Southwest jets have an average of around 140 seats. If I assume 100 of those are filled with passengers with cell phones, the weight becomes a bit more noticeable. Imagine loading up a box of 100 cell phones. :) and then repeat that for 1.6 million flights and the extra cargo weight adds up! In the article they crunched the numbers and determined that Southwest probably pays $1.2 million dollars in extra fuel costs just to carry all those phones. If passengers carry laptops that would add almost $21M in fuel costs. Entertainment systems – if they were present on Southwest – would add almost $40M a year in additional fuel costs. I found it interesting that entertainment systems apparently weighed more than laptops, but then I realized one factor was that planes are not always full. A laptop won’t fly by itself, but an entertainment system might.

Which brings me to what I thought was one of the more interesting propositions made in the article. Airlines that currently offer in-flight entertainment systems could save a ton (pun intended) by eliminating those systems and giving everyone an iPad instead! They could save even more by allowing passengers that bring their own iPad free access to the in-flight media server. Content could be uploaded to each plane while they’re loading passengers. The airline wouldn’t need to provide high-capacity iPads since everything would be viewed from the server, so the 16GB model would be plenty. The cost of a 16GB iPad currently hovers around $400, so they’re not cheap, but given the weight savings (and the fact that iPads could be loaded on an as-needed basis rather than always being on board) the fuel savings would likely more than make up for the added cost.

Using Southwest’s network as a proxy for similar-sized airlines carrying embedded in-flight entertainment systems, we found that fuel costs to carry these systems are approximately $39.7 million per year. When compared with installing embedded systems in the seats, simply handing everyone an iPad when they stepped onboard could save about $32.7 million per year in fuel costs.

Plus I have to imagine if Southwest Airlines called up Apple and ordered 100,000 iPads they would get a better price than $400. ;)

Related Links


Jun 24 2014

Did Florida State Win National Championship By Using Big Data?

Categories: Rants Dave Rathbun @ 9:53 am

Florida State University won the 2014 college football national championship game this past January in a thrilling come-back victory over upstart Auburn. Did big data help?

It turns out that a few years ago FSU started using GPS devices on their players during practice. Each device collects around 1,000 data points every second. The devices are downloaded and analyzed by a special group of coaches and analysts who report back to the head coach. Right now the team is not allowed to tag players during games, so they only use them during practice. Did they help?

One of the things that always amuses me is how often reports — based on big data or otherwise — reveal something we should already know. Two years ago the FSU coaching staff was getting frustrated with the lack of game-day performance from one of their star receivers. It turns out that because he was so good, the coaches were constantly asking him to run extra routes during practice to demonstrate to the other less experienced receivers on the team. By the time the game day on Saturday arrived, he had run much farther than anyone else, so of course he was tired!

Greene was Florida State’s most refined receiver, so when Fisher would grow agitated with poor routes or dropped balls by other players, he would ask Greene to illustrate the proper form. Again and again, Greene would run a route or catch a pass, and his workload mounted. The GPS device offered clear-cut data that showed Greene was simply doing too much.

After tagging the players with the data collection devices the coaching staff was able to recognize this (duh) and make sure he got the proper rest during the week. The player responded with a career best year last year during the FSU championship run.

Related Links


Jun 07 2014

Business Intelligence Lessons from Star Trek – Part Two

Categories: General, Rants Dave Rathbun @ 12:57 pm

Author Note: This blog post was originally a guest entry at The Decision Factor, a site that appears to have ceased publication. I have reproduced it here. It was fun to write, and I hate to see good content go missing. The original post was published in 2012 but I believe the content is still relevant today.

In my first Star Trek post, I explored two lessons learned from Captain Kirk’s leadership skills and how we could apply them to business intelligence. Today, I’ll cover three more lessons.

Be Part of the Away Team

Captain Picard of Star Trek: The Next Generation rarely joined the away team, leaving that role to his Number One, Commander Riker. Kirk operated with far more freedom. Whenever there was an opportunity to beam down to a new planet, he was there. His leadership style demanded that he lead from the front.

I think the corollary with business intelligence work here is obvious: the success or failure of our systems depends far more on establishing correct requirements than on the actual implementation. By being part of the away team, we can be directly involved. Lack of user involvement and poor requirements gathering are generally two of the top reasons projects of any kind fail, and business intelligence is not immune to this problem.

Play Poker, not Chess

In Star Trek, there were numerous times when Captain Kirk and his crew were in a bad spot. In one episode, The Corbomite Maneuver, the Enterprise and her crew were experiencing first contact with a new alien race and it wasn’t going well. At one point, Spock had to tell Kirk that they were out of options and the alien had backed them into a corner from which there was no escape. In chess terms, checkmate. Spock, as a character, was the very embodiment of chess: cold, logical, and process driven. It was natural that he would see things this way. But Kirk was different.

Capt. Kirk: There must be SOMETHING to do. Something I’ve overlooked.
Mr. Spock: Chess: When one is outmatched the game is over. Checkmate.
Capt. Kirk: Is that your BEST recommendation?
Mr. Spock: I’m s… I regret that I can find no other logical alternative.
Capt. Kirk: Not chess, Mr. Spock. Poker!

Kirk took inspiration from the game of poker and bluffed his way out of the situation. How can this possibly apply to business intelligence? We don’t want to lie to our business partners, do we?

Perhaps we do – but in a good way.

I often come into contact with people or business processes that have been rigidly followed for many years. To the mind of the many, there’s no other way to do that particular process. In order to convince them otherwise, sometimes we have to bluff. I like to call this a demo. ;-) In order to make a difference, we have to get past that initial resistance; and sometimes a well-placed bluff demonstration helps us along that road.

Blow Up the Enterprise

Sometimes there’s no way around it. A system that’s well known, trusted, and loved just isn’t cutting it any more. The original Enterprise made it through several years of TV shows and a couple of movies before ultimately meeting her end. As much it pained Kirk to let her go, he realized that in order to move forward, the Enterprise had to be sacrificed. (I may have shed a tear or two myself.)

Legacy systems can often inspire the same attachment, but at some point we have to blow them up so we can move on. “It’s always been done that way” is a poor reason to keep a system around when there are so many new and exciting technologies that have come to market recently. Don’t be afraid to blow up the Enterprise to move forward.

But I think the overwhelming lesson to be learned from Star Trek is don’t wear a red shirt to work.


May 26 2014

Business Intelligence Lessons From Star Trek

Categories: General, Rants Dave Rathbun @ 12:55 pm

Author Note: This blog post was originally a guest entry at The Decision Factor, a site that appears to have ceased publication. I have reproduced it here. It was fun to write, and I hate to see good content go missing. The original post was published in 2012 but I believe the content is still relevant today.

While the touch-panel displays on Star Trek: The Next Generation certainly foreshadowed the iPad, the focus of my discussion today is around another blog: “Five Leadership Lessons from James T. Kirk,” published by Forbes Magazine.

While Captain Kirk had his flaws—making his character all the more interesting—he certainly was the unquestioned leader of the Enterprise. I enjoyed reading the article and think the specific lessons outlined also apply to business intelligence.

I’ll share the first two with you today and save the others for a future post.

Never Stop Learning

When Kirk ran the Enterprise, the Federation was much younger. Their mission was to seek out and explore, encounter new civilizations, and learn from them. The opening monologue said it all:

Its five-year mission: to explore strange new worlds, to seek out new life and new civilizations, to boldly go where no man has gone before.

Sounds like good advice for business intelligence professionals too, right?

In order to deliver effective results, we have to first understand the business. Whether we work for a large company or a small start-up, knowing the business (and the data) is the first step towards providing business value. It doesn’t help to design solutions in a vacuum that the business won’t understand, won’t use, and doesn’t need.

As a business intelligence professional, I look forward to those times when I get sent to the far reaches of our enterprise (see what I did there? ;-) ), so that I can directly observe business processes in action. The lessons learned might not provide an immediate benefit. It might be months or even years, before I can put something I have learned to good use. What’s important is that I do interact with the business and learn from that experience.

Have Advisors with Different Worldviews

Captain Kirk was blessed with a diverse crew, not only on the bridge of the Enterprise but throughout the ship. This was no accident. James Roddenberry, the visionary behind what ultimately became one of the biggest TV and movie franchises of all time, intended this to show how a diverse crew of different ethnicities, genders, and planetary origins could act together in towards a common goal. Spock and Dr. McCoy could hardly have been more different, yet each was able to provide valuable input to Captain Kirk’s decisions. They each supported his role as their captain and leader in their own way, often in direct opposition with each other.

In the business intelligence role, it’s just as important for us to seek out and explore diversity. Before designing a system, I need to talk to everyone who might use the system and gather their input. Getting only one side of the story could lead to incomplete requirements, which in turn leads to wasted time and money. In this day and age (much less the 23rd century), we can’t afford to do much of either.

Stay tuned for more business intelligence lessons from Star Trek

In a few weeks I’ll return with the final three leadership lessons from Star Trek that have implications for business intelligence professionals. In the meantime, I wish you success in exploring the world of business intelligence.


May 23 2014

BI2014 Report Part 1

Categories: Conferences Dave Rathbun @ 2:20 am

I spent yesterday at the BI2014 event and had a very productive day. I delivered a presentation that compared Universe Designer to Information Design Tool and discussed some of the pros and cons of switching or converting universes. It was a small crowd, but I did get to meet Miss Universe from BOB and also Raphael Branger who delivered an excellent talk on various ways to help select the right BI tool. I have also had several folks comment on my LSU backpack. ;) I use the same backpack for almost all of my conferences because it’s quite likely to be the only one there, making me easier to find for folks who I haven’t met yet.

I attended a session on Mobi yesterday that was encouraging. Some of our projects that have tried to use Mobi have observed some performance issues, and apparently that has been addressed for the next (6.0) release of Mobi. The folks that build that tool are in an interesting situation… they have to try to take content that wasn’t necessarily designed for mobile device and render it there. On complex documents (those with lots of different blocks and charts, or even just crosstabs) the rendering process became complex. From what was suggested during the talk the Mobi folks have figured out a different way to interact with the Web Intelligence API that should show noticeable improvement. I’m looking forward to seeing that in action.

I also attended a session that was supposed to be an introduction to analytics but turned instead into a review of the KXEN acquisition that was made last fall. The presenter suggested that KXEN rendered most of what he was going to share obsolete because a user doesn’t need to know anything about the underlying analytics anymore. At some point I would still like to get a better understanding of the concepts.

I haven’t done much of anything outside of the conference yet, simply because I only flew in Wednesday and the jet lag is still affecting me some. It always seems to be easier to go West – even really far West such as Melbourne – than to head East. Oh well, it will be easy to go home. 8-)


May 07 2014

It’s So Nice To Be In Nice

Categories: Conferences Dave Rathbun @ 8:09 am

I’m going to be at the SAP Insider BI 2014 event in a few weeks in Nice, France. It’s my third time travelling to the “old continent” to present at a BI conference, but my first in a long time. If you’re in the neighborhood, please stop by and introduce yourself!

BI 2014 May 21-23 Nice, France

If you’re as old as I am, you might recognize the blog title was inspired by a song from the band Gallery


May 06 2014

Lessons in Business Intelligence: Be Careful What You Wish For

Categories: General, Rants Dave Rathbun @ 12:35 pm

Author Note: This post was originally a guest post published on “The Decision Factor” blog. Since that site seems to have disappeared I am re-posting it here. The original publication date was November 1, 2012, but I believe the content is still relevant.

What’s the purpose of a business intelligence (BI) dashboard? It’s not just to look sexy. The primary purpose of a dashboard is to convey information. A secondary purpose is to inspire a behavioral change based on the information that’s being conveyed. Nobody wants to be “red” on a dashboard reviewed by executives, so they’ll change their behavior in order to get into the “green” area.

But humans are a creative species. What happens if their behavior changes in an unanticipated way?
Continue reading “Lessons in Business Intelligence: Be Careful What You Wish For”


May 01 2014

Big Data Is Hard To Define… and Vulnerable

Categories: Rants Dave Rathbun @ 12:57 pm

Stephen Few weighed in on what is the proper definition of big data yesterday, and it’s an interesting read. If you don’t want to click through, I will summarize in one sentence: “Big data is nothing special, it’s just data.” Obviously Stephen’s opinions have not stopped (and won’t stop) people from using the term.

Next up on my blog reading list this morning was a trip to FiveThirtyEight.com. The headline article this morning was titled, “The Story Behind the Worst Movie on IMDB.” I’m guessing that IMDB doesn’t really qualify as “big data” as they have “only” 2.8 million titles in their database. :) But the story wasn’t about big data, it was about the worst movie in the database as determined by public rankings. I would have expected the soundly panned “Battlefield Earth” (and it was one of the worst with an overall rating of 2.4), the unfortunate Halle Berry stinker “Catwoman” (3.3) or perhaps even the Paris Hilton vehicle “The Hottie and the Nottie” (which I’m somewhat ashamed to admit I even knew about and brings in a lowly rating of 1.8).

It turns out the worst rated movie was not any of these, but instead was a Bollywood production called “Gunday” which has a rating of 1.4. Over 91% of the posted ratings are one star! What happened? Was the movie really that terrible?

For the full story, click through to the story on fivethirtyeight. In summary: an entire country decided they didn’t like the movie and decided to do something about it.

…the movement has since become an online alliance of bloggers focused on protecting Bangladesh’s history and promoting the country’s image. That includes protesting “Gunday,” because of the film’s reference to the Bangladesh Liberation War as the Indo-Pak war. In its first 11 minutes, the movie claims that India alone defeated Pakistan, and implies that an independent Bangladesh was simply a result of the fight.

What happens when an entire country decides that a movie is bad? The movie becomes perceived as historically bad. More from the article:

For Paris Hilton’s “The Hottie & the Nottie” — currently rated second-worst of all time — to take over IMDb’s bottom spot, the next 41,000 voters would have to give it a 1.

Last year I wrote a blog post titled Is External Data Always Good?. This is one more example of how social media / crowd-sourced data can be skewed by a concentrated effort. Is “Gunday” really the worst movie of all time? Probably not. Most professional critics were not nearly as harsh, especially when compared to Paris Hilton’s effort. One user reviewed Paris’s acting by saying, “Paris Hilton’s acting made me lose braincells.” The reviews on IMDB were not spammers; they were unique individuals. They just happened to be part of a focused effort to trash a movie they perceived as historically inaccurate. (Please note: I am not making any assessment as to the accuracy of the film. I am far from an expert in that area so I’m neither endorsing nor rejecting the movie.)

Ultimately I think the article from FiveThirtyEight wraps it up the best.

Crowdsourcing can be a tremendously powerful way to get a consensus understanding of the world. Because the sample size is so large, there’s an assumption that whatever it yields is robust and true. But even with oversight, aggregated rankings are subject to unforeseen biases. Crowds are always big — but they’re not always wise. Sometimes it’s impossible to control which crowds are being sourced.

Big data is just data. But you still have to understand where it’s coming from in order to benefit from it.


Feb 26 2014

Big Universe + Security Profiles = Slow Query Generation

Categories: Rants, Report Techniques, Web Intelligence Dave Rathbun @ 6:01 pm

The actual origin of the concept of a “red herring” is unknown, but that doesn’t stop it from causing grief while trying to diagnose a performance issue. If you are not familiar with the concept, a red herring is something that initially appears to be relevant but ultimately is proved to have nothing to do with the actual issue. It’s a popular technique for mystery novels… and in tech support calls.

Case in point: Today I had to help someone who was wondering why their report took over thirty seconds to display a prompt window when there was only one prompt in the document. Clearly it was a prompt issue, right? Or something related to the list of values definition for that object? Continue reading “Big Universe + Security Profiles = Slow Query Generation”


Next Page »