Aug 01 2013

Is External Data Always Good?

Categories: Rants Dave Rathbun @ 6:43 pm

Note to readers: I started this post back in 2011. After taking a break from blogging I am going back and looking through some of my old drafts and seeing what might still be current, and what has expired. I thought that this one merited some additional attention and publication, even though some of the notes are from two years ago. — Dave

SAP had some fun on the BI 4.0 launch in New York a while back. For years SAP (and other vendors) have been talking about their ability to bring in external data from various social medial sources. Two SAP presenters at the launch event took a vote via Twitter as to which tie would meet the “Scissors of Destiny” at the end of the session. (Steve Lucas made an impassioned plea to save his tie, which he said was a gift from his wife, versus Dave’s tie which he “… just bought last night.” Steve won, and his blue tie survived.) It was a fun display of technology, but is it really that important? How impressive would it have been if the “fail whale” had picked that moment to make an appearance?

I don’t usually spend a lot of time here on my blog talking about philosophical aspects of BI as I am personally more interested in technical issues and solving problems. But the apparent consensus as to the importance of social media bugs me.

The Internet is a wild place where rules are not always followed. If there is money to be made, then someone will figure out a way to abuse the system. It’s not just the “little guys” either, as evidenced by the way retailer JC Penney apparently took specific steps to trick Google during the holiday shopping season. Again, this was back in 2011.

What do you do with the information?

Does it do any good to listen to what is being said on social media without having an action plan to respond?

Do you really trust an external entity (such as Facebook) to host critical data?

Did you know that you can reportedly buy Twitter followers now? (Seriously, google for “buy twitter followers” and see what you find.)

There are rumors that Sarah Palin got caught setting up a secondary Facebook account, just so she could “like” herself and skew the results shown on her main page. This type of abuse – if performed manually – should have minimal impact. However it is apparently far too easy to set up bots that can be tasked to perform the same sort of task. In fact there are companies that you can legitimately hire (as opposed to going underground) to do this for you. One term I came across while researching background for this article was quite amusing: hactivist. :)

Is there a point to all of this rambling? Not really, I guess. Or if there is, it’s that despite SAP and everyone else appearing to really want to make social media relevant, I find myself asking why is it so important?

Human behavior – online or not – often boils down to risks and rewards. The problem is that rewards can inspire the wrong behavior. I talked about this in a guest blog at The Decision Factor: Lessons in Business Intelligence: Be Careful What You Wish For. The cost of setting up a web site today are extremely minimal. The ability to generate advertising revenue, however, is also very minimal. Suppose that it costs $10 a year to host a site and it makes $0.50 per month in revenue. It’s hardly worth doing, right? But what if you scale that up. Now it costs $1,000,000 to set up the sites, but you’re generating $50,000 per month in income. I can’t find a link at the moment, but there was some guy that was making millions of dollars buying expired domains and putting junk content on them.

By one estimate, the Internet will soon have more garbage than valuable content! Some might say that this has already happened.

That being said, there are certainly valid reasons to consider using social media. The recent (yes, this really is recent) phenomena of Sharknado proved that. ;)

2 Responses to “Is External Data Always Good?”

  1. Comment by Xeradox (Andreas J.A.)

    Nice to see you back posting on your blog, Dave!

    Your thoughts on social media data are definately interesting.
    The other big hype IMHO is BIG DATA, instead of focussing on data mining, predictive analysis, forecasts, correct sampling of data, etc.

  2. Comment by Dell Stinnett-Christy

    It’s good to see you back, Dave!

    And to bring Sharknado and Twitter back into the world of SAP, have you seen this: Sharknado Social Media Analysis with SAP HANA and Predictive Analysis


Leave a Reply

If you want to include formulas or code in your comment, please read my Tips for formatting comments first. Tags you can use are listed below.

XHTML: You can use these tags: <a href="" title=""> <acronym title=""> <blockquote cite=""> <code> <em> <strike> <strong> <sup> <sub> <u>

Confirm submission by clicking only the marked checkbox:


Please remember that comments that are not related to this blog post may be ignored or deleted without notice. If you're looking for help on a topic you have already posted on BOB then please do not repost your question here.