2009 July - MarlabsBlogs.com

RDBMS vs MapReduce Tuesday, July 28, 2009

It is established that RDBMS is the most performant technology available today for processing large scale data. There is no other technology which can provide a better ROI on time for an end user (i.e. developer or poweruser) than SQL. Even the best scripting language cannot compete with –>> select * from xyz. But every dinosaur sooner or later faces an ice age. Arrival of such a force for RDBMS is MapReduce.

http://www.computerworld.com/s/article/9131526/Researchers_Databases_still_beat_Google_s_MapReduce?taxonomyId=173&pageNumber=2

Will RDBMS technology get slayed tomorrow? Certainly not. But when leading researchers have this to say:

“The researchers did allow that parallel databases, which can be set up in large-scale grids that crunch hundreds of terabytes or even petabytes of data, were “much more challenging” than Hadoop to install and configure properly. Loading data into MapReduce or Hadoop was also three times faster than into Vertica, and 20 times faster than the unnamed database, they wrote.”

…and this..

“Since few data sets in the world even approach a petabyte in size, it is not at all clear how many MapReduce users really need 1,000 nodes.”

…well it looks like within next few years, we will witness rise of a new cluster programming paradigm which will mark the beginning of an end for RDBMS. Parallel databases may attempt to establish a new order, but without a new programming model which addresses : (1) scale & (2) failure, RDBMS’s will lose.

First it was mainframe (1960-90), then it was client/server (1990 – 2010), next will be client/cluster (2010–??)

Posted by Srinivasan Balram | No Comments

What is litigation readiness and how do you begin to prepare your organization? Wednesday, July 22, 2009

By implementing a formal litigation readiness program, organizations can be assured that they can produce the right documents within reasonable timeframes, lower overall discovery costs, and speeden case review cycles while streamlining the daily workflow and management of critical business information. The first step in creating a readiness plan, however, is to gain a sound understanding of the readiness concept.

Litigation readiness is most commonly defined as the state of being prepared for all litigation or regulatory inquires which are reasonably foreseeable. What exactly is ‘reasonably foreseeable’ is the real gotcha that many companies struggle to interpret. In some cases, the business plan for litigation readiness programs relies too heavily on document retention policies alone. The reality is that without a more holistic understanding and approach to litigation readiness an organization can leave itself open to unnecessary losses and expenses in the event of litigation.

Under the Federal Rules of Civil Procedure, discoverable items are typically defined as any matter, not privileged, which is relevant to the subject matter in the pending action and can be reasonably produced. In some cases the scope may be further increased by an order for specific disclosure, which means that any class of documents may be deemed disclosable if your opponent convinces the court or the court alone decides that they must be produced. With the enormous volume and velocity with which data moves in today’s business, increased scope presents a significant problem. Couple this with an increasingly narrow interpretation by the courts of what is unreasonable to produce and the true complexity of litigation readiness is revealed. A recent Ferris Research Report estimates that the total number of business e-mails sent in North America alone will surpass 139 million in 2009 and 143 million in 2010. Add to this the ever increasing adoption of Enterprise level use of Web 2.0 communication tools such as Texting, Instant Messaging, Twitter and Social Networking posts and you’re quickly faced with an information explosion of volcanic proportions.

As organizations that have not traditionally had a formal litigation readiness policy in place move towards enacting a game plan, they are finding that early preparation and categorization of tasks can be a daunting measure. To this end, I have prepared the following as a check list of items to help outline key areas where organizations can begin as they develop a framework for a formalized litigation readiness program.

Simplify and document your records policies while creating simple to follow retention schedules
Develop and promote concrete information privacy and security programs throughout the organization
Proactively design a comprehensive litigation hold strategy
Invest in an enterprise level archiving solution and search platform
Design, document, and implement backup maintenance and disposal policies
Document and track audit logs for archive activities
Create a formalized training program to address each facet of litigation readiness as it pertains to IT, HR, Legal, and Operations stakeholders.

Posted by Jeff Schott | No Comments

PPS on its way out Tuesday, July 21, 2009

The big BI news earlier this year was that PerformancePoint (that was launched with much back-slapping) was being sunset. The dashboarding and scorecarding features will be integrated with Microsoft Office Sharepoint (MOSS), but the planning module is effectively being “retired”.

You can view the original announcement here or read about Cidi Howson’s (of the BI scorecard fame) view here.

The impact of this decision by Microsoft is split down the middle. Supporters say that this could help reduce BI costs and help it reach more end consumers. The ABM (anyone but MS) camp insists that Microsoft has given up on the Performance Management battle all together. It is a little bit of both, I guess. It is undeniable that Performance Management is the big wave in BI currently and it shows no sign of ebbing away any time soon. Planning and forecasting are key components of the PM space and by moving away from it (or by integrating this with MS ERP), Microsoft is essentially asking us to use another tool for PM (if you need to). Considering that most of the spread of MS BI is in the SMB sector, I do not expect it to have an impact on market share in the short-term. However, if MS wants to be a market leader in the BI space, Performance Management is not something that it can ignore for much longer.

One of my colleagues also referred me to this blog. It is a good read!

Posted by Rajesh Ramaswamy | No Comments

Challenges in BI Implementations Tuesday, July 21, 2009

All of us will have our own, unique take on this; but hey, a blog is all about opinions!

I think it is imperative to classify the challenges faced by the host company and the solution provider separately and stress that they vary based on the type of implementation and the current state of the host.

From my experience these are probably the top three challenges faced in larger implementations (in no particular order):

Host Organization

1. Resistance to change: Introduction of a BI system changes operational processes and might change business processes. Every individual prefers the status-quo; getting their buy-in (especially when we are dealing with operational BI) is critical. Moreover, embracing a BI system often requires users to get trained on and be familiar with a new set of tools. That decision never wins the popularity sweepstakes. The best designed and implemented BI project is a failure if user adoption is low.

2. Master Data Management: Integrating and/or reconciling existing, disparate master data across various silos is a key challenge for most large organizations. It may be overwhelming to drive consensus on this, but it is imperative to keep moving forward in this exercise.

3. Justifying RoI: BI business benefits are difficult to quantify. How do you prove that an increase in sales is because of the better strategy adopted from the insights your BI system provided, and not because your sales personnel became “super” sales personnel? Quick wins and consistent visibility of key metrics to track project progress is a major driver of success

Solution Provider

1. Estimating for a BI implementation: Accurately estimating a BI project is challenging because the scope is extremely difficult to nail-down and expected output is often never known till it is actually seen. Prototyping plays a critical role in the entire exercise. There is no industry standard methodology, though the “agile” approach tends to work well and ETL is easier to estimate than reporting and analytics. I personally use a complex excel template that relies on several “weights” provided by past experience. But, on the whole, making an effort or schedule estimate for implementing a BI system running out of an EDW can be extremely hazardous. It takes a brave person to make a “fixed price” bid!

2. Data Quality: Everyone knows about this, yet it continues to be a critical bottleneck. Often the only solution to this is to change the way the transactional system handles data, but that is easier said than done. Anyway, garbage in is garbage out. Enough said.

3. Contain scope, yet be flexible; deliver quick wins, but don’t lose sight of the big picture: It is necessary for the provider to wear many hats at the same time; ensuring that varying objectives are met is critical to this. The client CIO, COO and CFO may have divergent, even contradictory objectives yet the solution provider has to meet them all! Be prepared for lots of prototyping and throwing away developed and working components.

Posted by Rajesh Ramaswamy | No Comments