Quoting from their site, the purpose of the project is to “is to improve the design and use of spreadsheets by individuals and organizations.” Their research focuses on “identifying best practices in spreadsheet development (design, testing, documenting); procedures for implementing, modifying, sharing, and archiving spreadsheets; and organizational policies relating to standards, training and quality control.”

Even though I have been around spreadsheets for two decades in large and small organizations, and can say “been there, done that” regarding many spreadsheet application scenarios, there’s a lot I haven’t seen. Looking at the survey results, there are several surprises where the survey shows data that’s not what I would have guessed.

This is the link to the project site: http://mba.tuck.dartmouth.edu/spreadsheet/index.html

The surveys were conducted in 2005-2006, with about 1600 respondents. Below are some selected questions from the survey.

The data (and the chart) probably came from and Excel spreadsheet on someone’s desktop. When the chart has to be reproduced for next month’s presentation, it can only be created by the author because the knowhow required is not recorded in some metrics document or standard operating procedure.

The same can be said about discussions in a 10K or 10Q filing, or a presentation to Wall Street analysts. How can an auditor validate those numbers if he doesn’t have access to the specific spreadsheet? How can a CIO, overseeing a vast business intelligence infrastructure, reproduce the same numbers in case the numbers are challenged?

Excel’s power and ease of use leads to the creation of “mini BI silos” where data from operational systems and even BI reports is further combined, analyzed and reported. The informality of these BI silos causes auditors and CIOs a lot of grief. They cannot put the kabbash on these Excel-based solutions because they aren’t able to provide a better alternative.

Excel is a fantastic tool. It is easy to learn, incredibly flexible and has powerful data management and business intelligence features. Our clients have sophisticated enterprise systems such as SAP, Oracle and Microsoft Dynamics that process massive volumes of transactions. Our clients also have the leading business intelligence tools in house. It is not uncommon to come across Teradata, Cognos, Business Objects and Microsoft BI products in the most environments. Yet, despite all the technology available within the enterprise, the employees continue to use Microsoft Excel.

One thing I find amusing is that a user will generate a report in Cognos or Business Objects, then immediately export the data from the report to Excel. Once the data is in Excel, it will be filtered, sorted and analyzed in a pivot table. Graphs will be produced and pasted into PowerPoint presentations. Often, additional columns are created alongside the exported columns to compute ratios or display categories. These new columns will then take a life of their own and will be used to make tactical and even strategic decisions.

The chain between the ERP system and Excel on the desktop can include a replicated database, an Operational Data Store, various ETL reporsitories, a dimensional data warehouse, cubes and the report. Borrowing a term from the telco industry, Excel is the “last mile” between decision workers at one end and the large enterprise transactional systems at the other.

My prediction is that Excel will continue to be an important part of the BI landscape. Microsoft continues to add impressive new features to it, and along with PowerPivot, it will fills a space that other enterprise tools cannot.