DW 2.0: The Architecture for the Next Generation of Data Warehousing (Morgan Kaufman Series in Data Management Systems) [Englisch] [Taschenbuch]

Awareness of this book arose following my recent reading of a white paper on Data Vault data modeling by Dan Linstedt that a recent client of mine had suggested. And although I was not impressed with that white paper, what I found intriguing is that Lindstedt quotes Bill Inmon as saying that "the Data Vault is the optimal choice for modeling the EDW in the DW 2.0 framework." Thus the acquisition of this text by Inmon. Almost everyone vaguely familiar with this industry space is probably familiar with Bill Inmon and Ralph Kimball. What is interesting is that Inmon, the "Father of Data Warehousing", is credited alongside two other individuals with writing this text. It is not transparent as to who actually wrote most of the content for "DW2.0", but what is quickly apparent is that most of the statements contained in the book are generalities, and the vast majority of the diagrams are deplorable, consisting mostly of inferior clip art that adds little to nothing to the discussion. Most of the material is presented in a theoretical manner with very little practical substance. This reviewer hesitates to even recommend this latest Inmon effort to client management. Even outside the domain of data warehousing, there seems to be something amiss with what the authors attempt to present. For example, chapter 6 consists of a 17-page discussion on "methodology and approach", and for the first 7 pages of this chapter, the authors discuss the spiral, waterfall, and iterative methodologies. Keeping in mind that there are various interpretations for each of these methodologies (see my reviews for "Agile & Iterative Development" by Larman and "Balancing Agility and Discipline" by Boehm and Turner, for example), the push of the authors to introduce spiral methodology as a "critical step toward success in second-generation data warehousing" is seemingly illogical. Despite all of this, however, what this text provides is as follows: (1) one of the first attempts to standardize data warehousing terminology in what is a very fragmented market segment, (2) explanation of high-level data warehousing concepts, and (3) suggestions on how to avoid some of the problems that have plagued enterprise data and how to manage the high influx of unstructured data that corporations are now creating. Keep in mind, however, that this book is tied into marketing "DW2.0" consulting and certification training, which may provide an explanation as to the vagueness of the material.

First of all, this book is not written with the DW novice in mind. Some of the chapters require a thorough understanding of DW theory and concepts.

Generally I found the book useful and I got some ideas that I will apply in one of my next projects. The biggest weakness of DW 2.0 is its lack in detail. In a lot of areas I found the book to be patchy and too high level. In my opinion DW 2.0 as presented in the book is not (yet) an elaborate data warehousing methodology.

What follows is a discussion of some of the more interesting concepts and chapters in the book.

(1) The different sectors of DW 2.0

To me it did not become fully clear what exactly the Interactive Sector is. Is it a cumulation of an enterprise's operational systems or is it a real time replication of these systems as an additional physical layer? A practical example really would have helped here. Personally I have my doubts if all the operational reporting requirements can be met by the Interactive Sector, e.g. how can a requirement that needs to query data from both the Interactive and Integrated Sector be met?

(2) Fluidity of technology sector

While this offers some interesting thoughts on how to shield the DW 2.0 from changes in business requirements and the operational source systems it only scratches on the surface. The idea as presented by the authors is to physically separate data that structurally does not change frequently (semantically stable date) from data that changes often (temporal data). From the book it does not become clear how this can be achieved. The only advice the authors give here is: "The answer is that semantically static and semantically temporal data should be physically separate in all database designs." (p.121). The authors mention Kalido as a software vendor that provides technology to separate the two different sets of data. From this it seems that they refer to generic data modelling to achieve this separation. However, this does not become clear at all. In my opinion the most frustrating chapter in the book. It raises very interesting questions that it does not answer.

(3) Methodology

Very good summary chapter on why agile and iterative methodologies also advocated by other practicioners in the industry work best for data warehouse projects. If you need to justify an agile approach to your data warehouse project this is a good chapter to refer to.

(4) Performance

Some good ideas on how to improve performance of data warehouses. What I found particularly useful is the concept of farmers and explorers as users of the warehouse that have different analytical needs.

(5) Cost justification

A chapter you can refer to if you need to justify your data warehouse project to management.

(6) Unstructered data

In my opinion this is the best chapter in the book. Before reading the book I had never thought much about unstructured data and how it can be integrated with structured data in the warehouse. The book gives you a good overview on how this might be achieved. However, once again it just scratches at the surface of the problem. It is probably a good idea to refer to Inmon's other book on unstructured data to get more detail.

Overall the book gives a good overview on the concepts of DW 2.0 and what will be required for the next generation of DW 2.0. However, in all chapters it lacks detail and practical examples. The discussion remains somewhat abstract, theoretical, and scientific. It would be nice to see a case study of a data warehouse built on the principles of DW 2.0. Also the quality of graphics and images are of poor quality and let the book down.

One area the authors get wrong is how they define ELT (in opposition to ETL). In contrast to what the authors say ELT does not load the data into the data warehouse and only then applies transformations to it. In ELT tools (such as Oracle Data Integrator or Oracle Warehouse Builder) transformations take place on the data warehouse server(s) using the data warehouse's database engine (using SQL or some dialect). However, transformations happen while the data is loaded or before (staging area on data warehouse servers). This is in contrast to traditional ETL where transformations take place on a separate server ETL server using Java or some other procedural language.

I bought this book because it had good reviews, however, after reading it I was extremely disappointed with Inmon as it does not appear to be a detailed discussion about data warehousing at all, but an over hyped book which tries to explain fundamental data warehousing techniques in a way that you would explain to a baby (repeated a thousand times and with diagrams of various shapes including triangles, circles, and squares). Seriously! Triangles, Circles and Squares!!!

This book is way overpriced for what it is and the fact that Inmon's name is printed on it, does not mean that the book follows Inmon's traditional data warehousing books. This book is different because it does not deliver much content nor does it deliver any new concepts, breakthroughs or strategies of building data warehouses today.

So whats in the book? Honestly, not much. Just very simple concepts scattered around the book in such little detail that it would be impossible to implementation or incorrect to apply in a real world scenario. For example, if you wanted to look at methods of correcting data in todays complex environments, you can turn to page 330 and find the most useless generic statement "A third approach of correcting values in DW 2.0 environment is the practice of finding bad records and then changing the values in those records". Really? So this is what an expert in the field has to say about it? I think not!

You can see DW 2.0 plastered all over the book for some of the most fundamental concepts that having nothing to do with the next generation of data warehousing, nor are many of these concepts discussed recommended for building the next generation of data warehouses.

If you would like to learn more about DW 2.0, I would recommend reading the free literature from Oracle, Hyperion, IBM, Cognos, Business Objects, and other data warehouse leaders. Although some of these may have product specific information, they have the latest and up to date information on techniques that you can use for DW 2.0.