Columnar Deduplication and Column Tokenization: Improving Database Performance, Security, and Interoperability
For some time now, a special technique called columnar deduplication has been implemented by a number of commercially available relational database management systems. In today’s blog post, I discuss the nature and benefits of this technique, which I will refer to as column tokenization for reasons that will become evident.
CBAT Part 2: Flexible Data Modeling for a Simplified End User Experience
In my last blog post, I explained how Column-Based Architecture Technology (CBAT) offers a distinct advantage over the traditional row-oriented RDBMS in terms of I/O workload, deriving primarily from basing the granularity of I/O operations on the column rather than the entire row. This technological advantage has a direct impact on the complexity of data modeling tasks and on the end-user’s experience of the data warehouse, and this is what I will discuss in today’s post.
CBAT Part 1: The I/O Advantage
Column Based Analytical Technology (CBAT) has been getting a lot of attention recently in the data warehouse marketplace and trade press. Interestingly, some of the newer companies offering CBAT-based products give the impression that this is a entirely new development in the RDBMS arena. I don’t know where they have been for the last 10 years! This technology has actually been around for quite a while, and at SAND we have been working with it since 1987. But the market has only recently started to recognize the many benefits of CBAT. So, why is CBAT now coming to be recognized as the technology that offers the best support for very large, complex data warehouses intended to support ad hoc analytics? In my opinion, one of the fundamental reasons is the reduction in I/O workload that it enables.
Read more…
Bus Ride Through the Data Storm: Thoughts on Column-Based Architecture and Corporate Information Memory
As a new contributor to the SAND blog, a few words of introduction are probably in order. I have been working at SAND Technology for 12 years (already!), predominantly in research & development and implementation of Data Warehouses and Intelligent Information Management systems, and I am currently Vice-President of R&D and Deployment for the company. My background is in mainframe OS system programming, mainly Assembler, and APL (my specialization at university) – a peculiar mix which should give you an idea of my age.
Read more…