Current status of the data warehousing market

Now the traditional database companies are also in the data warehousing market. They have begun to offer data warehousing solutions built around their database products. On one hand, data extraction and transformation tools are packaged with the database management system. 

Figure 3-2 Current status of the data warehousing market.

On the other hand, inquiry and reporting tools are enhanced for data warehousing. Some database vendors take the enhancement further by offering sophisticated products such as data milling tools.

With so many vendors and products, how can we classify the vendors and products, and thereby make .sense of the market'? It is best to separate the market broadly into two distinct groups. The first group consists of data warehouse vendors and products catering to the needs of corporate data warehouses in which all enterprise data is integrated and transformed. This segment has been referred to as the market for strategic data warehouses free download microsoft exam. This segment accounts for about a quarter of the total market. The second segment is MON loose and dispersed, consisting of departmental data marts, fragmented database marketing systems, and a wide range of decision support systems. Specific vendors and products dominate each segment.

We may also look at the list or products in another Way. Figure 3-3 shows a list of prod-ucts, grouped by the functions they per in a data warehouse.

Vendor Solutions and Products

Vendor Solutions and Products

As an information technology professional, you 'are familiar with database vendors and database products. In the same way, you are familiar with most of the operating systems and their vendors. How many leading database vendors are there? I hoc, many leading vendors of operating systems are there? A handful? The number of database and operating system vendors pales in comparison with data Warehousing products and vendors. There are hundreds of data warehousing vendors and thousands of data warehousing products and solutions.

In the beginning, the market was Filled with confusion and vendor hype. Every vendor, small or big, that had any product remotely connected to data warehousing jumped on the bandwagon. Data warehousing meant what each vendor defined it to be. Each company positioned its own products its the proper set of data warehousing tools. Data warehousing was a new concept for many of the businesses that adopted it. These businesses were at the mercy of the marketing hype of the vendors.

Over the past decade, the situation has improved tremendously. The market is reaching maturity to the extent of producing °off-the-shelf packages and becoming increasingly stable. Figure 3-2 shows the current state of the data warehousing market.

What do we normally sec in any maturing, market? We expect to find a process of Consolidation. And that is exactly what is taking place in the data warehousing market. Data warehousing vendors are merging to form stronger and more viable companies. Some major players in the industry are extending the range of their solutions by acquisition of other companies. Some vendors are positioning suites of products, their own or ones from groups of other vendors, piecing them together as integrated data warehousing solutions. 

Data Warehouse Expansion

Data Warehouse Expansion

Although earlier data warehouses concentrated on keeping summary data for high-level analysis. we now see larger and larger data warehouses being built by different businesses Now companies have the ability lo capture, cleanse, maintain, and use the vast amounts of data generated by their business transactions. The quantities or data kept in the data ware- houses continue to swell to the terabyte range. Data warehouses storing several terabytes of data are not uncommon in retail and telecommunications. 

Figure 3-1 Industries using data warehousing. 

For example, take the telecommunications industry. A telecommunications company generates hundreds of millions of call-detail transactions in a year. For promoting the proper products and services, the company needs to analyze these detailed transactions. The data warehouse for the company has to store data at the lowest level of detail.

Similarly, consider a retail chain with hundreds of stores. Every day, each store gener-ates many thousands of point-of-sale transactions. Again, another example is a company in the pharmaceutical industry that processes thousands of tests and measurements for getting product approvals from the government. Data warehouses in these industries tend to be very large.

Finally, let us look at the potential size of a typical Medicaid Fraud Control Unit of a large state. This organization is exclusively responsible for investigating and prosecuting health care fraud arising out of billions of dollars spent on Medicaid in that state. The unit also has to prosecute cases of patient abuse in nursing homes and monitor fraudulent 'billing practices by physicians, pharmacists, and other health care providers and vendors. Usually there are several regional offices. A fraud scheme detected in one region must be against all other regions. Can you imagine the size of data warehouse needed to supports such a fraud control unit? There could be many terabytes of data.