Purpose
Provide guidance to Information Managers and Architects on design concepts, development approaches, deployment/operational support, and future direction of the two fundamental types of information systems, Analytical and Operational Systems.
Background and History
In order to provide a good solution, one must understand the purpose of the solution. At the core of an enterprise software solution’s purpose resides one central design point. The design point determines the overall system type of an enterprise software solution and can be established by answering the following fundamental question: Is this system an Analytical or Operational system?
The roots of this question stem from the early 1960’s when the cost prohibitive barriers traditionally associated with computer systems were removed, thanks mostly to the IBM 360 series of computers. This gave institutions the opportunity to deliberately dedicate computing power to solve different business problems. Two main types of business problems were immediately identified as practical and highly valuable computer solution candidates:
● Transactional business problems:
○ The challenge of capturing and processing many business transactions quickly and reliably.
● Decisional business problems:
○ The challenge of crunching “large” volumes of data in a “short” amount of time to provide information to support business decisions.
Almost immediately computer scientists and engineers recognized that different computing architectures were required to support these distinct business problems. Batch processing, the method traditionally leveraged to solve Decisional business problems, has been around almost since the invention of the computer. However, in the late 60’s IBM recognized the need for a new method of capturing data for transactional business systems and introduced the world to the Customer Information Control System (CICS). CICS quickly became the de-facto standard for business transaction based computer solutions. After CICS, almost all computer systems were designed based on one explicit decision consideration; is the intent of this system to provide management decision support information or is the intent of this system to execute large volumes of business process transactions? (Sound familiar?)
It wasn’t until the early 1980’s when the term Data Warehouse was introduced to the business world that the language of this fundamental computer system design point evolved into its current form, Analytical vs. Operational systems.
Definitions
Now that you know the history, it’s time to clearly define the terms.
Analytical System
An Analytical System refers to any system that derives information for decision purposes from discrete data points to end users/processes.
Operational System
An Operational System refers to any system that captures and processes discrete data points in the context of a business process.
Identification
It’s all in the name
Regardless if you are embarking on a green field project or if you are working on a Production system, one of the easiest ways to determine the type of system is to listen to the terms used to describe the system.
The terms Analytical system and Operational system have progressed through many different “buzz word” phases, thanks to software companies. For example Analytical systems have been known as Executive Information Systems, Executive Support Systems, Online Analytical Processing (OLAP) Systems, Data Warehouses, Data Marts, Decision Support Systems, Management Information Systems, Analytics Systems, Business Intelligence Systems, etc.
Operational systems haven’t experienced the same level of name calling as Analytical Systems but here are a few examples; Transactional Systems, Event Systems, Business Process Systems, Online Transactional Processing (OLTP) Systems, etc.
| Survival Tip: Know the terms you use. Be sure to understand a term before you use it. Intrapersonal confusion can lead to all sorts of system issues! |
Architecture Reference
Just as the different terms associated with these two fundamental system classifications can help you identify the system’s type, there are some tell-tell architectural qualities that will allow you to easily distinguish between Analytical and Operational systems. As you may have guessed by now the easiest way to determine the type of system from an architectural standing point is to understand the context of the information flowing in the system. That is, understanding how and what data movement is occurring in the system will normally provide enough information for you to label the system as being either Analytical or Operational. It is also important to understand the business goal and purpose of the system.
The following table provides more details on the traditional architectural differences between Analytical and Operational systems. The key words listed in this table should be used in helping to identify the architecture of a system.
Architecture Attributes | Analytical Systems | Operational Systems |
Targeted End User Case “The Goal” of the System | Provide summarized information based on granular data to support decision making | Process and store granular data in real time to support business processes |
Data Intake Method | Batch, Extract Transform Load (ETL) or Extract Load Transform (ELT) Could be other near real time (Services, Messaging, etc) | Real time (Web forms, Service Oriented Architecture (SOA), Messaging, etc) Could be Batch |
Integration Method | Offline Integration Typically Asynchronous | Online Integration Typically Synchronous Asynchronous if Desired |
Data Model | Dimensional OLAP Third Normal Data Vault | Third Normal or Higher Data Vault Functionality Oriented (Data Model Aligns with Functionality, Leads to Data Spaghetti) |
Retention Periods | Usually Years | Usually Months or Less |
Data Volume | High | Low |
Transaction Throughput | Slow | Fast |
Subject Area Domain | Multiple Domains | Singular Domains |
Machine Resource Constraints | Disk I/O | CPU and RAM |
| Survival Tip: The distinction between synchronous and asynchronous is a biggie. Synchronous integration occurs when a system issues a call and waits for a response before proceeding with other tasks. Asynchronous exchanges occur when a system issues a call and then moves onto another task before receiving a response. Be deliberate when you design Synchronous integration points into your system, especially an Operational system. You could be creating a major failure point in the system! |
Data Flow Methodologies
The following image shows a typical conceptual architecture for each system type and highlights the different data flow methodologies leveraged by these systems.
The architecture of Analytical systems typically contains controlled information flow from source to end-user or downstream systems/processes. The process flow of data in Analytical systems typically looks like an information manufacturing line, where data is put into the system at one end and information is delivered out of the other end. Most processing in Analytical systems occurs in large coordinated batch methods.
The architecture of Operational systems typically have a more fluid and chaotic information flow compared to Analytical systems. Data flows into these systems on an ad hoc basis and through different integration channels. The demand for data processing can fluctuate drastically between peak and trough times in Operational systems. The ad hoc nature and fluctuations in data demands of Operational systems require that these systems be designed with a lot more flexibility and thus complexity compared to Analytical systems.
| Survival Tip: Balancing system data ensures that the information derived by the system matches the data put into the system. In other words, balancing a system ensures that outputs match inputs. Balancing information in an Analytical system is typically a straight forward exercise because the data in this system type is in a fixed state. Balancing an Operational system is a lot more complex especially if the system if in a state of constant flux. Avoid situations where you have to balance an inflight, ever changing Operational system against a static Analytical system. If you ever get a request like this, run! This is almost an impossible request. It is certainly impractical. If you have to do this, don’t try to balance the operational system while it is “in flight”. Instead, balance a point in time snapshot of the operational system with the analytical system. Also, set expectations that the two system types will never match exactly. |
Design Concepts
Now that you understand the classifications of the systems, it’s time to work through the different design concepts that should be built into each type of system. As we discussed in the Introduction to this blog, design concepts are the key topics that should be addressed within the system. They are the defining characteristics of the system unto which all functionality is built. Simply stated, design concepts are a system’s DNA. Almost everything in a system can be repaired except for a fundamentally bad design.
The following section walks through the design concept topics reviewed in the first posting of this blog and provides high level design considerations, through questions, to help guide you in the right direction with the design of your system. Because we are still talking about the two main categories of systems, instead of specific system implementations, the design considerations listed below are very abstract.
Use these design concept questions when reviewing prebuilt systems or when designing new systems.
Design Concept: End User Access Points
Application: Analytical and Operational Systems have traditionally satisfied two very different sets of end users. The desperate sets of end users require different and deliberate design considerations when designing how end users will access and integrate with the systems.
Common Design Considerations:
● How will end users access the system?
● Will end users require a live connection to a centralized data source? Will information be stored locally on the user’s device? Or are both scenarios accurate?
● On what devices will end users be connecting to the system?
● How will end users collaborate within the system?
● How will end users collaborate with other systems?
System Type Specific Design Considerations:
Analytical System | Operational System |
| ● How frequently should information be made available to end users? ● Will information be analyzed/sourced from a local, remote, or hybrid source? ● Will end users receive or request access to analytical information, i.e. push vs. pull. | ● How long will it take to complete a transaction/event/session? ● Will end users be able to save the state of in process transactions/events/sessions? ● Does an end user have to wait on a system response before moving to another part of the system? |
Design Concept: Integration Points
Application: As we saw when reviewing the high level information flow for each system type, the Integrations Points for Analytical and Operational Systems are very different. Though “content” is a shared design consideration for the system types, the timing, transformation, and logic required to handle information exchange for both system types requires unique design considerations.
Common Design Considerations:
● What are the sources of data for the system?
● Will data from the system be sent to other systems?
● For internal and external data exchange:
○ How and with what mechanisms will information be exchanged?
○ At what grain is data being exchanged?
○ At what frequency will information be exchanged (schedule)?
● Which integration points are synchronous and which are asynchronous?
System Type Specific Design Considerations:
Analytical System | Operational System |
● What are the “normal” values for each attribute? ● What will the system do with anomalies in source data? ● How will the system gracefully recover if an error occurs while consuming a batch file? | ● How will the system handle collisions, incorrectly sequenced transactions/events, or other unforeseen logic scenarios that can occur with Operational systems? ● What protocols will be used for service or messaging exchange? ● How will data be transformed between integrated systems? |
| Survival Tip: Know the grain of everything in the system, from a transaction to a summary table. Granularity is one of the fundamental keys to designing a system as it impacts the data model, canonical model, all reports/visualizations, all transactions, any object model, etc. To survive as an Information Manager, you must understand Grain! Be sure to learn what “incorrect” grain looks like. Here’s a hint, unpredictable behavior! |
Design Concept: Information Persistence
Application: Data in Analytical systems tends to persist, i.e. written to physical storage like disk, more often and for longer durations compared to Operational systems. The Information Persistence design considerations for Analytical systems involve guarding against over persistence of data, which leads to capacity or performance challenges. While the Information Persistence design considerations for Operational systems involve guarding against under persistence of data, which leads to data integrity and recovery challenges.
Common Design Considerations:
● Where will data persist in the system?
● What are the different ways data is persisted in a system, i.e. file, database, etc?
● How will cache be used within the system?
● How will persisted data be refreshed?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Will source data be archived after processing? If so, where and for how long? ● How often is data persisted through the transformation portion of Information Flow? ● Is duplicate data persisted in the system? If so where and why? | ● Will data be stored locally on an end user’s device? If so, how long will data “live” on the device? ● Where will transactional data be archived for audit purposes? ● Will data persistence be required to safeguard against the loss of transactional data? |
| Survival Tip: Learn the difference between volatile and non-volatile data? In simple terms, volatile data is any data that can be changed. Why non-volatile data is any data that can’t be changed. You will want to design different Information Persistent capabilities for volatile and non-volatile data. |
Design Concept: Business Logic
Application: Business Logic in Analytical systems combines the source system rules required to combine data from different source systems and is typically centralized in some form of a transformation engine, i.e. in the layer of the system where data is turned into information. Getting consensus on the correct Business Logic within an Analytical system can be very challenging as each source system will require a slightly different version of similar Business Logic. Business Logic in Operational systems is typically distributed to different points within the system where it is required to complete a task. A well designed Operational system will leverage code-reuse principles to mitigate Business Logic duplication which can introduce many system integrity issues.
Common Design Considerations:
● Where will business logic be embedded into the system?
● How will business logic be validated?
● How will business logic be configured into the system, i.e. hard-coded, etc?
● How transparent will business logic be made for end users?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Will the system have to adhere to any Information Governance policies? ● Does your system produce or consume Master Data? | ● How flexible is your system with regards to business logic? ● Can the system handle updates to business logic made in real time? |
| Survival Tip: Designing Business Logic considerations into a system may seem a bit redundant as the implementation of Business logic is typically the end result of building the system. Just remember that a business is constantly evolving and therefore the logic used within a business will always be changing. Knowing how to design a system so that Business Logic changes can be configured as opposed to coded ensures a healthy survival rate as an Information Architect. |
Design Concept: Security
Application: Analytical and Operational systems share many common Security design considerations. However, the “batch” nature of Analytical systems can introduce scenarios where changes to an end user’s security settings aren’t immediately reflected in the system. This is because an offline batch event may be required to “refresh” the security setting in the system. The complex flow of information and structure of Business Logic within Operational systems can introduce complex Security scenarios to resolve conflicting Security updates or requests. Also, Operational systems are typically exposed to more users, uncontrolled interactions, and integration points and therefore are more susceptible to security threats compared to Analytical systems. This doesn’t mean that if you are building an Analytical system you don’t have to worry about security. It means that you have to even more diligent with Security design considerations in Operational systems.
Common Design Considerations:
● What are the security policies for the system?
● How will information be secured for specific users?
● Should administrative users, i.e. system admins or database admins, be able to view data within a system?
● What authentication methodologies will be leveraged in the system?
● How will security integrate with other systems, i.e. single sign-on?
● Where are the system’s security vulnerabilities?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Will you model security at the database level or at the tool level or both? ● How frequently can an end-user’s security access information be updated in the system? What is the acceptable gap in duration between a security change and its application in the system? | ● Will forms or OS specific security be used in the system? ● Where will you allow non-standard, i.e. administrative or other, access to internet exposed systems? ● How does security integrate across the components of a system? This is especially pertinent in SOA systems with composite applications and exposed APIs. ● What are the possible security scenarios within the system, updates, changes, modifications, etc? How will the system behave if conflicting security events occur? |
|
Design Concept: Disaster and Recovery
Application: Disaster and Recovery will be heavily influenced by the Information Persistence design concept. Information and data can typically be recovered if it is persisted. Operational systems are typically required to recover from a disaster a lot quicker than Analytical systems. However, because data isn’t typically persisted as thoroughly in Operational systems compared to Analytical systems, Operational systems are typically more vulnerable with regards to a disaster compared to Analytical systems. Analytical systems have the luxury of being able to recover source data directly from an Operational system if necessary. This is a tremendous “safety-net” that is unintentionally built into most Analytical systems. Analytical systems should be built to never have to use this safety net! Don’t forget it’s there though.
Common Design Considerations:
● How often are system, database, and application back-ups created?
● For how long are backups stored?
● How much tolerance, i.e. SLA, is allowed for system downtime?
● How will the system recover from a disaster?
● What will end users experience if the system crashes?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Will the system be repopulated from source system data, from a back-up, or both in the event of a disaster? ● How will the system apply missed loads because of downtime? ● For how long does each source system retain source data? Do the source systems have accessible archives of source data? | ● Will end users lose any data if a disaster occurs before they complete a transaction? ● What happens when disasters occur at small integration points? Will data be lost? ● What happens when a disaster occurs at any layer of the system’s architecture, i.e. web layer, application layer, database layer, cache layer, etc? ● How will the system ensure 99.999% availability, the standard for operational systems? |
Survival Tip: There are two types of motorcycle riders in the world, the type that has “laid a bike down,” and the type that will “lay a bike down.” The same can be said for Information Managers. A system will blow up on you. That may or may not be your fault. The bigger item that you will be blamed for is if the system is not properly prepared to be brought back online, quickly and easily! Your survival is directly correlated to how well your system can handle the recovery portion of Disaster and Recovery.
Survival Tip: There are two types of motorcycle riders in the world, the type that has “laid a bike down,” and the type that will “lay a bike down.” The same can be said for Information Managers. A system will blow up on you. That may or may not be your fault. The bigger item that you will be blamed for is if the system is not properly prepared to be brought back online, quickly and easily! Your survival is directly correlated to how well your system can handle the recovery portion of Disaster and Recovery.
Design Concept: Transparency
Application: Transparency in both system types requires building alerts, diagnostic capabilities, and troubleshooting visibility into the systems. In addition to these Transparency fundamentals, Analytical systems also require deep data lineage visibility to track a single element of information from source through transformation and into its target form. While Operational systems require additional transparency into workflow or transaction flows to track the progress or state of a request or transaction.
Common Design Considerations:
● How will the system notify the correct people when an internal issue occurs?
● What troubleshooting tools will be used when debugging the system?
● Will the system provide a consolidated view into all of the logs, servers, and components of the system for troubleshooting purposes?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Will audit and lineage information be available for all data attributes in the system? ● What audit information will be available for each step in an ETL process? | ● Will the system provide a real-time operational dashboard so specific users can see transactional information, i.e. amount, type, messaging status, etc? |
| Survival Tip: Most Analytical systems are sold to a business on the premise of “You can’t manage what you can’t see.” This holds true for the lifecycle of a system itself. Survival in the world of Information Management is much more achievable if you can easily see how well your system is performing. |
Design Concept: Capacity and Scalability
Application: Systems will need to scale overtime, either up or sometimes down. Analytical systems tend to have a much more predictable workload placed on them from either end user requests or data transformation jobs compared to Operational systems. Therefore, the scalability of an Analytical system is typically more deliberate and scheduled compared to an Operational system. The fluctuations in work load and extreme differences placed onto Operational systems by peak and trough usage requires that this system type be a lot more flexible when scaling. It is not uncommon that an Operational system will need to dynamically add capacity to scale to double or triple its original size. The pressure to react very rapidly to capacity issues in Operational systems makes this design concept absolutely critical.
Common Design Considerations:
● How will the system grow overtime?
● What is the expected rate of growth for the system?
● How will each component in the system grow overtime?
● What system components will scale horizontally vs. vertically?
● What are the system breakout components, i.e. components that will be moved off onto their own set of servers?
● What are the threshold triggers for scaling, i.e. how much data, end users, CPU, etc before something is done to the system to increase capacity?
● What techniques will be used to scale the system very quickly, i.e. virtualization, cloud, etc?
System Type Specific Design Considerations:
Analytical System | Operational System |
● How much data will be added to the system on a daily, monthly, yearly basis? ● How will old data be archived out of the system? ● How many days’ worth of data will the system be able to persist? ● What type of load will analysis place on the system? ● What safeguards have been built into the system to mitigate the “run away” query scenario? | ● How many end users will be added to the system on a daily, monthly, yearly basis? ● What is the average transaction usage pattern? ● How many transactions is the system expecting to receive in an hour, minute, or second? |
Design Concept: Performance
Application: On the surface, performance appears to be a Design Concept that has no difference in traits when applied to Analytical and Operational systems. All systems need to perform quickly and within the stated Service Level Agreement (SLA). However, Performance in an Analytical system has two typical flavors:
1. Transformation process (ETL) performance
a. When will the system be refreshed with new data?
2. End User response performance
a. How long does it take to access information from an end user tool (report, dashboard, etc)
While performance in an Operational system is typically focused on throughput, how quickly a transaction can be completed. The ability to scale and add capacity to a system greatly impacts the performance of the system. Always remember though that no amount of hardware can improve the performance of a poorly designed system.
Common Design Considerations:
● What are the possible performance bottlenecks in the system and what can be done to mitigate the risks?
● What are the Service Level Agreement (SLA) requirements for system performance?
● How will performance be guaranteed if the system or part of the system resides on a different network, i.e. cloud, federated, etc?
System Type Specific Design Considerations:
Analytical System | Operational System |
● What are the load windows for batch processing? ● How will cache, indexing, etc be used in the batch loading processes to ensure great performance? ● What database tuning techniques will be used, partitioning, query tuning, indexing, etc to ensure reports and analytical tools performance well? | ● What hardware enabled techniques will be used to ensure adequate transaction performance? ● How will performance be guaranteed is part of the system runs on an end user’s device? |
Design Concept: Maintenance, Bugs, and Upgrades
Application: Systems have to be cared for and fed overtime. This should be made into a law! There is no escaping the fact that systems will evolve. Design Analytical systems to take advantage of nightly “down periods” or “offline” windows to perform maintenance. This will impact any transformational processes, but your system should be able to gracefully recover if a transformation process is started at a different time period anyway. Maintaining an Operational system is much more challenging as these systems are typically always available to end users. Be sure to resolve this Design Concept prior to launching an Operational system!
Common Design Considerations:
● What is the SLA for uptime of the system?
● What are the maintenance window thresholds for the system?
● What are the end user device requirements for the system and what happens to the system when the end user upgrades their device?
● What techniques will be used to enable end user functionality components be maintained and upgraded, i.e. asynchronous updates to end user components, push techniques, pull techniques?
System Type Specific Design Considerations:
Analytical System | Operational System |
● Can processing systems be upgraded prior to a batch load as part of the nightly patch load process? ● Can analysis tools be upgraded during the nightly batch load process window? | ● How will the system enable live, real time, upgrades to different components, i.e. business logic, database components, work flows, etc. ● How many hours a year will you have for system downtime? Hint: The standard 5 nines, 99.999%, availability means you will have about 5.265 minutes available each year for system downtime! |
| Survival Tip: Do you want to embed data latency into your system? A lot of the design concepts in this section can be boiled down to one point. How much data latency do you want do design into your system? Analytical systems possess a high level of data latency because data is exchanged in large offline batch processes. Operational systems have a low level of data latency because data is ALWAYS exchanged! Be cognizant of this question when designing your systems. |
The design concepts listed above are very broad and general categories of topics that require specific solution sets in a real-life Analytical or Operational system. We will address specific design concepts for some technology-agnostic solution sets, like Data Warehousing and Complex Event Processing, in subsequent postings of this blog. As Analytical and Operational systems have drastically different use cases, end users, etc, describing the specific solution set of each of the design concepts for each type of Analytical or Operational system would be impractical. Technology is also evolving at such a rapid pace, that if we created an exhaustive list of specific solution sets in the terms of today’s technical capabilities, this book would be outdated before it was published.
You should now have a solid understanding of the differences between Analytical and Operational systems and what these differences mean to an Information Architect. Over the past several years the divide between these two systems has started to narrow and in some cases disappear. This is due to advances in technology and architectural design. These advances have had a profound impact on systems and have placed a burden on Information Architects to broaden and deepen their skill sets; luckily you have a survival guide to help!
The last section of this post will touch on what’s occurring in the evolution of technology and how this evolution affects the two main types of systems. The purpose of this section of the chapter is to ensure you are aware and prepared to encounter the “hybrid system”!
Hybrid Systems
MapReduce, Data Grids, Cloud, MPP, In Memory Appliances, Solid State Disks (SSD), Columnar and Tabular Databases, etc. all of these terms describe technical advances that are allowing software engineering’s to create products that blur the lines between Analytical and Operational systems, i.e create hybrid systems. This is as it should be when you think about it. In the real world, as opposed to the software development world, people want to analyze, decide and act as a concurrent thought process in one discrete event. People want to see analytical information and act on it through one application/interface. It is the software world, due to hardware/technical constraints, that drove the specialization in architecture approaches and created the paradigms of Analytical and Operational systems. Yes, there are real world special use cases where you want a pure Analytical system, but these use cases are becoming less frequent.
The world is advancing at a rapid pace. Information driven systems are evolving very quickly as well. It is very easy to foresee a time in the near future where an Enterprise Data Warehouse is accessed and maintained in real time as opposed to the current nightly or sometimes weekly maintenance, ETL, process methodologies. Where statistical models are developed and processed in real time based on a sales transaction as opposed to being developed over several months requiring hours and hours of processing time to perfect. Where Operational systems act on Petabytes worth of atomic transactional data to make recommendations in real time as opposed to only storing Gigabytes worth of aggregated data.
Though the technology is evolving faster than I can write this blog, the design principals discussed earlier in this chapter still hold true. An Information Manager should look at hybrid systems based on all the design concept topics that have been discussed.
Below is a break out of some of the frameworks and technologies that are enabling hybrid systems. This table should help you understand if and how these technologies can help your organization achieve its goals.
Framework / Technology | Definition | Key Characteristics |
Massively Parallel Processing (MPP) | Commonly referred to as “shared-nothing” architecture, MPP is a technical architecture that supports distributed computing over independent processes or physical servers, nodes, to provide high processing performance. The unique part of MPP is that each node acts independently and controls its own resources, i.e. CPU, RAM, disk, etc, for processing tasks in parallel. This is as opposed to the more common system architecture approach where resources are shared across processing units. Many Relational Database Management Systems (RDBMS) vendors are providing products that support MPP architectures for huge performance gains. | ● Hardware architecture ● Distributed processing across nodes ● Self-Reliant nodes do not share resources, dedicated CPU, RAM, disk ● Large amount of parallel processing = fast processing ● Proven architecture ● Common DBMS support |
Map Reduce | A specific framework introduced by Google that leverages a cluster or grid of commodity servers to process large data sets. Data is first mapped by a “master” node, using key value pairs, and spread across a set of “worker” nodes for granular processing. Then data is reduced by the master node to return a result. Map Reduce is a framework/engine that can leverage different technical hardware architectures. | ● Key-value paired data sets ● Master notes and Worker nodes ● Fast processing of very large data sets ● Commodity Hardware = Low cost ● Leverages distributed file storage and processing for massively parallel processing power ● Not a database, no tables, relational integrity, etc ● Requires specialized language for data access and processing |
In Memory Appliances | In Memory Appliances are self-contained, fully engineered systems consisting of a hardware layer composed of a large amount of cache with a tightly coupled application layer sitting on top of the hardware. These systems are generally very fast because there is little disk i/o during processing, the cache is typically synched with other systems asynchronously in a “background” process. A lot of statistical solutions are leveraging an in memory approach to preform real time operations. | ● Terabytes of Cache ● Typically embedded in a database or other toolset ● Fast processing of very large data sets ● Low disk i/o |
Columnar Database | Columnar Databases are specialized databases that pivot the traditional table design found in a relational database management system (RDBMS). Columnar Databases provide 2 advantages over RDBMS. The first is compression, somewhere in the range of 5:1 – 10:1. The second is speed for analytical queries. Because tables in this type of database are shallow but very wide, as opposed to RDBMS which are narrow but very deep, analytical operations such as ranking, segmentation, etc are very fast. | ● Rows are the possible attributes ● Columns contain values ● High rates of compression ● Great for analytical operations |
Cloud | Cloud is a generic term that is used in many different ways to describe a desperate set of technologies. For this book, we will consider cloud any set of systems (infrastructure + software) that can scale very quickly to meet the demand of a system and shares resources across systems. Clouds are typically hosted solutions, but they can also live within an organizations data center. They offer great performance cost advantages due to their ability to host multiple solutions and scale rapidly. | ● Rapid deployment ● Dynamic scaling ● Multi-tenant hosts ● Cost reducing |
This table just scratches the surface of the technologies that are being introduced and proven in the world. As with any technology project, be sure to have a clear understanding of the end user requirements and match them to the right solution, especially when working with new hybrid technology. Just because a technology is faster and technically superior to another technology, doesn’t mean it is the right solution for the end user’s needs.
Summary
In this post we took a journey through the software world that started in the 1950s and has no end in sight. The purpose of this journey was to show Information Managers the evolution of Analytical and Operational systems to ensure you:
● Understand the characteristics of these two software paradigms
● Can classify a system correctly
● Know the important design concepts that should be incorporated into each system type
● Are aware of the trend of a technology enabled merging of these two system types
The remainder of this guide dives deeper into each system type to provide details on specific systems. Remember what was covered in this chapter, you will see it a lot in future segments.
