bullet2 Distributed Enterprise Architecture

{PRIVATE}

The business application architecture many companies use is as archaic as the decades old centralized mainframe model. Companies would be better served by a decentralized architecture in which their applications and databases are distributed across regional locations and local devices.
by D. Britton Johnston
Business applications today are stuck in a time warp. Web-based applications offer ubiquitous access, but the business application architecture many companies use is still locked into the decades old centralized mainframe model—a model where end-users often access a single instance of the application and almost always access a centralized corporate database.
Applications have moved from the client/server paradigm, to the multi-tier architecture, to Web-based GUIs, to application servers, to open-source platforms, and now are embracing Web services. Yet all of these changes have left the database layer of the application architecture largely untouched.
This is why, for example, a sales rep in Tokyo might have to connect over the Internet to a sales force automation application in New York to get information on a customer located half a block away. If the database administrator (DBA) in New York has taken down the database for some midnight maintenance, the sales rep in Tokyo—who's in the middle of her workday—is back to writing notes on paper for later data entry into the application. It's just like the 1960s.
Even when the database system in this example is up, the sheer distance to a server halfway across the world would cause delays, making the user experience poor. On top of it all, the investment her company made in providing sales staff with powerful laptops and PCs is largely wasted because the centralized application architecture uses these platforms as nothing more than browser-based terminals—just as mainframe architectures used dumb terminals.
All of this raises a number of questions. Why do we continue to lock applications into a centralized database model? When you take an inventory of your critical applications, can they run without connectivity to a centralized database? Are users forced to work around applications that they cannot run disconnected? Is the sales rep in Tokyo frustrated because she cannot access important facts about her account unless she is online? And even when she is online, is the quality of service she experiences at the level she needs to be successful?
The flaws of the centralized model in this example are clear: both the sales person and the New York-based DBA would be much more productive if the application and customer database were deployed in her local office or on her laptop. Each could do their job without dependence on the other, each would be more productive, and the existing resources of the enterprise would be used more effectively.
The Value of Decentralization
Most business applications are interactive in nature and are a bad match for the centralized deployment model. Whether it's a sales person in the Tokyo sales office, a maintenance technician on an airfield in Paris, or a national retail chain store manager in Omaha, all would be better served by a decentralized architecture in which their applications are deployed across regional locations and local devices, rather than in a distant data center.
Microsoft Word is an excellent example of the value of decentralization. Asking workers to connect to a centralized data center to access Microsoft Word would make no sense. Productivity would take an enormous hit, because people wouldn't be able to use their word processor offline, and outside of corporate headquarters, network latency would be so frustrating that typing pools probably would be reinstated.
The fact that field workers and distributed business locations today must be online and connected to access their applications is equally incongruous. Field workers need access to rich data when they're in the field driving business, not just when they're in the office connected to the Internet. And workers in a retail store in Omaha need to be able to capture customer information, view inventory, and serve customers efficiently, even during network outages. That being the case, why are so many business applications centralized and available only online, when more and more workers are remote or mobile?
The answer is that managing distributed applications in remote and disconnected offices has historically been difficult. Today, however, emerging heterogeneous bi-directional replication technologies make distributed applications practical.
I know what you're thinking: "I'm already doing this today with my Palm and my PC and my synchronization software." Well, you're wrong. The advanced synchronization technology available today can cope with very large databases—databases with hundreds of tables and millions of rows—while ensuring that only data required by the end users is transferred. The technology can preserve complex interdependencies between data (often the basis for business transactions) and restart a partial synchronization effort without skipping a beat. All this can be done with any application. This is a dramatic contrast to the typical brute-force comparison of all data available, as is typical with most first-generation mobile platforms.


{PRIVATE}
The Roadblock to Decentralization: The Corporate Database
The corporate database represents the thorniest problem in adopting decentralized, or distributed, application architectures, for the following reasons: Technology—The database is the most challenging technology aspect of a distributed application model. Traditional replication technologies have not provided two-way sharing of both structured and unstructured data among multiple or different databases. No technology has been able to automatically distribute, replicate, and synchronize multiple distributed and heterogeneous databases, while still providing corporate IT with centralized control. Security—Placing valuable data outside corporate offices traditionally has been considered a risk because of the threat of theft or other losses. Partitioning the dataset to provide only required data has been a complex undertaking and difficult to maintain in a changing environment. Furthermore, computing resources associated with the encryption of data within a device historically has been too compute-intensive to be viable protection against theft, and authentication technology has been too expensive to provide local use of biometric or multi-token devices. Economics—Corporate databases are diverse, expensive, and management intensive. Distributed databases that require local DBA support are expensive to deploy, and purchasing separate licenses for each remote location can quickly become cost-prohibitive. Cultural—Most IT personnel have been schooled in the culture of the untouchable centralized corporate database. Vendors of centralized, mainframe-type systems, such as Oracle and IBM, have promoted this view. Adopting a distributed model requires a shift in thinking, from maximizing IT efficiency to maximizing field-worker effectiveness.
A number of market forces at work today are breaking down these barriers to the distributed application model. For one, cheap disk storage and powerful PCs make it possible and cost-effective to distribute, store, and process vast amounts of data anywhere in the enterprise. Databases used to be centralized out of necessity. Today, however, it is quite possible to, for example, put the entire customer database on a salesperson's notebook. Additionally, today's corporate emphasis on disaster recovery and high availability, combined with slashed IT budgets, make an inherently redundant, decentralized application infrastructure far more attractive and cost-effective than today's capital-intensive "fail-over" or "hot-standby" backup schemes.
Network infrastructure has improved to the point where many systems actually operate in an "occasionally disconnected" mode. Remote workers generally have network connectivity or can easily gain access to a corporate network with limited effort. These connections—whether they are wired or wireless—likely have limited bandwidth or are subject to latency unsuitable for presenting a complex user interface, but they are ideal for providing a channel for synchronizing data on a near real-time frequency. This, combined with an approach that allows for disconnected operation, means that the user experience can have the highest quality possible.
To illustrate this, think about a simple trip from home to the airport during which a wireless network device might operate great on the highway, lose connectivity for a short time in an underpass, tunnel, or parking garage, and suffer complete loss once the airplane departs. That type of generally available, but occasionally disconnected, access is going to be the typical connectivity model for remote workers for the foreseeable future. This is a network infrastructure that cannot support true continuous access to critical, centralized applications and data, but is entirely capable of transferring data to provide the highest quality user experience for applications that can support disconnected operation. There is no reason enterprise applications cannot provide the same network transparency that a mail reader can.
While encryption of data being transferred is no longer an issue with widely available strong encrypted stream technology, the additional power of today's computing devices also makes it possible to encrypt data stored on disk without sacrificing basic performance characteristics. The advent of biometric devices such as the PC-Card thumbprint reader means that special devices can be deployed at a very low cost. The net result is data—whether stored in remote servers or on individual laptops—can enjoy a high level of security at a reasonable cost.
And finally, open-source databases and application servers enable companies to overcome the cost barriers inherent in traditional vendor licensing schemes, making it economically feasible to deploy these system components in multiple locations at the edge of the network.
The time is ripe for IT departments to decentralize their business applications and move databases and applications to the edge of the network. So, how do they do it?


{PRIVATE}
How to Decentralize
To date, companies have attempted to decentralize in the following two ways: Partitioned data sets—Perhaps the easiest approach to breaking the dependency on a centralized database is to break that database into multiple pieces. Organizational requirements may lead to this approach and often necessitate re-centralizing the database in the form of a reporting database or warehouse to allow reporting for management. In fact, many organizations evolve into this model without necessarily planning for it, but it results in multiple systems that tend to drift apart over time. Web services allow these "drifting systems" to remain connected, but at the cost of creating applications that are even more dependent on the network and even more likely to be unable to run disconnected. Brute-force synchronization—Today, the next generation of an application often promises some sort of support for mobile access. Nearly all mobile solutions today are plagued by several flaws, including hand-built and fragile synchronization code and a dumbed-down, limited-function flavor of the application. This type of brute-force synchronization is reinforced by the variety of mobile devices, each suggesting a new approach.
Bi-directional Replication for Distributed Databases
Today's new heterogeneous bi-directional replication technologies provide a much better approach to decentralizing. They provide a powerful way to distribute databases across the enterprise. Because these technologies are available as general-purpose software, they enable a new approach to building distributed software solutions. Their heterogeneity enables companies to use any combination of databases. Because they are bi-directional, they enable companies to create redundant architectures populated by replicated systems.
The products featuring heterogeneous bi-directional replication technology can eliminate the manual programming and integration efforts that are so common when building complex synchronization infrastructure. Once a product offering supports hundreds or thousands of deployed sites, software developers can rapidly deliver applications that support distributed deployment (see {HYPERLINK "javascript:showSupportItem('figure1')"} Figure 1 ) .
{PRIVATE}{HYPERLINK "javascript:showSupportItem('figure1')"}{PRIVATE "TYPE=PICT;ALT="}{HYPERLINK "javascript:showSupportItem('figure1')"}
{HYPERLINK "javascript:showSupportItem('figure1')"} Figure 1 : The Distributed Enterprise
Often software vendors find that once they build support for distributed deployment, they are able to address a much larger marketplace and provide competitive differentiation by delivering a new class of applications.
This new class of distributed database applications does not eliminate the need for large centralized databases. It provides a robust environment in which data can be distributed across an enterprise, providing end users with better quality of service and eliminating the dangerous single point of failure inherent in centralized database application architectures.
The Early Adopters
Retail is one of the early areas of adoption for this type of replication technology. Instead of centralizing point-of-sales (POS) systems and polling the systems, retailers are deploying distributed databases so all of their POS systems are local, but dynamically linked. This means that even if the network goes down, the retailer can continue conducting business as usual: capturing customer information, processing transactions, and even viewing sister stores' inventory.
Field force automation is another area of early adoption, especially in the area of field maintenance for complex systems because it is so data intensive. Service systems can be built for devices like jet engines, providing service technicians with direct access to complete history, specifications, and even expert systems that help diagnose and perform complex repairs—regardless of whether the system technician has a full-time network connection.
Application and business requirements will often limit the amount of time a worker can operate without network connectivity. For a sales force selling very large and complex products, configuring an order directly with a customer may be an important requirement, but confirmation of acceptance of that order may need to wait until it has been transmitted to the central system. An application may need to account for this change in order-processing behavior, and once it does can provide a significantly improved sales experience over a multi-step review process necessary when a salesperson must return to their office to do final configuration of an order.
In the healthcare industry, application developers are finding they can employ devices on existing desktops to provide superior authentication solutions at very low cost, supporting legislated security requirements even at individual doctors' offices.
Let's Not Do the Time Warp Again!
As organizations gain experience with distributed deployment, they discover secondary benefits that are often just as valuable to an organization as the improved quality of service for end users. For example, a distributed system that has not only applications but also data maintained locally provides a level of resilience not possible with traditional application architectures. Properly deployed applications can survive virtually any failure—including even the loss of the centralized database—for an extended time without any loss of service to most end users.
It's time for business applications to get out of the centralization time warp. Developers should take a long, hard look at their applications and evaluate the benefits that might be delivered by distributing both execution and data. Would end-users benefit if your applications could continue to operate disconnected from a central database? Do you need to provide mobile workers full-function applications but see no way to do it in a world dominated by centralized databases? Can your business process be modified to provide a better experience for a customer and support distributed processing? Do new low-cost authentication and encryption technologies provide adequate security for your data? Examine your application architecture, and you may find that the assumptions you have made about data are no longer valid.
In a world where advanced bi-directional heterogeneous replication technologies let you move data to the user, and keep that data synchronized without writing complex application code and without having to build a reduced function application, the centralized model seems as archaic as its forebear: the mainframe architecture. This concept of distributed data may seem new today, but technology evolution, combined with market opportunity and competitive pressure, will make it commonplace tomorrow. And at that point, end-users will expect all their critical applications to run disconnected from the network.
D. Britton Johnston, Chief Strategy Officer at {HYPERLINK "http://www.peerdirect.com"} PeerDirect , holds a wealth of enterprise database and open source expertise. He has more than 18 years experience in delivering software products to enterprise customers. You can reach him at {HYPERLINK "mailto:britt@peerdirect.com"} britt@peerdirect.com .