Case Study - The Hairball Model
Decoupling the "Hair Ball Model"
A railroad company needed to "untangle" the "hair ball" model within their IT infrastructure.
Challenge:The second largest Class 1 Railroad company in the America has grown its data model from a few tables in the mainframe to a highly complex, tightly coupled structure of hundreds of tables and thousands of columns. This data is in turn used by hundreds of widely diverse applications, which are in turn accessed by other, newer applications. Ultimately this has created a complex web of existing applications that have evolved over more than 20 years. These have become so intertwined that any accurate modelling resembles nothing more than a “hair ball”, as illustrated below.
The hairball inevitably resulted in a system where access to data has become prohibitively slow when accessed by modern applications.
While a multiyear program design to modernize the IT department within its every little nook and cranny is underway, the business needs to continue satisfying the needs of its customers. The question is: how can we design new systems without falling into the hairball problem? And more importantly, how can this data be accessed in a timely manner without sacrificing the integrity or richness of the existing information?
Solution:The Crossvale Solution was create an envelope around the enterprise data that decouples client systems from the complexity of the enterprise layer. This layer provides a uniform, standards-compliant and customer focused interface into the complex and diverse underlying systems. Our Customer Service Layer (CSL) not only separates the concerns of the User Interface with the back end systems but also provides a standardized resource-oriented REST interface that exposes services that other applications can consume. These resources are designed in context of the “customer lingo” with human readable properties that can be readily recognized and understood by the client. Interactions are the via simple, standard set of HTTP/REST “verbs” and data is accessed via the ubiquitous HTTP protocol. Our architecture also utilizes a “near realtime” strategy to ensure sub-second access to the underlying data. A background process known as the Enterprise Transition Layer (ETL) consolidated and prepares enterprise data and then feeds it into a CSL cache. This process runs with at most a 15 minute delay, which is sufficiently “near time” for most of the client needs. CSL also has access to the Enterprise Service Layer (ESL) to obtain real-time data such as user profiles, itineraries, and events. This information is integrated at request time by CSL and provided to the user in a uniform and consistent manner.
We used rapid development tools such as Groovy, TIBCO and Informatica as well as mature frameworks such as Spring Boot and Spring MVC to rapidly deliver value-added functionality. Our CSL layer leverages the big data and rapid access capabilities of MongoDB to provide instant access to the ETL-provided near-realtime data. This also allows a “design once” strategy where our data structures can be defined completely as Groovy objects. We then rely on Spring MVC and MongoDB to implement our resource API and database structures.