Sun Data Mashup Engine

Data mashup is the process of gathering content from more than one source and merging that information in order to present a clear and concise view of that data. The Sun Data Mashup Engine is a JBI-compliant service engine that provides Enterprise Data Federation, or mashup services, in multiple forms. It provides a single view of data from dissimilar sources, including static web pages, tabular data exposed as web services, relational databases, flat files, and so on. The Sun Data Mashup Engine joins, aggregates, and cleanses the source data, and then exposes the resulting data set in a unified view, providing real-time views of information for master data consumers.

• Sun Data Mashup Engine Overview

• Sun Data Mashup Engine Development Phase

• Sun Data Mashup Engine Runtime Phase

Sun Data Mashup Engine Overview

The Sun Data Mashup Engine provides server side mashup for dispersed data. More specifically, it provides a single relational view of data that have different origins but are located within the same enterprise. You use the service engine to integrate information from delimited flat files, fixed-width flat files, relational databases, RSS Feeds, web (HTML), XML, WebRowSet, and Microsoft Excel spreadsheets, and then join the various data sources, cleanse the data, and generate a report. After a successful deployment of the composite application, which contains the Data Mashup project, you can open the generated report in a browser, where it functions as a web service and can consume other applications.

You can use the Sun Data Mashup Engine to build a NetBeans project with unified views of your data from different sources, configure the project, create and deploy a composite application, and enable the generated report to function as a virtual database. Data Mashup is a standalone product. It can, however, expose certain JBI-based MDM Suite data sources as services.

There are several classifications of data mashup.

Sun Data Mashup Engine Features

Sun Data Mashup Engine provides your business with a powerful assortment of design-time features that you can use to create and configure Enterprise Data Mashups. The runtime features allow you to monitor the mashup processes and to review any data errors.

The Sun Data Mashup Engine provides the following features:

• Exposes data services for Web 2.0 mashups and real time views for MDM clients.

• Supports JSON as well as WebRowSet.

• Supports both SOAP and REST invocation.

• Leverages various Binding Components (BCs) to access ERP/CRM systems. The XSLT Service Engine can be used for further transformation.

• Creates a unified view of data from disparate sources.

• Integrates information from a variety of heterogeneous sources, including relational databases, flat files, DCOM documents, spreadsheets, XML documents, HTML documents, RSS and Atom feeds, and Xquery row sets. to provide unified view.

• Provides data mashup services capabilities for SOA using the OpenESB and NetBeans infrastructure.

• Exposes the aggregation of multiple data sources to mashup client frameworks, enabling Web 2.0 type applications in an enterprise.

• Provides time-bound view caching for improved response times and can be called as a virtual materialized view.

• Provides multiple views of the resulting data through XSLT transformation by applying a composite weaving pattern.

• Transforms the resulting data set into various formats by weaving the output with an XSLT Service Engine, enabling deployment on multiple channels.

• Ensures extensibility through the ability to consume JBI Services.

• Reuses the NetBeans Database Explorer to browse source tables and to drag and drop source tables into the Data Mashup Editor to define the join conditions.

• Provides the ability to view the result set using the Data Mashup Editor cache management view features.

• Creates a data services layer in true service-oriented architectures.

• Enables integration on-demand. The mashed up views can be used by clients to build highly responsive and interactive enterprise applications using existing client-side frameworks.

Sun Data Mashup Engine Architecture

The Sun Data Mashup Engine design-time components allow you to specify the data source and target databases, map source fields and columns to target fields and columns, define custom processing, and test and validate the Data Mashup processes. Design-time components include the NetBeans project system, a wizard to guide you through creating and configuring Data Mashup process, and a mapping editor where you can map source and target data and customize the transformation. The output of a Data Mashup process can be further weaved with the XSLT Service Engine to produce different output formats that can be deployed to multiple channels. The following diagram shows the Sun Data Integrator components and their relationship to one another.

Figure 11 Data Mashup Architecture

Sun Data Mashup Engine Development Phase

The development phase consists of standard tasks for specifying source and target databases and advanced tasks for further customizing the data transformation logic.

Standard Development Tasks

The following steps outline the basic procedure for developing an ETL process using Sun Data Integrator.

1. Create a virtual database.

2. Add the external tables containing the content to aggregate.

3. Create a new Data Mashup project in NetBeans.

4. Using the wizard, create the Data Mashup file.

5. Using the Data Mashup Editor, specify join and filter conditions.

6. Add the Data Mashup service to a composite application.

7. Build and deploy the composite application.

8. Test the deployment.

Advanced Development Tasks

You can perform additional tasks during the development phase to customize your Data Mashup application further.

• Data Transformation – Add operators to join and filter conditions to create advanced SQL queries.

• Business Process Integration - Call a Data Mashup process from a BPEL business process, web service, Java client, or other application.

• Multiple-Channel Deployment - Weave Data Mashup with XSLT using the CASA Editor to create multiple assemblies with different style sheets to support multiple distribution channels.

Data Mashup Wizard

The Data Mashup Wizard takes you through the steps to set up an Enterprise Data Mashup and, based on the information you specify, creates a collaboration that defines the configuration of the process.

Figure 12 New File Wizard for Data Mashup

Enterprise Data Mashup Editor

Once you define the Data Mashup framework using the wizard, you can further customize the configuration Collaboration Editor.

Figure 13 Enterprise Data Mashup Editor

Sun Data Mashup Engine Runtime Phase

Once all of the development tasks are complete and the system is running, you can perform any of these maintenance tasks.

• View the aggregated data in different formats and various distribution channels.

• Monitor and manage activities and alerts in the application server logs.

• Modify the configuration of the Data Mashup.