Defining and Configuring a Target in Your Stream Analytics Pipeline

Before You Begin

This 10-minute tutorial shows you how to configure a target in a Stream Analytics pipeline that lets you monitor public transportation in the Atlanta area.

This is the fifth tutorial in Monitoring Public Transportation Using Stream Analytics. Read them sequentially.

• Creating Artifacts for Your Stream Analytics Pipeline
• Building a Stream Analytics Pipeline
• Creating Business Rules in Your Stream Analytics Pipeline
• Configuring Graphical Views in Your Stream Analytics Pipeline
• Defining and Configuring a Target in Your Stream Analytics Pipeline
• Publishing Your Stream Analytics Pipeline

Background

Stream Analytics is a graphical tool with an intuitive web-based interface that enables you to explore, analyze, and manipulate streaming data sources in realtime.

This traffic management solution uses GPS fleet data. This data is low cost, accurate, and it's created in real time. Its value for government sector customers is that this fleet data reduces congestion on roadways and enhances the traveling experience. The general features represented in this solution are real-time traffic analytics, speed violation tracking, and congestion detection. These features are combined with GPS streaming sensor feeds and historical trend data using map-based visualizations. This solution uses enterprise-grade Spark Streaming, Kafka open-source messaging, a highly scalable, extensible platform built with Stream Analytics.

Some of the key benefits with this solution are low-cost rollout with zero-road, network disruption, real-time operational intelligence, which is essential for meaningful congestion reduction, and an enhanced traveler experience, with a leading streaming big data technology.

What Do You Need?

Perform the tasks described in Creating Artifacts for Your Stream Analytics Pipeline.

Create a Target

1. Go to the home page.
2. In the Create New Item menu, select Target.
   
3. In the Create Target dialog box, on the Type Properties tab, enter or select the following values and click Next:
• Name: TutorialTarget
• Tags: transportation and tutorial
  
• Target Type: Kafka
4. On the Target Details tab , enter or select the following values and click Next:
   • Connection: Tutorial
   • Topic name: tutorialTarget
5. On the Data Format page, leave all fields as is.
6. On the Shape tab, select Manual Shape option, and enter the following values and then click Save:
   • In the field, enter Violation, and from drop-down list, select Text.
   • In the field, enter AvgSpeed, and from drop-down list select Double.
   • In the field, enter Highway, and from drop-down list select Text.
   • In the field, enter Driver Number, and from drop-down list select Number.

     

Use the Target in Your Pipeline

1. Go to Catalog, and under Show Me click Pipelines, and then click Tutorial.
2. On the pipeline tree, click SpeedViolation.
3. Select Add a Stage and the Target.
4. In the Create Target Stage dialog, enter the name TutorialTarget and press Save.
5. In the Target Mapping drop-down list, select TutorialTarget. The Target Mapping table is populated with the target and output stream properties.

   

6. Review the Live Output Stream table to ensure that the required data and analytics are flowing through your pipeline deployed on the Spark/Kafka runtime.
   

   

   Because you specified a 10-second window during the pipeline development, in some cases, you may need to wait for at least 10 seconds before the live output stream is populated.

Next Tutorial

Publishing Your Stream Analytics Pipeline