1. Stream Analytics Documentation
  2. Manage
  3. Targets
  4. Create Targets
  5. Creating an AWS S3 Target

4.4.1.1 Creating an AWS S3 Target

To create an AWS S3 target:

  1. On the Catalog page, click Create New Item.
  2. Hover the mouse over Target and select AWS S3 from the submenu.
  3. On the Type Properties screen, enter the following details:
    • Name: Enter a unique name for the target. This is a mandatory field.
    • Display Name: Enter a display name for the target. If left blank, the Name field value is copied.
    • Description
    • Tags
    • Target Type: The selected target is displayed.
  4. Click Next.
  5. On the Target Details screen, enter the following details:
    • Connection: Select an AWS connection from the drop-down list.
    • File Name: Enter the name of the file used to save data to the AWS S3 bucket.
    • AWS S3 Path: Enter a name for folder to be created in the AWS S3 bucket. A new folder is created if there is no existing folder.
    • Bucket: Enter the name of the bucket to be created. A new bucket is created if there is no existing bucket in the region.

    • Roll Interval: Enter the roll-over interval to write a new file. The interval can be in 1000ms, 10s, 1m, or 1.5h format.

    • File Roll Max Size: Enter the roll-over file size to create a new file. The size can be in 1000, 10k, 1m, or 1g format.

    • NFS Path: Enter the local file or NFS path where the files are written first and then uploaded to the AWS S3 bucket.

    • Storage Format: Select a storage format from the drop-down list.

  6. Click Next.
  7. On the Data Format screen, enter the shape details, based on the storage format you have selected.
    • For FILE:

      • File Format: Select a file format from the drop-down list.

        • JSON Delimiter: Enter the JSON delimiter if you have selected the JSON file format. This is an optional field.
        • Avro Codec: Select a compression codec from the drop-down list. This option is enabled if you have selected the file format as AVRO or AVRO Object Container Format.
    • For PARQUET:
      • PARQUET Compression: Select a compression codec from the drop-down list.
    • For ORC:
      • ORC Compression: Select a compression codec from the drop-down list.
  8. Click Next.
  9. On the Shape screen, select one of the methods to define the shape:

    • Select Existing Shape: Select one of the existing shapes from the drop-down list.

    • Manual Shape: Select this option to manually infer the fields from a stream or file. You can also add to, or remove fields from, an existing shape. Enter the following details:
      • Shape Name: Enter a name for the shape.
      • Clear Fields: Click to delete all the fields in the shape.
      • Field Name: Add the necessary fields.
      • Field Type: Select the field data type from the drop-down list.
  10. Click Save.