Skip to content

Latest commit

 

History

History
23 lines (13 loc) · 1.77 KB

README.md

File metadata and controls

23 lines (13 loc) · 1.77 KB

Interaction with StreamSets Control Hub pipelines

This set contains tutorials for StreamSets Control Hub pipelines.

A pipeline describes the flow of data from the origin system to destination systems and defines how to transform the data along the way.

Prerequisites

Before starting on any of the tutorials in this set, make sure to complete Prerequisites for the pipelines tutorial.

Tutorials for Pipelines

  1. Common pipeline methods - Common operations for StreamSets Control Hub pipelines like update, duplicate, import, export.

  2. Loop over pipelines and stages and make an edit to stages - When there are many pipelines and stages that need an update, SDK for Python makes it easy to update them with just a few lines of code.

  3. Create CI CD pipeline used in demo - This covers the steps to create CI CD pipeline as used in the SCH CI CD demo. The steps include how to add stages like JDBC, some processors and Kineticsearch; and how to set stage configurations. Also shows, the use of runtime parameters.

Conclusion

To get to know more details about SDK for Python, check the SDK documentation.

If you don't have access to SCH, sign up for 30-day free trial by visiting https://streamsets.com/products/sch/control-hub-trial.