Skip to content

Branchpythonoperator Concepts

BranchPythonOperator

There isn't very good documentation for this feature yet.

Why?

The BranchPythonOperator is used to create control flow in a DAG.

Towards Automating Changes to Data

If there is unexpected input from a data source, then automate preprocessing the new data format so the automation can be improved.

NB: This type of branch would probably trigger another DAG.

Inter-task communication

There are two options:

  • variables
  • XComs

Variables sit in a global variables object, where as XComs are pushed and pulled between tasks.

For task-to-task communication, we will use XComs.

Passing Context to BranchPythonOperator

BranchPythonOperator inherits from PythonOperator and is defined in the same module. Both share use of provide_context=True as a keyword argument. If this is passed, the python_callable function must receive **kwargs. The context includes task_instance.xcom_pull which pulls information from other tasks.

Notes

Each branch must point to a task; if there isn't anything to do, use a DummyOperator.

References

General

Examples

Concepts & Tutorials

API Reference

  • BaseOperator
  • TaskInstance.xcom_pull
    • BaseOperator.xcom_pull imports TaskInstance.xcom_pull from context
    • You must pass key=None or any desired key/value to get tasks that are from xcom_push
    • By default tasks from xcom_push are ignored

Source Code