> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Airflow Retrain

> AWS Example

The following example shows how to kick off an Airflow retrain of a model from AWS.

<Warning>
  This setup uses **email** based triggers based on feedback from security teams. The goal of this approach was to kick off retraining events **without** requiring **direct network IP connectivity** between Arize and the customers AWS account.
</Warning>

<Frame caption="Arize to AWS event flow">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/38bbf8f8-image.jpeg" />
</Frame>

The flow of events on a model retrain:

1. Email generated on monitoring firing

2. SES receives the monitoring email

3. SES triggers lambda function

4. Lambda function contains all logic to kick off airflow retrain

### SES (Simple Email Service) Identity

This section walks through setting up the SES service.

Setup an SES Identity, this verifies the ownership of a domain or email address such that Amazon SES can receive emails sent to that address.

<Frame caption="Create Identity">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/9d8bb56f-image.jpeg" />
</Frame>

Once the identity is setup you can setup a rule to receive emails.

### Create Lambda Function for Airflow Retrain

This section walks through creating a lambda function that we will reference in the email reception call.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/bbc48feb-image.jpeg" />
</Frame>

This example uses a Python Lambda function.

The code of the Lambda function will follow this example:

<Frame caption="Example Kicking off Aiflow in Lambda">
  <Card title="Invoking DAGs with an AWS Lambda function - Amazon Managed Workflows for Apache Airflow" href="https://docs.aws.amazon.com/mwaa/latest/userguide/samples-lambda.html" icon="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/awsdocum.ico" horizontal>
    Amazon Managed Workflows for Apache Airflow
  </Card>
</Frame>

```python theme={null}
# Airflow V2
import boto3
import http.client
import base64
import ast
mwaa_env_name = 'YOUR_ENVIRONMENT_NAME'
dag_name = 'YOUR_DAG_NAME'
mwaa_cli_command = 'dags trigger'

client = boto3.client('mwaa')

def lambda_handler(event, context):
    # get web token
    mwaa_cli_token = client.create_cli_token(
        Name=mwaa_env_name
    )
    
    conn = http.client.HTTPSConnection(mwaa_cli_token['WebServerHostname'])
    payload = "dags trigger " + dag_name
    headers = {
      'Authorization': 'Bearer ' + mwaa_cli_token['CliToken'],
      'Content-Type': 'text/plain'
    }
    conn.request("POST", "/aws_mwaa/cli", payload, headers)
    res = conn.getresponse()
    data = res.read()
    dict_str = data.decode("UTF-8")
    mydata = ast.literal_eval(dict_str)
    return base64.b64decode(mydata['stdout'])
```

The above code shows an example of kicking off an Airflow job using a labmda function.

### SES (Simple Email Service) Rule

Lastly we are going to create a rule in our SES service that references the lambda function and calls it when the email is received.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/cb6ea740-image.jpeg" />
</Frame>

In the email receiving section create a rule set and then create a rule.

<Frame caption="Create Rule">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/c96c43cd-image.jpeg" />
</Frame>

Name the rule that will call the lambda function when the email is received.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/71ea6364-image.jpeg" />
</Frame>

Choose an email address to use in the Arize platform in the recipient section of the rule, emails to this address will execute the rule.

<Frame caption="Lambda Reference">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/359c960a-image.jpeg" />
</Frame>

The AWS Lambda function reference should be chosen in the action section.

<Info>
  On initial setup when you are debugging, it can be useful to setup a 2nd action with "**Deliver to S3 bucket**" so you can review received messages as well.
</Info>

<Frame caption="Lambda-airflow-retrain">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/0b1231dc-image.jpeg" />
</Frame>

In the lambda function reference the function name we created in the previous section - lambda-airflow-retrain.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/64a5cb52-image.jpeg" />
</Frame>

Create the rule.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/0a3d43c0-image.jpeg" />
</Frame>

Finally, the rule does need to be set as active to work, go back to email receiving at set rule as active.

### Use Lambda-Airflow Email Address in Arize

Use the email address created for the rule to send for key retrain alerts.

<Frame caption="">
  <img src="https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/0c51eb32-image.jpeg" />
</Frame>
