> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Programmatically Create Monitors

> Learn how to query, patch, and create monitors using our programmatic API

Monitors keep track of the various facets of your models. They track key metrics over time and let you analyze incidents that may be causing your model to degrade. The Arize platform provides workflows to troubleshoot incidents whenever a monitor is triggered.

However, it's also useful to export existing monitors programmatically and make configuration changes outside of the Arize platform. The [Arize GraphQL API](https://arize.com/docs/ax/integrations/graphql-api) is designed to do just that - to allow you to build tooling that lets you manage monitors using your own custom workflows.

For a brief overview of GraphQL itself, please consult our [introduction](/ax/graphql-reference/overview/how-to-use-graphql).

### Querying for Monitors

Monitors belong to a `Model` and thus also belong to a `Space`. Because of this, you can either query for monitors off of a `Model` node or a `Space` node.

<Info>
  The model or space id is in the url when you visit the corresponding page. The format looks like this: `spaces/:``space`**\_id**`/models/:``model_id``.`

  You can also query for a list of models given the space id. More details [here](/ax/graphql-reference/overview/how-to-use-graphql/querying-nested-data#2.-get-the-model-connection-in-your-space).
</Info>

<Tabs>
  <Tab title="Query by Model">
    ```javascript theme={null}
    query {
      node(id: "model_id") {
        ... on Model {
          monitors(first: 50) {
            edges {
              node {
                id
                name
                monitorCategory
              }
            }
          }
        }
      }
    }
    ```
  </Tab>

  <Tab title="Query by Space">
    ```javascript theme={null}
    query {
      node(id: "space_id") {
        ... on Space {
          monitors(first: 50) {
            edges {
              node {
                id
                name
                monitorCategory
              }
            }
          }
        }
      }
    }
    ```
  </Tab>
</Tabs>

The `MonitorsConnection` off of `Model` and `Space` provide a convenient way for you to pull monitors that match the given criteria. If you have a large number of monitors, you will have to use pagination to pull the complete list. This data can then be structured to your liking and be exported to the platform of your choosing.

For a full working example, consult one of our example use case [Colabs](/ax/machine-learning/machine-learning/how-to-ml/monitors/monitors-api#examples).

### Updating / Patching Monitors

If you are managing a large number of monitors with Arize, you may want to periodically update certain parts of the configuration of a group of monitors. The Arize API provides `patch` mutations ([partial modification](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PATCH)) for each type of monitor. Here are a few examples:\\

<Tabs>
  <Tab title="Patch Drift Monitor">
    ```php theme={null}
    mutation updateDriftMonitorThreshold($monitorId: ID!, $threshold: Float!) {
      patchDriftMonitor(
        input: {
          monitorId: $monitorId,
          set: {
            threshold: $threshold,
            dynamicAutoThresholdEnabled: false
          }
        }
      ) {
        monitor { id, name, threshold }
      }
    }
    ```

    **variables**

    ```json theme={null}
    {
      "monitorId": "monitor_id",
      "threshold": 0.25
    }
    ```
  </Tab>

  <Tab title="Patch Performance Monitor">
    ```php theme={null}
    mutation updatePerfMonitorThreshold($monitorId: ID!, $threshold: Float!) {
      patchPerformanceMonitor(
        input: {
          monitorId: $monitorId,
          set: {
            threshold: $threshold,
            dynamicAutoThresholdEnabled: false
          }
        }
      ) {
        monitor { id, name, threshold }
      }
    }
    ```

    **variables**

    ```json theme={null}
    {
      "monitorId": "monitor_id"
      "threshold": 0.97
    }
    ```
  </Tab>
</Tabs>

Each `patch` mutation takes a monitor's `node_id` an a `set` input that contains the properties of the monitor. Each mutation returns the monitor that has been modified - there is no need to re-query for the data! View an example Colab below.

<Frame caption="Patching Drift Monitors">
  <Card icon="https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/gc.ico" horizontal title="Google Collaboratory" href="https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Update_Drift_Monitors_Use_Case.ipynb" />
</Frame>

<Info>
  Notice that the mutations above make use of naming a query and using variables to make the mutations re-usable.

  This is a very powerful abstraction and will help you construct maintainable queries and mutations. See [GraphQL docs](https://graphql.org/learn/queries/)
</Info>

### Deleting Monitors

Deleting monitors using a mutation simply requires you to know the `ID` of the monitor you want to delete.

```javascript theme={null}
mutation {
  deleteMonitor(input: {monitorId: "monitor_id"}) {
    clientMutationId
  }
}
```

### Muting Monitors

You can mute a monitor by using the generic `patchMonitor` or a specific mutation like `patchDriftMonitor`, `patchPerformanceMonitor`, or `patchDataQualityMonitor`.

[](#tab-generic-mutation)
[](#tab-specific-mutation)

<Tabs>
  <Tab title="Generic mutation">
    ```javascript theme={null}
    mutation {
      patchMonitor(
        input: {
          monitorId: "monitor_id"
          set: {
            notificationsEnabled: false
          }
        }) {
        clientMutationId
      }
    }
    ```
  </Tab>

  <Tab title="Specific mutation">
    ```javascript theme={null}
    mutation {
      patchMonitor(
        input: {
          monitorId: "monitor_id"
          set: {
            notificationsEnabled: false
          }
        }) {
        clientMutationId
      }
    }
    ```
  </Tab>
</Tabs>

### Creating Monitors

In most cases, the Arize platform will provide you with the best experience for creating monitors as the UI gives you a live preview of your monitor as you fill out its configuration. Additionally, Arize's managed monitors, auto-thresholding capabilities, and model baselines can often provide great monitoring coverage for your models.

However, there may be cases where you would like to create a known set of custom monitors that track a specific cohort (filtered dataset) of features or a set of monitors that track a specific tag. Programmatically creating monitors can provide you the flexibility you need so you can ensure sufficient monitoring coverage.

Let's suppose you have a lending model and you would like to exhaustively monitor the performance of the model for all 50 states. You may want to create a re-usable mutation that looks something like

```php theme={null}
mutation createPerformanceMonitorForState($state: String!){
  createPerformanceMonitor(
    input: {
      modelId: "model_id",
      operator: lessThan,
      performanceMetric: accuracy,
      dynamicAutoThreshold: {
        stdDevMultiplier: 1.2
      }
      filters: [{
        dimensionType: featureLabel,
        operator: equals
        name: "stateName"
        values: [$state]
      }],
      contacts: [{
        notificationChannelType: email,
        emailAddress: "you@me.com"
      }]
    }
  ) {
    monitor {
      id
    }
  }
}
```

Notice that we have used a variable to pass in a state to the `createPerformanceMonitor` mutation. We now can loop over all 50 states, invoke this mutation and get 100% coverage for all 50 states. The end result will be 50 monitors that will alert you when the performance dips below the specified threshold.

#### Thresholding

Two different variants of thresholds are supported: fixed thresholds and dynamic auto thresholds. Fixed thresholds are a great fit when you explicitly know when you would like the monitor to fire. However, if you would like the monitor to automatically calculate a threshold for you, you can supply an auto-threshold configuration. Note that by default, auto thresholds are enabled with a standard deviation multiplier of 2.

[](#tab-auto-threshold)
[](#tab-manual-threshold)

<Tabs>
  <Tab title="Auto Threshold">
    ```
    dynamicAutoThreshold: {
      stdDevMultiplier: 2
    }
    ```
  </Tab>

  <Tab title="Manual Threshold">
    ```
    threshold: 0.01
    ```
  </Tab>
</Tabs>

This configuration will set your threshold to `mean +/- (stdDevMultiplier * stdDev)` of a historical sample. The `+/-` part of the threshold calculation is determined by the `operator`.

<Info>
  Having trouble? Reach out to us via email [support@arize.com](mailto::support@arize.com) or [Slack us](https://arize-ai.slack.com/) in the #arize-support channel for more support.
</Info>

#### Monitor Filters

The `filters` list lets you filter down the metric you are monitoring down to a cohort of inferences. It lets you filter down the data you are monitoring by features, tags, and actuals. Here are some examples:

[](#tab-features-and-tags)
[](#tab-predictions)
[](#tab-actuals)
[](#tab-graphql-type)

<Tabs>
  <Tab title="Features and Tags">
    ```csharp theme={null}
    filters: [
        {
            dimensionType: featureLabel,
            name: "stateName"
            operator: equals
            values: ["california"]
        },
        {
            dimensionType: tagLabel,
            name: "business_impact_score"
            operator: greaterThan
            values: ["2"]
        }
    ]
    ```
  </Tab>

  <Tab title="Predictions">
    ```csharp theme={null}
    filters: [
        {
            dimensionType: predictionClass,
            operator: equals
            values: ["fraud"]
        },
        {
            dimensionType: predictionScore,
            operator: lessThan
            values: ["0.7"]
        }
    ]
    ```
  </Tab>

  <Tab title="Actuals">
    ```csharp theme={null}
    filters: [
        {
            dimensionType: actualClass,
            operator: equals
            values: ["fraud"]
        },
        {
            dimensionType: actualScore,
            operator: lessThan
            values: ["0.7"]
        }
    ]
    ```
  </Tab>

  <Tab title="GraphQL Type">
    ```python theme={null}
    """Filters for a particular model's inference data"""
    input DimensionFilterInput {
      """The part of the model to filter on."""
      dimensionType: DimensionFilterType!

      """The operator of the filter."""
      operator: ComparisonOperator!

      """Required when specifying a tag or a feature label filter."""
      name: String

      """The values to match on."""
      values: [String!]!

    }

    ```
  </Tab>
</Tabs>

<Info>
  Note that numeric filter values are passed in as strings but they must be parsable as numbers.
</Info>

#### Drift Monitor Baselines

Drift is monitored by comparing your model's current distribution to a baseline distribution. For more information on baselines and how to choose a baseline, refer to our documentation [here](https://arize.com/docs/ax/machine-learning/machine-learning/concepts-ml/glossary#baseline). There are a few ways to set a baseline using our API:

<Tabs>
  <Tab title="Model Baseline">
    A model baseline refers to the baseline used for *all* monitors in a model. Your model is preset with a baseline defined by a moving time range of your model's production data for 17 days.

    ```php theme={null}
    mutation {
    createDriftMonitor(input: {
      modelId: "model_id"
      dimensionCategory: predictionClass
      name: "Drift Monitor using Moving Custom Baseline"
      operator: greaterThan
      driftMetric: psi
      baseline: {
          useModelPrimaryBaseline: true,
      }
    }) {
      monitor {
        uri
      }
    }
    }
    ```
  </Tab>

  <Tab title="Custom Moving Baseline">
    Custom baselines can be set for individual monitors separate of the overall model baseline. In this case, the baseline utilizes a *moving* time window.

    ```php theme={null}
    mutation {
      createDriftMonitor(input: {
        modelId: "model_id"
        dimensionCategory: predictionClass
        name: "Drift Monitor using Moving Custom Baseline"
        operator: greaterThan
        driftMetric: psi
        baseline: {
            useModelPrimaryBaseline: false,
            movingWindowSeconds: 604800, // 1 week
            movingWindowDelaySeconds: 864000, // 10 days
        }
      }) {
        monitor {
          uri
        }
      }
    }
    ```
  </Tab>

  <Tab title="Custom Fixed Baseline">
    Custom baselines can be set for individual monitors separate of the overall model baseline. In this case, the baseline utilizes a *fixed* time window.

    ```php theme={null}
    mutation {
      createDriftMonitor(input: {
        modelId: "model_id"
        dimensionCategory: predictionClass
        name: "Drift Monitor using Fixed Custom Baseline"
        operator: greaterThan
        driftMetric: psi
        baseline: {
            useModelPrimaryBaseline: false,
            fixedStartDate: "2020-12-23T04:59:59.999Z",
            fixedEndDate: "2021-12-23T04:59:59.999Z",
        }
      }) {
        monitor {
          uri
        }
      }
    }
    ```
  </Tab>
</Tabs>

<Frame caption="Creating Performance Monitors">
  <Card icon="https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/gc.ico" horizontal title="Google Collaboratory" href="https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Create_Performance_Monitors_Use_Case.ipynb" />
</Frame>

### Querying for Integrations

Integrations belong to a `Model` and thus also belong to a `Space`. Because of this, you can either query for monitors off of a `Model` node or a `Space` node.

<Info>
  The model or space id is in the url when you visit the corresponding page. The format looks like this: `spaces/:``space`**\_id**`/models/:``model_id``.`

  You can also query for a list of models given the space id. More details [here](/ax/graphql-reference/overview/how-to-use-graphql/querying-nested-data#2.-get-the-model-connection-in-your-space).
</Info>

[](#tab-query-by-model-1)
[](#tab-query-by-space-1)

<Tabs>
  <Tab title="Query by Model">
    ```javascript theme={null}
    query {
      node(id: "model_id") {
        ... on Model {
          monitors(first: 50) {
            edges {
              node {
                id
                name
                monitorCategory
              }
            }
          }
        }
      }
    }
    ```
  </Tab>

  <Tab title="Query by Space">
    ```javascript theme={null}
    query {
      node(id: "space_id") {
        ... on Space {
          monitors(first: 50) {
            edges {
              node {
                id
                name
                monitorCategory
              }
            }
          }
        }
      }
    }
    ```
  </Tab>
</Tabs>

### Monitoring Alerts

Monitors surface potential issues with performance and data, sending real-time alerts so you can take immediate action. Learn how to programmatically update email alerts and integrate with other altering systems.

#### Setting Monitor Alerting Contacts

You can update either an email or paging service your monitor alerts by using the generic `patchMonitor` or a specific mutation such as: `patchDriftMonitor`, `patchPerformanceMonitor`, or `patchDataQualityMonitor`.

To view and set up external integrations [see below](/ax/machine-learning/machine-learning/how-to-ml/monitors/configure-monitors/notifications-and-integrations).

<Tab title="Generic mutation">
  ```javascript theme={null}
  mutation {
    patchMonitor(
      input: $input) {
      clientMutationId
    }
  }
  ```

  variables

  ```json theme={null}
  {
    "monitorId": ":monitor_id",
    "set": {
      "contacts": [
        {
          "emailAddress": "example-email-contact@test.com",
          "notificationChannelType": "email"
        },
        {
          "integrationKeyId": ":integration_id",
          "notificationChannelType": "integration"
        }
      ]
    }
  }
  ```
</Tab>

[](#tab-generic-mutation-1)

### Alerting Integrations

Natively integrate with an alerting service to identify and resolve issues faster. Learn more about our alerting integrations [here](/ax/machine-learning/machine-learning/how-to-ml/monitors/configure-monitors/notifications-and-integrations).

#### Querying Current Integrations

Integrations belong to an `AccountOrganization`, thus belonging to an `Account`. Because of this, you must query for integrations off of an `AccountOrganization` node.

<Info>
  `You can find the account organization in the url organizations/``:account_organization_id``/`
</Info>

[](#tab-query)
[](#tab-create)
[](#tab-patch)
[](#tab-delete)
[](#tab-test-integration)

<Tabs>
  <Tab title="Query">
    ```javascript theme={null}
    query GetIntegrationsForOrganization {
      node(id: "account_organization_id") {
        ... on AccountOrganization {
          integrations {
            id
            providerName
            name
          }
        }
      }
    }
    ```
  </Tab>

  <Tab title="Create">
    ```python theme={null}
    mutation CreateIntegrationKey($input: CreateIntegrationKeyInput!) {
      createIntegrationKey(input: $input) {
        integrationKey {
          providerName
          name
          id
        }
      }
    }
    ```

    variables

    ```json theme={null}
    {
      "input": {
        "providerName": "provider_name",
        "accountOrganizationId": "account_organization_id",
        "serviceName": "your_custom_service_name",
        "apiKey": "api_key_from_provider"
      }
    }
    ```
  </Tab>

  <Tab title="Patch">
    ```python theme={null}
    mutation PatchIntegrationKey($input: PatchIntegrationKeyInput!) {
      patchIntegrationKey(input: $input) {
        integrationKey {
          providerName
          name
          id
        }
      }
    }
    ```

    variables

    ```json theme={null}
    {
      "input": {
        "integrationKeyId": "integration_key_id",
        "set": {
          "serviceName": "new custom service name",
          "apiKey": "new_api_key_from_provider"
        }
      }
    }
    ```
  </Tab>

  <Tab title="Delete">
    ```javascript theme={null}
    mutation DeleteIntegrationKey {
      deleteIntegrationKey(input: {integrationKeyId: "integration_key_to_delete"}) {
        integrationKey {
          providerName
          name
          id
        }
      }
    }
    ```
  </Tab>

  <Tab title="Test Integration">
    ```javascript theme={null}
    mutation {
      testIntegrationKey(
        input: {
          integrationKeyId: "integration_key_id"
        }
      ) {
        statusCode
      }
    }
    ```
  </Tab>
</Tabs>

### Examples

| API Use Cases                  | Example Colabs                                                                                                                                            |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Patching Drift Monitors        | [Colab](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Update_Drift_Monitors_Use_Case.ipynb)        |
| Creating Performance Monitors  | [Colab](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Create_Performance_Monitors_Use_Case.ipynb)  |
| Creating Data Quality Monitors | [Colab](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Create_Data_Quality_Monitors_Use_Case.ipynb) |
| Creating Drift Monitors        | [Colab](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/GraphQL/Create_Drift_Monitors_Use_Case.ipynb)        |