Overview
Use the initialized Arize client to callarize.logTrainingRecords() with collections of predicted and actual labels, its feature inputs, and corresponding prediction ids. Once records are sent to Arize’s platform, you’ll be able to visualize and analyze data holistically or within aggregated slices and use this data to compare against your production and validation results.
arize.logTrainingRecords() returns a Response. You can await on the Response to complete to ensure successful delivery of records.
When logging a prediction for the first time for a new model, we classify the model in the Arize platform based on the data type of the prediction.
For more information on model schema discovery, visit here:
What Is A Model Schema
These are training “inferences”, the response of your model to training data or a sample of training data. It is not just the training data set. Inferences include the prediction output of the model and data inputs - they give the Arize platform the ability to compare the models response (evaluations) to data versus production.Different model versions will have different responses to the same training data and capturing inferences allows Arize to evaluate the response.
API
<T> Response logTrainingRecords(
final String modelId,
final String modelVersion,
final List<Map<String, ?>> features,
final List<Map<String, ?>> tags,
final List<Map<String, Embedding>> embeddingFeatures,
final List<T> predictionLabels,
final List<T> actualLabels,
final List<Long> predictionTimestamps) throws IOException, IllegalArgumentException;
API arguments
| Paramater | Data Type | Description | |
| modelId | String | The unique identifier for your model. | Required |
| modelVersion | String | Used to group together a subset of predictions and actuals for a given model_id. | Required |
| features | List<Map<String, ?>>Where value can be oneOf: String, int, long, short, double, float, boolean, List<String> | List of Maps containing human readable and debuggable model features. Keys must be Strings and values one of: String, int, long, short, double, float, boolean, List<String> | Optional |
| embeddingFeatures | Map<String, Embedding> | Map containing human readable and debuggable model embedding features. Map keys must be String and values Embedding | Optional |
| tags | Map<String, ?> Where value can be oneOf: String, int, long, short, double, float, boolean, List<String> | Map containing human readable and debuggable model features. Map keys must be String and values one of: String, int, long, short, double, float, boolean, List<String> | Optional |
| predictionLabels | List<T> where T is oneof String, boolean, int, long, short, float, double, ScoreCategorical | The predicted labels for your given model inputs contained in a List<T>Important: If sent in as an argument, entries are matched respectively to the entries in prediction ids, feature values, and feature importances in the same index.Important: Must have the same number of elements as feature, actuals, and importances is all sent together. | Required |
| actualLabels | List<T> where T is oneof String, boolean, int, long, short, float, double, ScoreCategorical | The actual observed labels for a given model input.Important: If passed together in a single call with predictionLabels, both inputs must have the same shape.Important: If model is Score Categorical, Arize.ScoreCategorical object should be passed in with corresponding predictedLabel, probabilityScore. | Required |
Sample Code
import com.arize.ArizeClient;
import com.arize.Response;
final ArizeClient arize = new ArizeClient(System.getenv("ARIZE_API_KEY"), System.getenv("ARIZE_SPACE_KEY"));
final List<Map<String, ?>> features = new ArrayList<Map<String, ?>>();
features.add(new HashMap<String, Object>() {{ put("days", 5); put("is_organic", 1);}});
features.add(new HashMap<String, Object>() {{ put("days", 3); put("is_organic", 0);}});
features.add(new HashMap<String, Object>() {{ put("days", 7); put("is_organic", 0);}});
final List<Map<String, Embedding>> embeddingFeatures = new ArrayList<Map<String, Embedding>>();
embeddingFeatures.add(new HashMap<String, Embedding>() {{ put("embedding_feature_1", new Embedding(Arrays.asList(1.0, 0.5), Arrays.asList("test", "token", "array"), "https://example.com/image.jpg")); put("embedding_feature_2", new Embedding(Arrays.asList(1.0, 0.8), Arrays.asList("this", "is"), "https://example.com/image_3.jpg"));}});
embeddingFeatures.add(new HashMap<String, Embedding>() {{ put("embedding_feature_1", new Embedding(Arrays.asList(0.0, 0.6), Arrays.asList("another", "example"), "https://example.com/image_2.jpg")); put("embedding_feature_2", new Embedding(Arrays.asList(0.1, 1.0), Arrays.asList("an", "example"), "https://example.com/image_4.jpg"));}});
embeddingFeatures.add(new HashMap<String, Embedding>() {{ put("embedding_feature_1", new Embedding(Arrays.asList(1.0, 0.8), Arrays.asList("third"), "https://example.com/image_3.jpg")); put("embedding_feature_2", new Embedding(Arrays.asList(1.0, 0.4), Arrays.asList("token", "array"), "https://example.com/image_5.jpg"));}});
final List<Map<String, ?>> tags = new ArrayList<Map<String, ?>>();
tags.add(new HashMap<String, Object>() {{ put("metadata", 5); put("my business metric", 1);}});
tags.add(new HashMap<String, Object>() {{ put("metadata", 3); put("my business metric", 0);}});
tags.add(new HashMap<String, Object>() {{ put("metadata", 7); put("my business metric", 8);}});
final List<String> predictionLabels = new ArrayList<String>(Arrays.asList("pear", "banana", "apple"));
final List<String> actualLabels = new ArrayList<String>(Arrays.asList("pear", "strawberry", "apple"));
final Response asyncResponse = arize.logTrainingRecords("exampleModelId", "v1", features, embeddingFeatures, tags, predictionLabels, actualLabels);
// This is a blocking call similar to future.get()
asyncResponse.resolve();
// Check that the API call was successful
switch (asyncResponse.getResponseCode()) {
case OK:
// TODO: Success!
System.out.println("Success!!!");
break;
case AUTHENTICATION_ERROR:
// TODO: Check to make sure your Arize API KEY and Space key are correct
break;
case BAD_REQUEST:
// TODO: Malformed request
System.out.println("Failure Reason: " + asyncResponse.getResponseBody());
case NOT_FOUND:
// TODO: API endpoint not found, client is likely malconfigured, make sure you
// are not overwriting Arize's endpoint URI
break;
case UNEXPECTED_FAILURE:
// TODO: Unexpected failure, check for a reason on response body
System.out.println("Failure Reason: " + asyncResponse.getResponseBody());
break;
}
System.out.println("Response Code: " + asyncResponse.getResponseCode());
System.out.println("Response Body: " + asyncResponse.getResponseBody());
// Don't forget to shutdown the client with your application shutdown hook.
arize.close();
System.out.println("Done");
Questions? Email us at support@arize.com or Slack us in the #arize-support channel