{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Integrate with Prefect\n", "\n", "``rubicon_ml`` offers an integration with [Prefect](https://www.prefect.io/), an open source\n", "workflow management engine used heavily within the Python ecosystem. In Prefect, a unit of\n", "work is called a **task**, and a collection of **tasks** makes up a **flow**. A **flow**\n", "represents your workflow pipeline. You can integrate ``rubicon_ml`` into your workflows to\n", "persist metadata about your experimentation.\n", "\n", "We'll [run a Prefect server locally](https://docs.prefect.io/core/getting_started/installation.html#running-the-local-server-and-ui)\n", "for this example. If you already have Prefect and Docker installed, you can start a\n", "Prefect server and agent with these commands:\n", "\n", "```bash\n", "prefect backend server\n", "prefect server start\n", "\n", "# and in a new terminal\n", "prefect agent start\n", "```\n", "\n", "For more context, check out the\n", "[workflow README on GitHub](https://github.com/capitalone/rubicon-ml/tree/main/rubicon/workflow).\n", "\n", "### Setting up a simple flow\n", "\n", "Now we can get started! Creating Prefect **tasks** is easy enough on its own, but we've added\n", "some simple ones to the ``rubicon_ml`` library." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from rubicon_ml.workflow.prefect import (\n", " get_or_create_project_task,\n", " create_experiment_task,\n", " log_artifact_task,\n", " log_dataframe_task,\n", " log_feature_task,\n", " log_metric_task,\n", " log_parameter_task,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll need a workflow to integrate ``rubicon_ml`` logging into, so let's put together a simple one.\n", "To mimic a realistic example, we'll create tasks for loading data, splitting said data, extracting\n", "features, and training a model." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from prefect import task\n", "\n", "\n", "@task\n", "def load_data():\n", " from sklearn.datasets import load_wine\n", " \n", " return load_wine()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "@task\n", "def split_data(dataset):\n", " from sklearn.model_selection import train_test_split\n", " \n", " \n", " return train_test_split(\n", " dataset.data,\n", " dataset.target,\n", " test_size=0.25,\n", " )" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "@task\n", "def get_feature_names(dataset):\n", " return dataset.feature_names" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "@task\n", "def fit_pred_model(\n", " train_test_split_result,\n", " n_components,\n", " n_neighbors,\n", " is_standardized\n", "):\n", " from sklearn import metrics\n", " from sklearn.decomposition import PCA\n", " from sklearn.neighbors import KNeighborsClassifier\n", " from sklearn.pipeline import make_pipeline\n", " from sklearn.preprocessing import StandardScaler\n", "\n", " \n", " X_train, X_test, y_train, y_test = train_test_split_result\n", "\n", " if is_standardized:\n", " classifier = make_pipeline(\n", " StandardScaler(),\n", " PCA(n_components=n_components),\n", " KNeighborsClassifier(n_neighbors=n_neighbors),\n", " )\n", " else:\n", " classifier = make_pipeline(\n", " PCA(n_components=n_components),\n", " KNeighborsClassifier(n_neighbors=n_neighbors),\n", " )\n", " \n", " classifier.fit(X_train, y_train)\n", " pred_test = classifier.predict(X_test)\n", " accuracy = metrics.accuracy_score(y_test, pred_test)\n", " \n", " return accuracy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Without ``rubicon_ml``, here's what this simple **flow** would look like:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "from prefect import flow\n", "\n", "\n", "n_components = 2\n", "n_neighbors = 5\n", "is_standardized = True\n", "\n", "@flow\n", "def wine_flow():\n", " wine_dataset = load_data()\n", " \n", " feature_names = get_feature_names(wine_dataset)\n", " train_test_split = split_data(wine_dataset)\n", " \n", " return fit_pred_model(\n", " train_test_split,\n", " n_components,\n", " n_neighbors,\n", " is_standardized,\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Running a flow and viewing results\n", "\n", "Now we'll run the flow." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
00:56:44.598 | INFO | prefect.engine - Created flow run 'rugged-pony' for flow 'wine-flow'\n", "\n" ], "text/plain": [ "00:56:44.598 | \u001b[36mINFO\u001b[0m | prefect.engine - Created flow run\u001b[35m 'rugged-pony'\u001b[0m for flow\u001b[1;35m 'wine-flow'\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:44.646 | INFO | Flow run 'rugged-pony' - Created task run 'load_data-0' for task 'load_data'\n", "\n" ], "text/plain": [ "00:56:44.646 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'load_data-0' for task 'load_data'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:44.647 | INFO | Flow run 'rugged-pony' - Executing 'load_data-0' immediately...\n", "\n" ], "text/plain": [ "00:56:44.647 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'load_data-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.352 | INFO | Task run 'load_data-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.352 | \u001b[36mINFO\u001b[0m | Task run 'load_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.372 | INFO | Flow run 'rugged-pony' - Created task run 'get_feature_names-0' for task 'get_feature_names'\n", "\n" ], "text/plain": [ "00:56:45.372 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'get_feature_names-0' for task 'get_feature_names'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.373 | INFO | Flow run 'rugged-pony' - Executing 'get_feature_names-0' immediately...\n", "\n" ], "text/plain": [ "00:56:45.373 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'get_feature_names-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.423 | INFO | Task run 'get_feature_names-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.423 | \u001b[36mINFO\u001b[0m | Task run 'get_feature_names-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.440 | INFO | Flow run 'rugged-pony' - Created task run 'split_data-0' for task 'split_data'\n", "\n" ], "text/plain": [ "00:56:45.440 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'split_data-0' for task 'split_data'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.441 | INFO | Flow run 'rugged-pony' - Executing 'split_data-0' immediately...\n", "\n" ], "text/plain": [ "00:56:45.441 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'split_data-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.547 | INFO | Task run 'split_data-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.547 | \u001b[36mINFO\u001b[0m | Task run 'split_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.566 | INFO | Flow run 'rugged-pony' - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n", "\n" ], "text/plain": [ "00:56:45.566 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.567 | INFO | Flow run 'rugged-pony' - Executing 'fit_pred_model-0' immediately...\n", "\n" ], "text/plain": [ "00:56:45.567 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'fit_pred_model-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.693 | INFO | Task run 'fit_pred_model-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.693 | \u001b[36mINFO\u001b[0m | Task run 'fit_pred_model-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.719 | INFO | Flow run 'rugged-pony' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.719 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "0.9333333333333333" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "accuracy = wine_flow()\n", "accuracy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding Rubicon to your flow\n", "\n", "We can leverage the Prefect tasks within the ``rubicon_ml`` library to log all the\n", "info we want about our model run. Then, we can use the standard ``rubicon_ml`` logging\n", "library to retrieve and inspect our metrics and other logged data. This is much\n", "simpler than digging through the state of each executed **task** and extracting\n", "its results.\n", "\n", "Here's the same flow from above, this time with ``rubicon_ml`` **tasks** integrated." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "from prefect import unmapped\n", "\n", "\n", "n_components = 2\n", "n_neighbors = 5\n", "is_standardized = True\n", "\n", "@flow\n", "def rubicon_wine_flow(): \n", " project = get_or_create_project_task(\n", " \"memory\",\n", " \".\",\n", " \"Wine Classification with Prefect\",\n", " )\n", " experiment = create_experiment_task(\n", " project,\n", " name=\"logged from a Prefect task\",\n", " )\n", " \n", " wine_dataset = load_data()\n", " feature_names = get_feature_names(wine_dataset)\n", " train_test_split = split_data(wine_dataset)\n", " \n", " log_feature_task.map(unmapped(experiment), feature_names)\n", " log_parameter_task(experiment, \"n_components\", n_components)\n", " log_parameter_task(experiment, \"n_neighbors\", n_neighbors)\n", " log_parameter_task(experiment, \"is_standardized\", is_standardized)\n", " \n", " accuracy = fit_pred_model(\n", " train_test_split,\n", " n_components,\n", " n_neighbors,\n", " is_standardized,\n", " )\n", " \n", " log_metric_task(experiment, \"accuracy\", accuracy)\n", "\n", " return accuracy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, we'll register and run the **flow**." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
00:56:45.890 | INFO | prefect.engine - Created flow run 'laughing-newt' for flow 'rubicon-wine-flow'\n", "\n" ], "text/plain": [ "00:56:45.890 | \u001b[36mINFO\u001b[0m | prefect.engine - Created flow run\u001b[35m 'laughing-newt'\u001b[0m for flow\u001b[1;35m 'rubicon-wine-flow'\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.942 | INFO | Flow run 'laughing-newt' - Created task run 'get_or_create_project_task-0' for task 'get_or_create_project_task'\n", "\n" ], "text/plain": [ "00:56:45.942 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'get_or_create_project_task-0' for task 'get_or_create_project_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.943 | INFO | Flow run 'laughing-newt' - Executing 'get_or_create_project_task-0' immediately...\n", "\n" ], "text/plain": [ "00:56:45.943 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'get_or_create_project_task-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:45.995 | INFO | Task run 'get_or_create_project_task-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:45.995 | \u001b[36mINFO\u001b[0m | Task run 'get_or_create_project_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.011 | INFO | Flow run 'laughing-newt' - Created task run 'create_experiment_task-0' for task 'create_experiment_task'\n", "\n" ], "text/plain": [ "00:56:46.011 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'create_experiment_task-0' for task 'create_experiment_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.012 | INFO | Flow run 'laughing-newt' - Executing 'create_experiment_task-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.012 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'create_experiment_task-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.052 | INFO | Task run 'create_experiment_task-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.052 | \u001b[36mINFO\u001b[0m | Task run 'create_experiment_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.068 | INFO | Flow run 'laughing-newt' - Created task run 'load_data-0' for task 'load_data'\n", "\n" ], "text/plain": [ "00:56:46.068 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'load_data-0' for task 'load_data'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.069 | INFO | Flow run 'laughing-newt' - Executing 'load_data-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.069 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'load_data-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.112 | INFO | Task run 'load_data-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.112 | \u001b[36mINFO\u001b[0m | Task run 'load_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.129 | INFO | Flow run 'laughing-newt' - Created task run 'get_feature_names-0' for task 'get_feature_names'\n", "\n" ], "text/plain": [ "00:56:46.129 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'get_feature_names-0' for task 'get_feature_names'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.129 | INFO | Flow run 'laughing-newt' - Executing 'get_feature_names-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.129 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'get_feature_names-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.168 | INFO | Task run 'get_feature_names-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.168 | \u001b[36mINFO\u001b[0m | Task run 'get_feature_names-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.184 | INFO | Flow run 'laughing-newt' - Created task run 'split_data-0' for task 'split_data'\n", "\n" ], "text/plain": [ "00:56:46.184 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'split_data-0' for task 'split_data'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.185 | INFO | Flow run 'laughing-newt' - Executing 'split_data-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.185 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'split_data-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.232 | INFO | Task run 'split_data-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.232 | \u001b[36mINFO\u001b[0m | Task run 'split_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.311 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-12' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.311 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-12' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.312 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-12' for execution.\n", "\n" ], "text/plain": [ "00:56:46.312 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-12' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.322 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-7' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.322 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-7' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.323 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-7' for execution.\n", "\n" ], "text/plain": [ "00:56:46.323 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-7' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.334 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-8' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.334 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-8' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.335 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-8' for execution.\n", "\n" ], "text/plain": [ "00:56:46.335 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-8' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.356 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-6' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.356 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-6' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.357 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-6' for execution.\n", "\n" ], "text/plain": [ "00:56:46.357 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-6' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.399 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-3' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.399 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-3' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.400 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-3' for execution.\n", "\n" ], "text/plain": [ "00:56:46.400 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-3' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.417 | INFO | Task run 'log_feature_task-12' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.417 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-12' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.425 | INFO | Task run 'log_feature_task-8' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.425 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-8' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.444 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-10' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.444 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-10' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.445 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-10' for execution.\n", "\n" ], "text/plain": [ "00:56:46.445 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-10' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.459 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-4' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.459 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-4' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.461 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-4' for execution.\n", "\n" ], "text/plain": [ "00:56:46.461 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-4' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.499 | INFO | Task run 'log_feature_task-6' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.499 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-6' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.508 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-0' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.508 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-0' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.509 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-0' for execution.\n", "\n" ], "text/plain": [ "00:56:46.509 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-0' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.524 | INFO | Task run 'log_feature_task-7' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.524 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-7' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.543 | INFO | Task run 'log_feature_task-10' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.543 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-10' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.556 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-11' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.556 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-11' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.557 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-11' for execution.\n", "\n" ], "text/plain": [ "00:56:46.557 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-11' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.567 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-0' for task 'log_parameter_task'\n", "\n" ], "text/plain": [ "00:56:46.567 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-0' for task 'log_parameter_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.568 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.568 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.612 | INFO | Task run 'log_feature_task-11' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.612 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-11' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.620 | INFO | Task run 'log_feature_task-3' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.620 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-3' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.627 | INFO | Task run 'log_parameter_task-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.627 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.640 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-1' for task 'log_parameter_task'\n", "\n" ], "text/plain": [ "00:56:46.640 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-1' for task 'log_parameter_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.641 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-1' immediately...\n", "\n" ], "text/plain": [ "00:56:46.641 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-1' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.651 | INFO | Task run 'log_feature_task-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.651 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.668 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-1' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.668 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-1' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.669 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-1' for execution.\n", "\n" ], "text/plain": [ "00:56:46.669 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-1' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.696 | INFO | Task run 'log_parameter_task-1' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.696 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-1' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.713 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-2' for task 'log_parameter_task'\n", "\n" ], "text/plain": [ "00:56:46.713 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-2' for task 'log_parameter_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.714 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-2' immediately...\n", "\n" ], "text/plain": [ "00:56:46.714 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-2' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.724 | INFO | Task run 'log_feature_task-1' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.724 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-1' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.749 | INFO | Task run 'log_parameter_task-2' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.749 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-2' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.761 | INFO | Flow run 'laughing-newt' - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n", "\n" ], "text/plain": [ "00:56:46.761 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.761 | INFO | Flow run 'laughing-newt' - Executing 'fit_pred_model-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.761 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'fit_pred_model-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.771 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-2' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.771 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-2' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.772 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-2' for execution.\n", "\n" ], "text/plain": [ "00:56:46.772 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-2' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.817 | INFO | Task run 'fit_pred_model-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.817 | \u001b[36mINFO\u001b[0m | Task run 'fit_pred_model-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.839 | INFO | Flow run 'laughing-newt' - Created task run 'log_metric_task-0' for task 'log_metric_task'\n", "\n" ], "text/plain": [ "00:56:46.839 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_metric_task-0' for task 'log_metric_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.840 | INFO | Flow run 'laughing-newt' - Executing 'log_metric_task-0' immediately...\n", "\n" ], "text/plain": [ "00:56:46.840 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_metric_task-0' immediately...\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.854 | INFO | Task run 'log_feature_task-2' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.854 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-2' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.871 | INFO | Task run 'log_feature_task-4' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.871 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-4' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.885 | INFO | Task run 'log_metric_task-0' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:46.885 | \u001b[36mINFO\u001b[0m | Task run 'log_metric_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.971 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-5' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:46.971 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-5' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:46.972 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-5' for execution.\n", "\n" ], "text/plain": [ "00:56:46.972 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-5' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:47.004 | INFO | Task run 'log_feature_task-5' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:47.004 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-5' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:47.083 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-9' for task 'log_feature_task'\n", "\n" ], "text/plain": [ "00:56:47.083 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-9' for task 'log_feature_task'\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:47.084 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-9' for execution.\n", "\n" ], "text/plain": [ "00:56:47.084 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-9' for execution.\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:47.132 | INFO | Task run 'log_feature_task-9' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:47.132 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-9' - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
00:56:47.152 | INFO | Flow run 'laughing-newt' - Finished in state Completed()\n", "\n" ], "text/plain": [ "00:56:47.152 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m()\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "1.0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "accuracy = rubicon_wine_flow()\n", "accuracy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This time we can use ``rubicon_ml`` to inspect our accuracy, among other things!" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "experiment 35173154-a912-4ed0-9920-188d62e6f3d2\n", "features: ['proline', 'proanthocyanins', 'flavanoids', 'alcalinity_of_ash', 'nonflavanoid_phenols', 'hue', 'alcohol', 'od280/od315_of_diluted_wines', 'malic_acid', 'ash', 'magnesium', 'total_phenols', 'color_intensity']\n", "parameters: [('n_components', 2), ('n_neighbors', 5), ('is_standardized', True)]\n", "metrics: [('accuracy', 1.0)]\n" ] } ], "source": [ "from rubicon_ml import Rubicon\n", "\n", "\n", "rubicon = Rubicon(persistence=\"memory\", root_dir=\".\")\n", "project = rubicon.get_project(\"Wine Classification with Prefect\")\n", " \n", "experiment = project.experiments()[0]\n", "\n", "features = [f.name for f in experiment.features()]\n", "parameters = [(p.name, p.value) for p in experiment.parameters()]\n", "metrics = [(m.name, m.value) for m in experiment.metrics()]\n", "\n", "print(\n", " f\"experiment {experiment.id}\\n\"\n", " f\"features: {features}\\nparameters: {parameters}\\n\"\n", " f\"metrics: {metrics}\"\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 4 }