{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Integrate with Prefect\n",
"\n",
"``rubicon_ml`` offers an integration with [Prefect](https://www.prefect.io/), an open source\n",
"workflow management engine used heavily within the Python ecosystem. In Prefect, a unit of\n",
"work is called a **task**, and a collection of **tasks** makes up a **flow**. A **flow**\n",
"represents your workflow pipeline. You can integrate ``rubicon_ml`` into your workflows to\n",
"persist metadata about your experimentation.\n",
"\n",
"We'll [run a Prefect server locally](https://docs.prefect.io/core/getting_started/installation.html#running-the-local-server-and-ui)\n",
"for this example. If you already have Prefect and Docker installed, you can start a\n",
"Prefect server and agent with these commands:\n",
"\n",
"```bash\n",
"prefect backend server\n",
"prefect server start\n",
"\n",
"# and in a new terminal\n",
"prefect agent start\n",
"```\n",
"\n",
"For more context, check out the\n",
"[workflow README on GitHub](https://github.com/capitalone/rubicon-ml/tree/main/rubicon/workflow).\n",
"\n",
"### Setting up a simple flow\n",
"\n",
"Now we can get started! Creating Prefect **tasks** is easy enough on its own, but we've added\n",
"some simple ones to the ``rubicon_ml`` library."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from rubicon_ml.workflow.prefect import (\n",
" get_or_create_project_task,\n",
" create_experiment_task,\n",
" log_artifact_task,\n",
" log_dataframe_task,\n",
" log_feature_task,\n",
" log_metric_task,\n",
" log_parameter_task,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We'll need a workflow to integrate ``rubicon_ml`` logging into, so let's put together a simple one.\n",
"To mimic a realistic example, we'll create tasks for loading data, splitting said data, extracting\n",
"features, and training a model."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from prefect import task\n",
"\n",
"\n",
"@task\n",
"def load_data():\n",
" from sklearn.datasets import load_wine\n",
" \n",
" return load_wine()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"@task\n",
"def split_data(dataset):\n",
" from sklearn.model_selection import train_test_split\n",
" \n",
" \n",
" return train_test_split(\n",
" dataset.data,\n",
" dataset.target,\n",
" test_size=0.25,\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"@task\n",
"def get_feature_names(dataset):\n",
" return dataset.feature_names"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"@task\n",
"def fit_pred_model(\n",
" train_test_split_result,\n",
" n_components,\n",
" n_neighbors,\n",
" is_standardized\n",
"):\n",
" from sklearn import metrics\n",
" from sklearn.decomposition import PCA\n",
" from sklearn.neighbors import KNeighborsClassifier\n",
" from sklearn.pipeline import make_pipeline\n",
" from sklearn.preprocessing import StandardScaler\n",
"\n",
" \n",
" X_train, X_test, y_train, y_test = train_test_split_result\n",
"\n",
" if is_standardized:\n",
" classifier = make_pipeline(\n",
" StandardScaler(),\n",
" PCA(n_components=n_components),\n",
" KNeighborsClassifier(n_neighbors=n_neighbors),\n",
" )\n",
" else:\n",
" classifier = make_pipeline(\n",
" PCA(n_components=n_components),\n",
" KNeighborsClassifier(n_neighbors=n_neighbors),\n",
" )\n",
" \n",
" classifier.fit(X_train, y_train)\n",
" pred_test = classifier.predict(X_test)\n",
" accuracy = metrics.accuracy_score(y_test, pred_test)\n",
" \n",
" return accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Without ``rubicon_ml``, here's what this simple **flow** would look like:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from prefect import flow\n",
"\n",
"\n",
"n_components = 2\n",
"n_neighbors = 5\n",
"is_standardized = True\n",
"\n",
"@flow\n",
"def wine_flow():\n",
" wine_dataset = load_data()\n",
" \n",
" feature_names = get_feature_names(wine_dataset)\n",
" train_test_split = split_data(wine_dataset)\n",
" \n",
" return fit_pred_model(\n",
" train_test_split,\n",
" n_components,\n",
" n_neighbors,\n",
" is_standardized,\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Running a flow and viewing results\n",
"\n",
"Now we'll run the flow."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"00:56:44.598 | INFO | prefect.engine - Created flow run 'rugged-pony' for flow 'wine-flow'\n",
"\n"
],
"text/plain": [
"00:56:44.598 | \u001b[36mINFO\u001b[0m | prefect.engine - Created flow run\u001b[35m 'rugged-pony'\u001b[0m for flow\u001b[1;35m 'wine-flow'\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:44.646 | INFO | Flow run 'rugged-pony' - Created task run 'load_data-0' for task 'load_data'\n",
"\n"
],
"text/plain": [
"00:56:44.646 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'load_data-0' for task 'load_data'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:44.647 | INFO | Flow run 'rugged-pony' - Executing 'load_data-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:44.647 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'load_data-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.352 | INFO | Task run 'load_data-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.352 | \u001b[36mINFO\u001b[0m | Task run 'load_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.372 | INFO | Flow run 'rugged-pony' - Created task run 'get_feature_names-0' for task 'get_feature_names'\n",
"\n"
],
"text/plain": [
"00:56:45.372 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'get_feature_names-0' for task 'get_feature_names'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.373 | INFO | Flow run 'rugged-pony' - Executing 'get_feature_names-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:45.373 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'get_feature_names-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.423 | INFO | Task run 'get_feature_names-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.423 | \u001b[36mINFO\u001b[0m | Task run 'get_feature_names-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.440 | INFO | Flow run 'rugged-pony' - Created task run 'split_data-0' for task 'split_data'\n",
"\n"
],
"text/plain": [
"00:56:45.440 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'split_data-0' for task 'split_data'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.441 | INFO | Flow run 'rugged-pony' - Executing 'split_data-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:45.441 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'split_data-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.547 | INFO | Task run 'split_data-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.547 | \u001b[36mINFO\u001b[0m | Task run 'split_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.566 | INFO | Flow run 'rugged-pony' - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n",
"\n"
],
"text/plain": [
"00:56:45.566 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.567 | INFO | Flow run 'rugged-pony' - Executing 'fit_pred_model-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:45.567 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Executing 'fit_pred_model-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.693 | INFO | Task run 'fit_pred_model-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.693 | \u001b[36mINFO\u001b[0m | Task run 'fit_pred_model-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.719 | INFO | Flow run 'rugged-pony' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.719 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'rugged-pony'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"0.9333333333333333"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"accuracy = wine_flow()\n",
"accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Adding Rubicon to your flow\n",
"\n",
"We can leverage the Prefect tasks within the ``rubicon_ml`` library to log all the\n",
"info we want about our model run. Then, we can use the standard ``rubicon_ml`` logging\n",
"library to retrieve and inspect our metrics and other logged data. This is much\n",
"simpler than digging through the state of each executed **task** and extracting\n",
"its results.\n",
"\n",
"Here's the same flow from above, this time with ``rubicon_ml`` **tasks** integrated."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from prefect import unmapped\n",
"\n",
"\n",
"n_components = 2\n",
"n_neighbors = 5\n",
"is_standardized = True\n",
"\n",
"@flow\n",
"def rubicon_wine_flow(): \n",
" project = get_or_create_project_task(\n",
" \"memory\",\n",
" \".\",\n",
" \"Wine Classification with Prefect\",\n",
" )\n",
" experiment = create_experiment_task(\n",
" project,\n",
" name=\"logged from a Prefect task\",\n",
" )\n",
" \n",
" wine_dataset = load_data()\n",
" feature_names = get_feature_names(wine_dataset)\n",
" train_test_split = split_data(wine_dataset)\n",
" \n",
" log_feature_task.map(unmapped(experiment), feature_names)\n",
" log_parameter_task(experiment, \"n_components\", n_components)\n",
" log_parameter_task(experiment, \"n_neighbors\", n_neighbors)\n",
" log_parameter_task(experiment, \"is_standardized\", is_standardized)\n",
" \n",
" accuracy = fit_pred_model(\n",
" train_test_split,\n",
" n_components,\n",
" n_neighbors,\n",
" is_standardized,\n",
" )\n",
" \n",
" log_metric_task(experiment, \"accuracy\", accuracy)\n",
"\n",
" return accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Again, we'll register and run the **flow**."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"00:56:45.890 | INFO | prefect.engine - Created flow run 'laughing-newt' for flow 'rubicon-wine-flow'\n",
"\n"
],
"text/plain": [
"00:56:45.890 | \u001b[36mINFO\u001b[0m | prefect.engine - Created flow run\u001b[35m 'laughing-newt'\u001b[0m for flow\u001b[1;35m 'rubicon-wine-flow'\u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.942 | INFO | Flow run 'laughing-newt' - Created task run 'get_or_create_project_task-0' for task 'get_or_create_project_task'\n",
"\n"
],
"text/plain": [
"00:56:45.942 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'get_or_create_project_task-0' for task 'get_or_create_project_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.943 | INFO | Flow run 'laughing-newt' - Executing 'get_or_create_project_task-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:45.943 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'get_or_create_project_task-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:45.995 | INFO | Task run 'get_or_create_project_task-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:45.995 | \u001b[36mINFO\u001b[0m | Task run 'get_or_create_project_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.011 | INFO | Flow run 'laughing-newt' - Created task run 'create_experiment_task-0' for task 'create_experiment_task'\n",
"\n"
],
"text/plain": [
"00:56:46.011 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'create_experiment_task-0' for task 'create_experiment_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.012 | INFO | Flow run 'laughing-newt' - Executing 'create_experiment_task-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.012 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'create_experiment_task-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.052 | INFO | Task run 'create_experiment_task-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.052 | \u001b[36mINFO\u001b[0m | Task run 'create_experiment_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.068 | INFO | Flow run 'laughing-newt' - Created task run 'load_data-0' for task 'load_data'\n",
"\n"
],
"text/plain": [
"00:56:46.068 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'load_data-0' for task 'load_data'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.069 | INFO | Flow run 'laughing-newt' - Executing 'load_data-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.069 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'load_data-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.112 | INFO | Task run 'load_data-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.112 | \u001b[36mINFO\u001b[0m | Task run 'load_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.129 | INFO | Flow run 'laughing-newt' - Created task run 'get_feature_names-0' for task 'get_feature_names'\n",
"\n"
],
"text/plain": [
"00:56:46.129 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'get_feature_names-0' for task 'get_feature_names'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.129 | INFO | Flow run 'laughing-newt' - Executing 'get_feature_names-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.129 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'get_feature_names-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.168 | INFO | Task run 'get_feature_names-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.168 | \u001b[36mINFO\u001b[0m | Task run 'get_feature_names-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.184 | INFO | Flow run 'laughing-newt' - Created task run 'split_data-0' for task 'split_data'\n",
"\n"
],
"text/plain": [
"00:56:46.184 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'split_data-0' for task 'split_data'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.185 | INFO | Flow run 'laughing-newt' - Executing 'split_data-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.185 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'split_data-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.232 | INFO | Task run 'split_data-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.232 | \u001b[36mINFO\u001b[0m | Task run 'split_data-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.311 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-12' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.311 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-12' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.312 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-12' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.312 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-12' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.322 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-7' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.322 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-7' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.323 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-7' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.323 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-7' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.334 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-8' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.334 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-8' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.335 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-8' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.335 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-8' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.356 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-6' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.356 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-6' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.357 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-6' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.357 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-6' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.399 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-3' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.399 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-3' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.400 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-3' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.400 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-3' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.417 | INFO | Task run 'log_feature_task-12' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.417 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-12' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.425 | INFO | Task run 'log_feature_task-8' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.425 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-8' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.444 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-10' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.444 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-10' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.445 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-10' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.445 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-10' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.459 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-4' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.459 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-4' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.461 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-4' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.461 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-4' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.499 | INFO | Task run 'log_feature_task-6' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.499 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-6' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.508 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-0' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.508 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-0' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.509 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-0' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.509 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-0' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.524 | INFO | Task run 'log_feature_task-7' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.524 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-7' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.543 | INFO | Task run 'log_feature_task-10' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.543 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-10' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.556 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-11' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.556 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-11' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.557 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-11' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.557 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-11' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.567 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-0' for task 'log_parameter_task'\n",
"\n"
],
"text/plain": [
"00:56:46.567 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-0' for task 'log_parameter_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.568 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.568 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.612 | INFO | Task run 'log_feature_task-11' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.612 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-11' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.620 | INFO | Task run 'log_feature_task-3' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.620 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-3' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.627 | INFO | Task run 'log_parameter_task-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.627 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.640 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-1' for task 'log_parameter_task'\n",
"\n"
],
"text/plain": [
"00:56:46.640 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-1' for task 'log_parameter_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.641 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-1' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.641 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-1' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.651 | INFO | Task run 'log_feature_task-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.651 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.668 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-1' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.668 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-1' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.669 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-1' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.669 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-1' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.696 | INFO | Task run 'log_parameter_task-1' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.696 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-1' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.713 | INFO | Flow run 'laughing-newt' - Created task run 'log_parameter_task-2' for task 'log_parameter_task'\n",
"\n"
],
"text/plain": [
"00:56:46.713 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_parameter_task-2' for task 'log_parameter_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.714 | INFO | Flow run 'laughing-newt' - Executing 'log_parameter_task-2' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.714 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_parameter_task-2' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.724 | INFO | Task run 'log_feature_task-1' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.724 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-1' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.749 | INFO | Task run 'log_parameter_task-2' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.749 | \u001b[36mINFO\u001b[0m | Task run 'log_parameter_task-2' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.761 | INFO | Flow run 'laughing-newt' - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n",
"\n"
],
"text/plain": [
"00:56:46.761 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'fit_pred_model-0' for task 'fit_pred_model'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.761 | INFO | Flow run 'laughing-newt' - Executing 'fit_pred_model-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.761 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'fit_pred_model-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.771 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-2' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.771 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-2' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.772 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-2' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.772 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-2' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.817 | INFO | Task run 'fit_pred_model-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.817 | \u001b[36mINFO\u001b[0m | Task run 'fit_pred_model-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.839 | INFO | Flow run 'laughing-newt' - Created task run 'log_metric_task-0' for task 'log_metric_task'\n",
"\n"
],
"text/plain": [
"00:56:46.839 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_metric_task-0' for task 'log_metric_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.840 | INFO | Flow run 'laughing-newt' - Executing 'log_metric_task-0' immediately...\n",
"\n"
],
"text/plain": [
"00:56:46.840 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Executing 'log_metric_task-0' immediately...\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.854 | INFO | Task run 'log_feature_task-2' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.854 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-2' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.871 | INFO | Task run 'log_feature_task-4' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.871 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-4' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.885 | INFO | Task run 'log_metric_task-0' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:46.885 | \u001b[36mINFO\u001b[0m | Task run 'log_metric_task-0' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.971 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-5' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:46.971 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-5' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:46.972 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-5' for execution.\n",
"\n"
],
"text/plain": [
"00:56:46.972 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-5' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:47.004 | INFO | Task run 'log_feature_task-5' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:47.004 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-5' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:47.083 | INFO | Flow run 'laughing-newt' - Created task run 'log_feature_task-9' for task 'log_feature_task'\n",
"\n"
],
"text/plain": [
"00:56:47.083 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Created task run 'log_feature_task-9' for task 'log_feature_task'\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:47.084 | INFO | Flow run 'laughing-newt' - Submitted task run 'log_feature_task-9' for execution.\n",
"\n"
],
"text/plain": [
"00:56:47.084 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Submitted task run 'log_feature_task-9' for execution.\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:47.132 | INFO | Task run 'log_feature_task-9' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:47.132 | \u001b[36mINFO\u001b[0m | Task run 'log_feature_task-9' - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"00:56:47.152 | INFO | Flow run 'laughing-newt' - Finished in state Completed()\n",
"\n"
],
"text/plain": [
"00:56:47.152 | \u001b[36mINFO\u001b[0m | Flow run\u001b[35m 'laughing-newt'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m()\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"1.0"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"accuracy = rubicon_wine_flow()\n",
"accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This time we can use ``rubicon_ml`` to inspect our accuracy, among other things!"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"experiment 35173154-a912-4ed0-9920-188d62e6f3d2\n",
"features: ['proline', 'proanthocyanins', 'flavanoids', 'alcalinity_of_ash', 'nonflavanoid_phenols', 'hue', 'alcohol', 'od280/od315_of_diluted_wines', 'malic_acid', 'ash', 'magnesium', 'total_phenols', 'color_intensity']\n",
"parameters: [('n_components', 2), ('n_neighbors', 5), ('is_standardized', True)]\n",
"metrics: [('accuracy', 1.0)]\n"
]
}
],
"source": [
"from rubicon_ml import Rubicon\n",
"\n",
"\n",
"rubicon = Rubicon(persistence=\"memory\", root_dir=\".\")\n",
"project = rubicon.get_project(\"Wine Classification with Prefect\")\n",
" \n",
"experiment = project.experiments()[0]\n",
"\n",
"features = [f.name for f in experiment.features()]\n",
"parameters = [(p.name, p.value) for p in experiment.parameters()]\n",
"metrics = [(m.name, m.value) for m in experiment.metrics()]\n",
"\n",
"print(\n",
" f\"experiment {experiment.id}\\n\"\n",
" f\"features: {features}\\nparameters: {parameters}\\n\"\n",
" f\"metrics: {metrics}\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}