Agents/Parallel_Subtask_Agent_Workflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Orchestrator Subtask Agent Workflow\n",
    "Author: [Zain Hasan](https://x.com/ZainHasan6)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/togethercomputer/together-cookbook/blob/main/Agents/Parallel_Subtask_Agent_Workflow.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this notebook, we'll demonstrate how you can create an agent workflow that will break up original tasks into multiple subtasks. These subtasks are then completed using parallel LLM calls, and then these answers are either directly provided as output or synthesized into a singular response.\n",
    "\n",
    "This strategy is similar to the parallel execution of LLMs proposed in the [Mixture of Agents paper](https://arxiv.org/abs/2406.04692); however, in this setup, we have a dedicated orchestrator LLM that breaks a task up into smaller subtasks. Then each of the parallel worker LLMs executes a different aspect of the main task - the prompt going to the worker LLMs might be different, and the LLMs themselves might also be different for each subtask.\n",
    "\n",
    "For example, given a coding task, we might want to break it up into planning, coding, and testing steps. For each step, we would use different prompts and even different LLMs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Orchestrator Subtask Workflow\n",
    "\n",
    "<img src=\"../images/parallel_different.png\" width=\"700\">\n",
    "\n",
    "In this parallel **orchestrator subtask workflow**, we demonstrate how you can create an agent system that intelligently decomposes complex tasks into smaller, specialized components.\n",
    "\n",
    "The workflow begins with an orchestrator LLM that analyzes the main task and strategically breaks it down into distinct subtasks, which are then executed simultaneously by different worker LLMs.\n",
    "\n",
    "Unlike traditional parallel LLM approaches where multiple models work on the same task, this system assigns each worker LLM to a unique aspect of the overall problem, with customized prompts and potentially different model architectures suited to their specific subtask.\n",
    "\n",
    "For our specific use case, the workflow operates as follows:\n",
    "\n",
    "1. The orchestrator LLM analyzes the main task and breaks it into distinct, parallel subtasks\n",
    "2. Each subtask is assigned to an appropriate worker LLM with specialized prompts\n",
    "3. The results are either presented individually or synthesized into a unified response\n",
    "\n",
    "Now let's see the coded implementation of this workflow.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup and Utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install libraries\n",
    "!pip install -qU pydantic together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import libraries\n",
    "import json\n",
    "import asyncio\n",
    "import together\n",
    "from together import AsyncTogether, Together\n",
    "\n",
    "from typing import Any, Optional, Dict, List, Literal\n",
    "from pydantic import Field, BaseModel, ValidationError\n",
    "\n",
    "TOGETHER_API_KEY = \"--Your API Key--\"\n",
    "\n",
    "client = Together(api_key= TOGETHER_API_KEY)\n",
    "async_client = AsyncTogether(api_key= TOGETHER_API_KEY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Simple LLM call helper function\n",
    "def run_llm(user_prompt : str, model : str, system_prompt : Optional[str] = None):\n",
    "    \"\"\" Run the language model with the given user prompt and system prompt. \"\"\"\n",
    "    messages = []\n",
    "    if system_prompt:\n",
    "        messages.append({\"role\": \"system\", \"content\": system_prompt})\n",
    "    \n",
    "    messages.append({\"role\": \"user\", \"content\": user_prompt})\n",
    "    \n",
    "    response = client.chat.completions.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0.7,\n",
    "        max_tokens=4000,        \n",
    "    )\n",
    "\n",
    "    return response.choices[0].message.content\n",
    "\n",
    "# Simple JSON mode LLM call helper function\n",
    "def JSON_llm(user_prompt : str, schema : BaseModel, system_prompt : Optional[str] = None):\n",
    "    \"\"\" Run a language model with the given user prompt and system prompt, and return a structured JSON object. \"\"\"\n",
    "    try:\n",
    "        messages = []\n",
    "        if system_prompt:\n",
    "            messages.append({\"role\": \"system\", \"content\": system_prompt})\n",
    "        \n",
    "        messages.append({\"role\": \"user\", \"content\": user_prompt})\n",
    "        \n",
    "        extract = client.chat.completions.create(\n",
    "            messages=messages,\n",
    "            model=\"meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo\",\n",
    "            response_format={\n",
    "                \"type\": \"json_object\",\n",
    "                \"schema\": schema.model_json_schema(),\n",
    "            },\n",
    "        )\n",
    "        \n",
    "        response = json.loads(extract.choices[0].message.content)\n",
    "        return response\n",
    "        \n",
    "    except ValidationError as e:\n",
    "        raise ValueError(f\"Schema validation failed: {str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Orchestrator Agent Workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The function below will call the reference LLMs in parallel\n",
    "async def run_llm_parallel(user_prompt : str, model : str, system_prompt : str = None):\n",
    "    \"\"\"Run parallel LLM call with a reference model.\"\"\"\n",
    "    for sleep_time in [1, 2, 4]:\n",
    "        try:\n",
    "            messages = []\n",
    "            if system_prompt:\n",
    "                messages.append({\"role\": \"system\", \"content\": system_prompt})\n",
    "    \n",
    "            messages.append({\"role\": \"user\", \"content\": user_prompt})\n",
    "\n",
    "            response = await async_client.chat.completions.create(\n",
    "                model=model,\n",
    "                messages=messages,\n",
    "                temperature=0.7,\n",
    "                max_tokens=2000,\n",
    "            )\n",
    "            break\n",
    "        except together.error.RateLimitError as e:\n",
    "            print(e)\n",
    "            await asyncio.sleep(sleep_time)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will implement the workflow that uses this parallel LLM calling function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "from typing import Literal, List\n",
    "\n",
    "ORCHESTRATOR_PROMPT = \"\"\"\n",
    "Analyze this task and break it down into 2-3 distinct approaches:\n",
    "\n",
    "Task: {task}\n",
    "\n",
    "Provide an Analysis:\n",
    "\n",
    "Explain your understanding of the task and which variations would be valuable.\n",
    "Focus on how each approach serves different aspects of the task.\n",
    "\n",
    "Along with the analysis, provide 2-3 approaches to tackle the task, each with a brief description:\n",
    "\n",
    "Planning for the task: Write a detailed plan to execute on the task without actually solving the task\n",
    "Solving the task: Write a technical solution for the task provided\n",
    "Tests for the task: Write a potential test plan for a solution to the task provided, do not solve the task\n",
    "\n",
    "Return only JSON output.\n",
    "\"\"\"\n",
    "\n",
    "WORKER_PROMPT = \"\"\"\n",
    "Generate content based on:\n",
    "Task: {original_task}\n",
    "Style: {task_type}\n",
    "Guidelines: {task_description}\n",
    "\n",
    "Return only your response:\n",
    "[Your content here, maintaining the specified task and fully addressing requirements.]\n",
    "\"\"\"\n",
    "\n",
    "task = \"\"\"Write a program that prints the next 20 leap years.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Task(BaseModel):\n",
    "    type: Literal[\"plan\", \"code/solve\", \"test\"]\n",
    "    description: str\n",
    "\n",
    "class TaskList(BaseModel):\n",
    "    analysis: str\n",
    "    tasks: List[Task]  = Field(..., default_factory=list)\n",
    "\n",
    "async def orchestrator_workflow(task : str, orchestrator_prompt : str, worker_prompt : str): \n",
    "    \"\"\"Use a orchestrator model to break down a task into sub-tasks and then use worker models to generate and return responses.\"\"\"\n",
    "\n",
    "    # Use orchestrator model to break the task up into sub-tasks\n",
    "    orchestrator_response = JSON_llm(orchestrator_prompt.format(task=task), schema=TaskList)\n",
    " \n",
    "    # Parse orchestrator response\n",
    "    analysis = orchestrator_response[\"analysis\"]\n",
    "    tasks= orchestrator_response[\"tasks\"]\n",
    "\n",
    "    print(\"\\n=== ORCHESTRATOR OUTPUT ===\")\n",
    "    print(f\"\\nANALYSIS:\\n{analysis}\")\n",
    "    print(f\"\\nTASKS:\\n{json.dumps(tasks, indent=2)}\")\n",
    "\n",
    "    worker_model =  [\"Qwen/Qwen2.5-Coder-32B-Instruct\"]*len(tasks)\n",
    "\n",
    "    # Gather intermediate responses from worker models\n",
    "    return tasks , await asyncio.gather(*[run_llm_parallel(user_prompt=worker_prompt.format(original_task=task, task_type=task_info['type'], task_description=task_info['description']), model=model) for task_info, model in zip(tasks,worker_model)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== ORCHESTRATOR OUTPUT ===\n",
      "\n",
      "ANALYSIS:\n",
      "The task of writing a program that prints the next 20 leap years can be broken down into distinct approaches, each serving a different aspect of the task. These approaches include planning, solving, and testing. The planning approach involves outlining the steps required to execute the task, the solving approach involves writing a technical solution to the task, and the testing approach involves verifying that the solution works as expected. Valuable variations of this task could include handling different date ranges, handling different cultures or calendar systems, or optimizing the solution for performance.\n",
      "\n",
      "TASKS:\n",
      "[\n",
      "  {\n",
      "    \"type\": \"plan\",\n",
      "    \"description\": \"Write a detailed plan to execute on the task\"\n",
      "  },\n",
      "  {\n",
      "    \"type\": \"code/solve\",\n",
      "    \"description\": \"Write a technical solution for the task provided\"\n",
      "  },\n",
      "  {\n",
      "    \"type\": \"test\",\n",
      "    \"description\": \"Write a potential test plan for a solution to the task provided\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "task = \"\"\"Write a program that prints the next 20 leap years.\n",
    "\"\"\"\n",
    "\n",
    "tasks, worker_resp = await orchestrator_workflow(task, orchestrator_prompt=ORCHESTRATOR_PROMPT, worker_prompt=WORKER_PROMPT)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== WORKER RESULT (plan) ===\n",
      "### Plan to Write a Program That Prints the Next 20 Leap Years\n",
      "\n",
      "#### 1. Problem Understanding\n",
      "- **Objective**: Develop a program that outputs the next 20 leap years from the current year.\n",
      "- **Leap Year Criteria**:\n",
      "  - A year is a leap year if it is divisible by 4.\n",
      "  - However, if the year is divisible by 100, it is not a leap year, unless:\n",
      "  - The year is also divisible by 400, in which case it is a leap year.\n",
      "\n",
      "#### 2. Environment Setup\n",
      "- **Programming Language**: Choose Python due to its simplicity and readability.\n",
      "- **Development Environment**: Set up a Python development environment, such as using an IDE like PyCharm or a simple text editor with a Python interpreter.\n",
      "\n",
      "#### 3. Algorithm Development\n",
      "- **Step 1**: Determine the current year using the `datetime` module.\n",
      "- **Step 2**: Initialize a counter to keep track of the number of leap years found.\n",
      "- **Step 3**: Create a loop that checks each subsequent year to see if it is a leap year.\n",
      "- **Step 4**: For each leap year found, print it and increment the counter.\n",
      "- **Step 5**: Exit the loop once 20 leap years have been found and printed.\n",
      "\n",
      "#### 4. Implementation\n",
      "- **Step 1**: Import the `datetime` module to get the current year.\n",
      "- **Step 2**: Define a function `is_leap_year(year)` that returns `True` if the year is a leap year, and `False` otherwise.\n",
      "- **Step 3**: Use a `while` loop to iterate through years starting from the current year.\n",
      "- **Step 4**: Inside the loop, use the `is_leap_year` function to check if the current year is a leap year.\n",
      "- **Step 5**: If it is a leap year, print the year and increment the leap year counter.\n",
      "- **Step 6**: Break out of the loop once the counter reaches 20.\n",
      "\n",
      "#### 5. Testing\n",
      "- **Test Case 1**: Run the program and verify that it prints 20 consecutive leap years starting from the next leap year after the current year.\n",
      "- **Test Case 2**: Modify the current year to a known leap year and verify that the program correctly identifies and prints the next 20 leap years.\n",
      "- **Test Case 3**: Set the current year to a non-leap year and verify that the program correctly identifies the next leap year and prints the subsequent 20 leap years.\n",
      "\n",
      "#### 6. Code Review and Refinement\n",
      "- **Review**: Check the code for readability and efficiency.\n",
      "- **Refinement**: Make sure the code adheres to Python best practices, such as using meaningful variable names and including comments.\n",
      "\n",
      "#### 7. Documentation\n",
      "- **Comments**: Add comments within the code to explain the logic and purpose of each section.\n",
      "- **Readme File**: Create a README file that explains the purpose of the program, how to run it, and any dependencies.\n",
      "\n",
      "#### 8. Deployment\n",
      "- **Execution**: Run the program in a Python environment to ensure it functions as expected.\n",
      "- **Output**: Verify that the output is correct and matches the expected list of the next 20 leap years.\n",
      "\n",
      "#### 9. Maintenance\n",
      "- **Updates**: Plan for potential updates if the program needs to be modified for different requirements or future leap year calculations.\n",
      "- **Support**: Provide support or documentation for users who may need assistance with running or understanding the program.\n",
      "\n",
      "\n",
      "=== WORKER RESULT (code/solve) ===\n",
      "```python\n",
      "def is_leap_year(year):\n",
      "    \"\"\"Determine if a given year is a leap year.\"\"\"\n",
      "    return (year % 4 == 0 and year % 100 != 0) or (year % 400 == 0)\n",
      "\n",
      "def find_next_leap_years(start_year, count):\n",
      "    \"\"\"Find the next 'count' leap years starting from 'start_year'.\"\"\"\n",
      "    leap_years = []\n",
      "    year = start_year\n",
      "    while len(leap_years) < count:\n",
      "        if is_leap_year(year):\n",
      "            leap_years.append(year)\n",
      "        year += 1\n",
      "    return leap_years\n",
      "\n",
      "# Start from the current year or any specified year\n",
      "start_year = 2023\n",
      "next_leap_years = find_next_leap_years(start_year, 20)\n",
      "print(next_leap_years)\n",
      "```\n",
      "\n",
      "This program defines a function `is_leap_year` to check if a year is a leap year and another function `find_next_leap_years` to find the next 20 leap years starting from a specified year. It then prints these leap years.\n",
      "\n",
      "\n",
      "=== WORKER RESULT (test) ===\n",
      "### Test Plan for Leap Year Generator\n",
      "\n",
      "#### Objective:\n",
      "The objective of this test plan is to validate the correctness and functionality of a program that prints the next 20 leap years starting from the current year.\n",
      "\n",
      "#### Test Environment:\n",
      "- Programming Language: Python\n",
      "- Development IDE: PyCharm\n",
      "- Operating System: Windows 10\n",
      "\n",
      "#### Test Cases:\n",
      "\n",
      "1. **Test Case 1: Current Year is a Leap Year**\n",
      "   - **Input/Condition**: Current year is a leap year (e.g., 2024).\n",
      "   - **Expected Output**: The program should correctly identify and print the next 20 leap years starting from 2024.\n",
      "\n",
      "2. **Test Case 2: Current Year is Not a Leap Year**\n",
      "   - **Input/Condition**: Current year is not a leap year (e.g., 2021).\n",
      "   - **Expected Output**: The program should correctly identify and print the next 20 leap years starting from the next applicable leap year (e.g., 2024).\n",
      "\n",
      "3. **Test Case 3: Current Year is Close to a Leap Year**\n",
      "   - **Input/Condition**: Current year is close to a leap year (e.g., 2023).\n",
      "   - **Expected Output**: The program should correctly identify and print the next 20 leap years starting from the leap year that follows (e.g., 2024).\n",
      "\n",
      "4. **Test Case 4: Edge Case - Year 2100**\n",
      "   - **Input/Condition**: Check the behavior around the edge case year 2100.\n",
      "   - **Expected Output**: The year 2100 should not be considered a leap year, and the program should correctly print the next 20 leap years starting from 2024, skipping 2100.\n",
      "\n",
      "5. **Test Case 5: Correctness of Leap Year Logic**\n",
      "   - **Input/Condition**: Test a range of years including century years and non-century years.\n",
      "   - **Expected Output**: The program should accurately identify leap years according to the rules: divisible by 4, not divisible by 100 unless also divisible by 400.\n",
      "\n",
      "6. **Test Case 6: Performance Test**\n",
      "   - **Input/Condition**: Check the performance and efficiency of the program when calculating leap years.\n",
      "   - **Expected Output**: The program should execute efficiently and print the next 20 leap years within a reasonable timeframe.\n",
      "\n",
      "7. **Test Case 7: Output Format Verification**\n",
      "   - **Input/Condition**: Verify the format of the output.\n",
      "   - **Expected Output**: The output should be a list or sequence of 20 consecutive leap years, each on a new line or separated by a delimiter.\n",
      "\n",
      "8. **Test Case 8: No External Dependencies**\n",
      "   - **Input/Condition**: Ensure that the program does not rely on any external libraries or APIs.\n",
      "   - **Expected Output**: The program should be self-contained and use only standard Python libraries.\n",
      "\n",
      "9. **Test Case 9: Robustness to Invalid Inputs**\n",
      "   - **Input/Condition**: Provide invalid or non-applicable inputs (e.g., negative years).\n",
      "   - **Expected Output**: The program should handle invalid inputs gracefully and either prompt for valid input or terminate with an appropriate error message.\n",
      "\n",
      "10. **Test Case 10: Code Review and Best Practices**\n",
      "    - **Input/Condition**: Review the code for adherence to best practices, readability, and maintainability.\n",
      "    - **Expected Output**: The code should be clean, well-documented, and easy to understand.\n",
      "\n",
      "#### Test Execution:\n",
      "- **Test Execution Approach**: Manual and automated testing.\n",
      "- **Test Execution Tools**: Unit testing frameworks like `unittest` or `pytest` in Python.\n",
      "\n",
      "#### Test Data:\n",
      "- Use a variety of years including edge cases (e.g., century years) to ensure comprehensive testing.\n",
      "\n",
      "#### Risk Assessment:\n",
      "- **High-Risk**: Correctness of leap year logic.\n",
      "- **Medium-Risk**: Performance and efficiency.\n",
      "- **Low-Risk**: Output format and handling of invalid inputs.\n",
      "\n",
      "#### Post-Test Activities:\n",
      "- Review all test results and document any issues or bugs.\n",
      "- Perform code reviews and necessary refactoring.\n",
      "- Update the test plan as required based on test results and feedback.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for task_info, response in zip(tasks, worker_resp):\n",
    "    print(f\"\\n=== WORKER RESULT ({task_info['type']}) ===\\n{response}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Synthesize a final answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SYNTHESIZER_PROMPT = \"\"\"Given the task: {task} and the following responses below, that each address different aspects of the \n",
    "task, synthesize a final response.\n",
    "\n",
    "{worker_responses}\n",
    "\"\"\"\n",
    "concatenated_responses = \" \".join([f\"\\n=== WORKER RESULT ({task_info['type']}) ===\\n{response}\\n\" for task_info, response in zip(tasks, worker_resp)])\n",
    "\n",
    "final_answer = run_llm(SYNTHESIZER_PROMPT.format(task=task, worker_responses=concatenated_responses), model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "### Final Response: Program to Print the Next 20 Leap Years\n",
      "\n",
      "To develop a program that prints the next 20 leap years from the current year, we have outlined a comprehensive plan, implemented the solution in Python, and devised a thorough test plan. Below is a synthesized version of the key aspects:\n",
      "\n",
      "#### Program Overview\n",
      "\n",
      "The program is designed to identify and print the next 20 leap years, starting from the current year or any specified year. A leap year is defined as a year that is divisible by 4, except for years that are divisible by 100 but not by 400.\n",
      "\n",
      "#### Implementation\n",
      "\n",
      "The solution is implemented in Python, utilizing its simplicity and readability. The code consists of two primary functions:\n",
      "\n",
      "1. **`is_leap_year(year)`**: This function determines whether a given year is a leap year based on the leap year criteria.\n",
      "2. **`find_next_leap_years(start_year, count)`**: This function finds the next 'count' leap years starting from 'start_year'. It iterates through the years, checks each year using the `is_leap_year` function, and appends leap years to a list until it reaches the desired count.\n",
      "\n",
      "#### Code\n",
      "\n",
      "```python\n",
      "def is_leap_year(year):\n",
      "    \"\"\"Determine if a given year is a leap year.\"\"\"\n",
      "    return (year % 4 == 0 and year % 100 != 0) or (year % 400 == 0)\n",
      "\n",
      "def find_next_leap_years(start_year, count):\n",
      "    \"\"\"Find the next 'count' leap years starting from 'start_year'.\"\"\"\n",
      "    leap_years = []\n",
      "    year = start_year\n",
      "    while len(leap_years) < count:\n",
      "        if is_leap_year(year):\n",
      "            leap_years.append(year)\n",
      "        year += 1\n",
      "    return leap_years\n",
      "\n",
      "# Example usage\n",
      "start_year = 2023  # Start from the current year or any specified year\n",
      "next_leap_years = find_next_leap_years(start_year, 20)\n",
      "print(next_leap_years)\n",
      "```\n",
      "\n",
      "#### Testing\n",
      "\n",
      "A comprehensive test plan has been devised to ensure the correctness and functionality of the program. This includes testing various scenarios such as:\n",
      "\n",
      "- Starting from a leap year or a non-leap year\n",
      "- Handling edge cases like the year 2100\n",
      "- Verifying the correctness of the leap year logic\n",
      "- Checking performance and efficiency\n",
      "- Ensuring the program handles invalid inputs gracefully\n",
      "\n",
      "The test plan involves both manual and automated testing using Python's unit testing frameworks.\n",
      "\n",
      "#### Conclusion\n",
      "\n",
      "The program to print the next 20 leap years has been successfully designed, implemented, and tested. It provides a straightforward and efficient solution to the problem, adhering to best practices in programming and testing. The code is well-documented, readable, and maintainable, making it easy to understand and modify if necessary.\n"
     ]
    }
   ],
   "source": [
    "print(final_answer)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}