Chat Application using DeepSeek-R1 (Python)

This sample Python app uses the Azure AI Inference SDK to call the DeepSeek-R1 model to generate responses to user messages.

The project includes all the infrastructure and configuration needed to provision a DeepSeek deployment in Azure AI and deploy the app to Azure Container Apps using the Azure Developer CLI. By default, the app will use managed identity to authenticate with Azure AI.

We recommend first going through the deploying steps before running this app locally, since the local app needs credentials for Azure AI to work properly.

• Features
• Architecture diagram
• Getting started
  • GitHub Codespaces
  • VS Code Dev Containers
  • Local Environment
• Deploying
• Development server
• Guidance
  • Costs
  • Security Guidelines

Important Security Notice

This template, the application code and configuration it contains, has been built to showcase Microsoft Azure specific services and tools. We strongly advise our customers not to make this code part of their production environments without implementing or enabling additional security features. See Security Guidelines for more information on how to secure your deployment.

Features

• A Python Quart that uses the Azure AI Inference SDK package to generate responses to user messages.
• A basic HTML/JS frontend that streams responses from the backend using JSON Lines over a ReadableStream.
• Bicep files for provisioning Azure resources, including Azure AI Services, Azure Container Apps, Azure Container Registry, Azure Log Analytics, and RBAC roles.

Architecture diagram

Getting started

You have a few options for getting started with this template. The quickest way to get started is GitHub Codespaces, since it will setup all the tools for you, but you can also set it up locally.

GitHub Codespaces

You can run this template virtually by using GitHub Codespaces. The button will open a web-based VS Code instance in your browser:

1. Open the template (this may take several minutes):

   

2. Open a terminal window

3. Continue with the deploying steps

VS Code Dev Containers

A related option is VS Code Dev Containers, which will open the project in your local VS Code using the Dev Containers extension:

1. Start Docker Desktop (install it if not already installed)

2. Open the project:

   

3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window.

4. Continue with the deploying steps

Local Environment

If you're not using one of the above options for opening the project, then you'll need to:

1. Make sure the following tools are installed:

   • Azure Developer CLI (azd)
   • Python 3.10+
   • Docker Desktop
   • Git

2. Download the project code:

   azd init -t deepseek-python

3. Open the project folder

4. Create a Python virtual environment and activate it.

5. Install required Python packages:

   pip install -r requirements-dev.txt

6. Install the app as an editable package:

   python3 -m pip install -e src

7. Continue with the deploying steps.

Deploying

Once you've opened the project in Codespaces, in Dev Containers, or locally, you can deploy it to Azure.

Azure account setup

1. Sign up for a free Azure account and create an Azure Subscription.

2. Request access to Azure OpenAI Service by completing the form at https://aka.ms/oai/access and awaiting approval.

3. Check that you have the necessary permissions:

   • Your Azure account must have Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner. If you don't have subscription-level permissions, you must be granted RBAC for an existing resource group and deploy to that existing group.
   • Your Azure account also needs Microsoft.Resources/deployments/write permissions on the subscription level.

Deploying with azd

1. Login to Azure:

   azd auth login

2. Provision and deploy all the resources:

   azd up

   It will prompt you to provide an azd environment name (like "chat-app"), select a subscription from your Azure account, and select a location where DeepSeek-R1 is available (like "westus"). Then it will provision the resources in your account and deploy the latest code. If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the Azure AI resource.

3. When azd has finished deploying, you'll see an endpoint URI in the command output. Visit that URI, and you should see the chat app! 🎉

4. Remember to take down your app once you're no longer using it, either by deleting the resource group in the Portal or running this command:

   azd down

Continuous deployment with GitHub Actions

This project includes a Github workflow for deploying the resources to Azure on every push to main. That workflow requires several Azure-related authentication secrets to be stored as Github action secrets. To set that up, run:

azd pipeline config

Development server

Assuming you've run the steps to open the project and the steps in Deploying, you can now run the Quart app in your development environment:

1. Copy .env.sample.azure into .env:

   cp .env.sample .env

2. Run this command to get the value of AZURE_INFERENCE_ENDPOINT from your deployed resource group and paste it in the .env file:

   azd env get-value AZURE_INFERENCE_ENDPOINT

3. Run this command to get the value of AZURE_TENANT_ID from your deployed resource group and paste it in the .env file:

   azd env get-value AZURE_TENANT_ID

4. Run the development server:

   python -m quart --app src.quartapp run --port 50505 --reload

This will start the app on port 50505, and you can access it at http://localhost:50505.

Guidance

Costs

Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. However, Azure Container Registry has a fixed cost per registry per day.

You can try the Azure pricing calculator for the resources:

• Azure AI Service: S0 tier, DeepSeek-RW model. Pricing is based on token count. Pricing
• Azure Container App: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. Pricing
• Azure Container Registry: Basic tier. Pricing
• Log analytics: Pay-as-you-go tier. Costs based on data ingested. Pricing

⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, either by deleting the resource group in the Portal or running azd down.

Security Guidelines

This template uses Managed Identity for authenticating to the Azure OpenAI service.

This template also enables the Container Apps built-in authentication feature with a Microsoft Entra ID identity provider. The Bicep files use the new Microsoft Graph extension (public preview) to create the Entra application registration using managed identity with Federated Identity Credentials, so that no client secrets or certificates are necessary.

Additionally, we have added a GitHub Action that scans the infrastructure-as-code files and generates a report containing any detected issues. To ensure continued best practices in your own repository, we recommend that anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled.

You may want to consider additional security measures, such as:

• Protecting the Azure Container Apps instance with a firewall and/or Virtual Network.
• Enabling Microsoft Defender for Cloud on the resource group and setting up security policies.