As I write this, less than a week after the end of .NET Conf 2023 and the release of .NET 8, .NET Aspire is in preview version (8.0.0-preview.1.23557.2). Therefore, it’s possible that some aspects may have changed by the time you read this article.
During the .NET Conf 2023, Microsoft announced .NET Aspire, a new .NET workload designed to ease the development of applications and microservices in a cloud-native context. Having personally experienced difficulties with developing and orchestrating multiple microservices in a local environment, I was pleasantly surprised by this announcement.
If you haven’t yet seen the deep-dive video by Glenn Condron and David Fowler about .NET Aspire, I invite you to immediately stop reading this article and watch it. It will better equip you to understand the rest of this discussion.
This isn’t just another high-level introductory article on .NET Aspire. I’m sure many others have already done that, and done it better than I could. What I want to delve into here concerns the inner workings of .NET Aspire, beyond its open-source code.
Being very familiar with the source code of the Tye project — the experiment that inspired Microsoft’s development teams to create .NET Aspire — one of my first reactions was to try to understand the internals of .NET Aspire. Specifically, I was interested in how it orchestrates the resources developers declare in their .NET Aspire host. How does .NET Aspire compile and launch other projects? How does it manage the lifecycle of arbitrary executables? How does it interact with the Docker engine to start containers? How does service discovery work?
In the next few minutes, you will discover that .NET Aspire, as it was presented, is just the tip of the iceberg. Indeed, .NET Aspire is built on top of an undocumented orchestrator, also developed by Microsoft. This is the Microsoft Developer Control Plane, otherwise referred to by the acronym DCP. In short, DCP is a sort of miniature Kubernetes, which can be controlled with tools such as kubectl or the official C# client for Kubernetes.
A different approach from project Tye
In the Tye project, the execution of services, containers, and other executables declared in the tye.yaml YAML configuration was orchestrated by C# code, as can be seen in the source code of the ProcessRunner class and the DockerRunner.
Tye’s application model, being aware of all the resources to orchestrate, thus knows all the URLs, ports, and connection strings of these resources. It can then inject them via environment variables, for example.
The application model of .NET Aspire is similar. Instead of using YAML code, the developer declares their resources with C# code.
var builder = DistributedApplication.CreateBuilder(args);
var backend = builder.AddProject<Projects.MyBackend>("backend");
var frontend = builder.AddProject<Projects.MyFrontend>("frontend")
.WithReference(backend);
builder.Build().Run();
The developer then has two ways to execute their host application:
With a simple dotnet run, .NET Aspire will:
- Build and consolidate its application model based on the resources declared by the developer.
- Start an instance of the Microsoft Developer Control Plane (DCP).
- Ask DCP to start the resources declared in the application model.
- Listen to DCP events and receive logs and traces from the started resources.
- Start and open the .NET Aspire dashboard.
This main mode of operation is what developers will use to start their local microservices environment. This is particularly what interests us in this article.
The second mode of operation allows .NET Aspire to generate a JSON manifest representing all the resources declared in the developer’s C# code.
dotnet run -- --publisher manifest --output-path manifest.json
This manifest can then be consumed by other tools such as azd to provision the Azure infrastructure necessary for running the application in a cloud environment.
You could also write your own tool that consumes this manifest and outputs Terraform code according to your needs.
The Microsoft Developer Control Plane, an unknown and undocumented component
Start a .NET Aspire host project and list the processes running on your machine. You should see the following processes (here, on Windows):
dcp.exedcpctrl.exedcpd.exe
These processes are part of the .NET Aspire workload. They can be found in the folder <dotnet-sdk-dir/packs/Aspire.Hosting.Orchestration.<RID>/8.0.0-preview.1.23557.2/tools/. This directory also includes an EULA file outlining the terms of use for “Microsoft Developer Control Plane”. I have searched extensively online, particularly on GitHub, for information regarding DCP, but I haven’t found anything that clarifies its purpose and operation. The DCP’s code does not seem to be open-source, at least not currently. According to this GitHub issue, the URL for the DCP Git repository is https://github.com/microsoft/usvc-apiserver, but it is private.
Let’s engage in some debugging within the publicly available code of .NET Aspire to figure out how it engages with DCP.
Starting DCP through .NET Aspire
At the beginning of the .NET Aspire host startup, the DcpHostService class launches an instance of the dcp.exe process.
ProcessSpec dcpProcessSpec = new ProcessSpec(dcpExePath)
{
WorkingDirectory = Directory.GetCurrentDirectory(),
Arguments = $"start-apiserver --monitor {Environment.ProcessId} --detach --kubeconfig \"{locations.DcpKubeconfigPath}\"",
OnOutputData = Console.Out.Write,
OnErrorData = Console.Error.Write,
};
There is a mention of kubeconfig. Initially, I thought that Kubernetes was being used for resource orchestration. With a breakpoint in the decompiled code, here is what the content of this kubeconfig file looks like:
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true
server: https://[::1]:60803
name: apiserver_cluster
contexts:
- context:
cluster: apiserver_cluster
user: apiserver_user
name: apiserver
current-context: apiserver
kind: Config
preferences: {}
users:
- name: apiserver_user
user:
token: owamluvvhtdf
My first reflex was to try to list all the pods with kubectl pointing to this kubeconfig file:
kubectl --kubeconfig "C:\Users\simmo\AppData\Local\Temp\aspire.jbnsxb1r.4u2\kubeconfig" get pods
But it did not work. Indeed, the server does not seem to recognize the resource type pods:
error: the server doesn't have a resource type "pods"
I then tried to list the supported resource types:
kubectl --kubeconfig "C:\Users\simmo\AppData\Local\Temp\aspire.jbnsxb1r.4u2\kubeconfig" api-resources
The result was much more interesting:
NAME SHORTNAMES APIVERSION NAMESPACED KIND
containers ctr usvc-dev.developer.microsoft.com/v1 false Container
containervolumes ctrvol usvc-dev.developer.microsoft.com/v1 false ContainerVolume
endpoints end usvc-dev.developer.microsoft.com/v1 false Endpoint
executablereplicasets exers usvc-dev.developer.microsoft.com/v1 false ExecutableReplicaSet
executables exe usvc-dev.developer.microsoft.com/v1 false Executable
services svc usvc-dev.developer.microsoft.com/v1 false Service
It appears that this DCP server, which resembles Kubernetes, supports resource types specific to .NET Aspire, especially Executable and Container.
Having started my .NET Aspire host with the starter template, I slightly modified my setup to have a frontend server, two backend server replicas, and a Redis container:
var builder = DistributedApplication.CreateBuilder(args);
var cache = builder.AddRedisContainer("cache");
var apiservice = builder.AddProject<Projects.Aspire01_ApiService>("apiservice")
.WithReplicas(2);
builder.AddProject<Projects.Aspire01_Web>("webfrontend")
.WithReference(cache)
.WithReference(apiservice);
builder.Build().Run();
I then tried to describe the Container resource associated with my Redis container:
> kubectl --kubeconfig "C:\Users\simmo\AppData\Local\Temp\aspire.jbnsxb1r.4u2\kubeconfig" get containers
NAME CREATED AT
cache 2023-11-19T04:38:57Z
> kubectl --kubeconfig "C:\Users\simmo\AppData\Local\Temp\aspire.jbnsxb1r.4u2\kubeconfig" describe containers cache
Name: cache
Namespace:
Labels: <none>
Annotations: service-producer: [{"serviceName":"cache","address":null,"port":6379}]
API Version: usvc-dev.developer.microsoft.com/v1
Kind: Container
Metadata:
Creation Timestamp: 2023-11-19T04:38:57Z
Finalizers:
usvc-dev.developer.microsoft.com/container-reconciler
Managed Fields:
API Version: usvc-dev.developer.microsoft.com/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"usvc-dev.developer.microsoft.com/container-reconciler":
Manager: dcpctrl.exe
Operation: Update
Time: 2023-11-19T04:38:57Z
API Version: usvc-dev.developer.microsoft.com/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:service-producer:
f:spec:
f:image:
f:ports:
f:restartPolicy:
Manager: unknown
Operation: Update
Time: 2023-11-19T04:38:57Z
Resource Version: 35
UID: 936080b7-e85c-4b1b-9366-3ec489a5895b
Spec:
Image: redis:latest
Ports:
Container Port: 6379
Protocol: TCP
Restart Policy: no
Status:
Container Id: 40f382e164839b6bb5cbc3e7acf25ff182141316eeb64cc1ee70cbf448581c12
Finish Timestamp: <nil>
Startup Timestamp: 2023-11-19T04:38:57Z
State: Running
Events: <none>
My Redis cache is indeed orchestrated by DCP. I can also see the Docker instance that was started in Docker Desktop:
DCP, a resource orchestrator controlled by .NET Aspire
Subsequently, I went through almost the entire codebase of .NET Aspire to better understand the nature of the project. I was still struggling to differentiate between the open-source .NET Aspire project and the responsibilities of the Microsoft Developer Control Plane.
With the help of Rider, I was able to debug the .NET Aspire code and see how it interacts with DCP. Once DCP is started and the application model (the list of resources and their dependencies) is consolidated, .NET Aspire uses the .NET Kubernetes client to execute “Kubernetes-ish” commands on DCP. The C# class in .NET Aspire responsible for interacting with DCP is ApplicationExecutor.
Here is the content of one of the HTTP requests sent by the .NET Aspire host to DCP to start my Redis container:
{
"spec": {
"image": "redis:latest",
"ports": [
{
"containerPort": 6379,
"protocol": "TCP"
}
],
"env": [],
"restartPolicy": "no"
},
"metadata": {
"annotations": {
"service-producer": "[{\"serviceName\":\"cache\",\"address\":null,\"port\":6379}]"
},
"name": "cache",
"namespace": ""
},
"apiVersion": "usvc-dev.developer.microsoft.com/v1",
"kind": "Container"
}
All these clues lead to the belief that DCP is the central element of .NET Aspire for local development. It is DCP that knows how to:
- Interact with Docker Desktop and start containers.
- Interact with the operating system and start arbitrary processes.
- Expose the ports of the started resources to the host machine.
- Manage the lifecycle of the started resources.
- Handle multiple replicas of the same resource.
What a pity that at this stage there is no documentation on DCP. Running dcp.exe --help does not yield anything very useful:
DCP is a developer tool for running multi-service applications.
It integrates your code, emulators and containers to give you a development environment
with minimum remote dependencies and maximum ease of use.
Usage:
dcp [command]
Available Commands:
generate-file Generate file from a template.
help Help about any command
up Runs an application
version Prints version information
Flags:
-h, --help help for dcp
-v, --verbosity level Logging verbosity level (e.g. -v=debug). Can be one of 'debug', 'info', or 'error', or any positive integer corresponding to increasing levels of debug verbosity. Levels more than 6 are rarely used in practice.
Use "dcp [command] --help" for more information about a command.
The command start-apiserver, which is used by .NET Aspire to start DCP, is not found there. One can try to invoke it manually with .\dcp.exe start-apiserver --help:
Starts the API server and controllers, but does not attempt to run any application
Usage:
dcp start-apiserver [flags]
Flags:
--detach If present, instructs DCP to fork itself as a detached process.
-h, --help help for start-apiserver
--kubeconfig string Paths to a kubeconfig. Only required if out-of-cluster.
-m, --monitor int If present, tells DCP to monitor a given process ID (PID) and gracefully shutdown if the monitored process exits for any reason. (default -1)
-i, --monitor-interval uint8 If present, specifies the time in seconds between checks for the monitor PID.
--port int32 Use a specific port when scaffolding the Kubeconfig file. If not specified, a random port will be used.
-r, --root-dir string If present, tells DCP to use specific directory as the application root directory. Defaults to current working directory.
Global Flags:
-v, --verbosity level Logging verbosity level (e.g. -v=debug). Can be one of 'debug', 'info', or 'error', or any positive integer corresponding to increasing levels of debug verbosity. Levels more than 6 are rarely used in practice.
This provides us with a bit more information. At this stage, we could imagine being able to interact directly with DCP. For that, we would need to know the specification of the resources supported by DCP. Fortunately, the open-source code of .NET Aspire already contains part of this information. In the folder src/Aspire.Hosting/Dcp/Model/, one can find the specifications of all the resources supported by DCP.
It seems there is still a long way to go for .NET Aspire and the Microsoft Developer Control Plane. For instance, the specification of a container describes that one could declare the command to execute within the container as well as the arguments. However, the public C# API of .NET Aspire does not yet allow these properties to be defined. Thus, it is impossible to start an instance of Redis or any other container with custom arguments.
Conclusion
.NET Aspire is a project which, in my opinion, has the potential to become an indispensable tool for facilitating the development of cloud-native applications, even if those applications are not developed with .NET. The Aspire dashboard and the presence of observability and instrumentation as first-class citizens will undoubtedly allow many developers to become aware of what is happening in their applications.
However, the public C# API of .NET Aspire does not yet allow today to exploit the full potential of the Microsoft Developer Control Plane. It is also not possible to stop or restart applications once the .NET Aspire host has started. If one of the resources crashes, the entire application must be restarted. The number of issues on GitHub has noticeably increased since the announcement of .NET Aspire at .NET Conf 2023. The Microsoft development teams will have a lot of work in the coming weeks to decide and prioritize the next features to implement.
For me, the big surprise was discovering the existence of the Microsoft Developer Control Plane. This program, probably coded in Go, seems much more complex than the C# code of .NET Aspire. I hope it will become open-source one day.
Buy me a coffee