Introduction

This article explores various ways in which Dapr helps with development and operations of distributed applications.

If you build software today, you are likely building a distributed system. Whether it’s two services talking or a monolith calling a lambda, you have crossed the process boundary.

Distributed applications are software systems that consist of multiple components or modules running independently over network. These components work together to achieve a common goal while communicating and coordinating their actions across the network.

Most organizations build distributed applications for two main reasons.

  1. They have multiple development teams that need to work independently while contributing to a larger system.

  2. They want to scale up or down specific parts of the application depending on business needs

  3. They require a solution where components built using different programming languages can interact with each other.

Developing distributed applications is challenging because numerous components need to work cohesively. Developers must consider resiliency, observability, security, and scalability across multiple services and runtimes. Furthermore, distributed applications typically don’t operate in isolation; they interact with message brokers, data stores, and external services. Integrating with these resources requires knowledge of specific APIs and SDKs which increases the complexity to build such systems.

To summarize,

  • Development - They are harder to develop and operate. Programming models are often optimized for in-process communication in terms of development and testing. However, Services, in a distributed application, involve remote/out of process calls (e.g. https/gRPC). This makes it difficult to program, debug/test and operate them.
  • Eventual Consistency - Microservices introduce eventual consistency because of their laudable insistence on decentralized data management. One is most likely to face issue of “Dual writes” (The dual-write problem occurs when two external systems must be updated in an atomic fashion) in distributed application.
  • Operability - Managing multiple independent services have its pros but induce complexity when it comes to frequent deployments and monitoring. As aptly put by Martin fowler here, “Low bar for skill is higher in a microservice environment”.
  • Deployment - Cloud brings in another dimension where companies want to leverage them for their benefits but want to be agnostic or avoid lock-in to a particular vendor.

All these aspects are articulated well in Fallacies of Distributed Computing.

This post details how Dapr helps with Distributed Applications Development as well as Agentic workloads.

Dapr

Dapr, short for Distributed Application Runtime, is a CNCF graduated open source project. In a nutshell, it provides runtime containing set of building blocks, that are useful for distributed application, and is used during local development and running in production.

Since it’s inception in 2019, it has evolved and is battle tested. True to the moto of CNCF, It provides vendor-neutral approach that decouples the infrastructure from application code so developers can focus on business logic instead of infra. plumbing.

It uses a sidecar pattern that runs as a separate process alongside the application. It provides multiple building blocks, from service invocation and messaging, jobs, workflow to distributed lock, all of which are common patterns in distributed communication. It makes it seamless to design, develop and run distributed applications from local environment to remote (be it private data center or cloud).

Key benefits of Dapr are,

  • No more boilerplate: Store state through the Dapr API and let the platform team pick Redis, Cosmos DB, DynamoDB, etc. Doing pub/sub messaging? Swap RabbitMQ for Kafka via config, not code.

  • Polyglot by design: Java, .NET, Python, Go, JavaScript; mix and match safely across teams.

  • Built-in Resilience and security : mTLS, retries, circuit breakers, and consistent observability patterns are part of the runtime “golden path.”

  • Operability - Supports integration with observability tools to make it easier to run the system.

  • sidecars, runtime, components, and configuration can all be managed and deployed easily and securely to match your organization’s needs.

    Abstractions by Dapr

Let us look at some of building blocks. For complete list, refer here.

Service invocation

Helps abstract service discovery and perform secure calls between services, transparently manages it via mDns in local setup and can be integrated with external store like Hashicorp consul.

Typical use case(s),

  • Abstract details (location etc.) needed for invoking services thus making them deployable on various target platforms without code changes

Workflows

Code-first approach for authoring workflow steps. Diagrid offers workflow code generator which prompts for workflow modelled as diagram (UML, BPMN etc.) and generates boilerplate code that uses Dapr SDK. Check it out here.

Typical use cases,

  • Event-driven data pipelines and ETL jobs
  • Payment and transaction processing with compensations
  • Customer onboarding and document verification flows
  • Automated deployment or model training pipelines
  • IoT device provisioning and firmware rollout sequences

Pub/sub

Provides a platform-agnostic API to send and receive messages for a service. Avoids hard dependency on specific Messaging system.

State Management

Provides HTTP API to persist and retrieve key-value pairs from State while having pluggable state store. supports automatic client encryption of application state with support for key rotations. As part of state management, it enables developers to use the outbox pattern for achieving a single transaction across a transactional state store and any message broker. Refer here for more details.

Jobs

jobs API is an orchestrator for scheduling these future jobs, either at a specific time or for a specific interval.

Typical use cases,

  • Batch processing - Long running processing of large number of transactions. reports generation etc.

Demo

For the purpose of the Demo, Lets have hypothetical use case for a Tenant Management System with below services,

  • Tenant Service - Responsible for managing lifecycle of Tenant starting with onboarding. It receives request for new tenant, validates it and invokes notification to intimate tenant itself via e-mail.

  • Notification Service - This services exposes interface to receive requests from notification and then pushes it to pub/sub queue for processing by specific services meant for sending e-mail, SMS and so on.

Below is depiction of the flow,

use case flow
  • Setup Dapr by following instructions here

  • Tenant service is developed using Spring Boot while Notification Service uses .NET Core.

    • Below snapshot shows Controller class of Tenant Service that calls notification service,

      
         @PostMapping
      public ResponseEntity<Tenant> createTenant(@Valid @RequestBody Tenant tenant) {
          log.info("POST /tenant - Create tenant request received");
      
          Tenant createdTenant = tenantService.createTenant(tenant);
          log.info("sending notification ");
          //https://devblogs.microsoft.com/java/embracing-virtual-threads-migration-tips-for-java-developers/
          virtualThreadExecutor.submit(() -> {
              SendNotification(tenant);
          });
          return ResponseEntity.status(HttpStatus.CREATED).body(createdTenant);
      }
      
      private void SendNotification(Tenant tenant) { 
          String dapr_url = "http://localhost:" + DAPR_HTTP_PORT + "/notification";
          // Example assuming tenant.getName() and tenant.getEmailId() return strings
          String tenantName = tenant.getName();
          String tenantEmail = tenant.getEmailId();
      
              // Proper JSON string construction
              String tBody = String.format("{\"Name\":\"%s\",\"Emailid\":\"%s\"}",
              tenantName.replace("\"", "\\\""), // Escape any double quotes within the values
              tenantEmail.replace("\"", "\\\"")); // Escape any double quotes within the values
      
          // System.out.println("notification service url "+ dapr_url  + "," + tBody);
      		HttpRequest request = HttpRequest.newBuilder()
      				.POST(HttpRequest.BodyPublishers.ofString(tBody))
      				.uri(URI.create(dapr_url))
      				.header("Content-Type", "application/json")
      			    .header("dapr-app-id", "notification-service")
      				.build();
      
              try {
                  // System.out.println("Request URL: " + dapr_url);
                  // System.out.println("Request Body: " + tBody);
                  HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
                  log.info("tenant notified: "+ tenant.getId());
              } catch (InterruptedException e) {
                  log.error("Notification service call interrupted"); // Or something more intellegent
              } catch (IOException e) {
                  e.printStackTrace();
                  log.error("Notification service call failed - " + e.getStackTrace().toString()); // Or something more intellegent
              }
      }
      
    • Below snapshot shows Notification service using Dapr .NET SDK to push message,

      
      const string PUBSUB_NAME = "my-pub-sub";
      const string TOPIC_NAME = "tenants";
      
      Environment.SetEnvironmentVariable("DAPR_HTTP_PORT","3500");
      
      var dapr_port = Environment.GetEnvironmentVariable("DAPR_HTTP_PORT");
      
      // initialize httpclient
      var sharedClient = new HttpClient() { 
          BaseAddress = new Uri($"http://localhost:{dapr_port}"),
      }
      
      var builder = WebApplication.CreateBuilder(args);
      builder.Services.AddDaprClient();
      
      var random = new Random();
      
      
      var app = builder.Build();
      
      
      var client = app.Services.GetRequiredService<DaprClient>();
      
      // Configure the HTTP request pipeline.
      if (app.Environment.IsDevelopment())
      {
      app.MapOpenApi();
      }
      
      app.UseHttpsRedirection();
      
      
      app.MapPost("/notification",async (CancellationToken token,  Tenant tenant) =>
      {
      
          app.Logger.LogInformation("Sending notification for " + tenant);
      
          // Publish to topic using Dapr HTTP API
          using HttpResponseMessage resp = await sharedClient.PostAsJsonAsync(
              $"/v1.0/publish/{PUBSUB_NAME}/{TOPIC_NAME}",(tenant: tenant),token)
          );
      
          if (resp.IsSuccessStatusCode) {
                  string responseBody = await resp.Content.ReadAsStringAsync();
                  if (responseBody.Contains("errorCode")) {
                      app.Logger.LogError("Error while pub/sub {responseBody}", responseBody);
                      return string.Empty;
                  }
          } else { 
              app.Logger.LogError("Error while pub/sub with Status code {StatusCode}", app.StatusCode);
              return string.Empty;
      
          }
      
          app.Logger.LogInformation($"Dapr Pubsub response\n Statuscode :{resp.StatusCode}");
          return tenant.ToString();
      
      });
      
  • Demo uses below building blocks are used for the demo,

    • Service invocation
    • Pub/sub
  • Dapr’s Multi-app run is very handy for local development lifecycle and same is used with below configuration ,

    • Dapr Multi app config
  • Resiliency configuration looks like below,

    Resiliency configuration

All is good but what is the trade-off?

Extensibility

While Dapr provides abstraction over many of infrastructure concerns (as shown in earlier diagram), in the event you have requirement to integrate with system not supported by Dapr then do take note of below.

All of the building blocks and infrastructure to abstract implementation (Redis for State mgmt, SQS for messaging etc.) are bundled as single binary. Any need to support new implementation may require forking the repository or via Pluggable components/middleware. Refer here for details. Note that pluggable component runs as distinct process separate from runtime itself.

Performance

All this abstraction isn’t free. The Sidecar pattern introduces an extra “hop” for network traffic

(Service A –> Sidecar A –> Sidecar B –> Service B).

I ran a quick load test using k6 to quantify this overhead on a local development machine.

Disclaimer: This test was performed as-is, on my laptop without any optimization. Hence, YMMV.

Load testing criteria is setup as below,

Load test criteria

The Results:

ModeMeasureLatency
Direct HTTPP95~9.4ms
Via DaprP95~13.4ms

The Verdict: Dapr sidecar, induced some (~4ms) of overhead per request in this scenario.Is it worth it?If you are building a High Frequency Trading platform where microseconds count, Dapr might not be for you. However, for 99% of business applications, trading 4ms of latency for automatic mTLS, distributed tracing, and retry logic is a bargain. You would likely incur more latency implementing those features poorly in your own code.

Debugging with Dapr

It is possible to debug your application by running dapr sidecar separately,

  • Run dapr sidecar as follows (Note: better to use default ports for http and grpc. This is important if you are using dapr sdk which uses these ports and I am unable to find a way to update these from say VS Code in case sidecar is being run separately as below),

    dapr run --app-id notification-service --app-port 8081 --dapr-http-port 3500 --dapr-grpc-port 50001 --resources-path  .\components
    
  • Start debugging the application in your IDE (e.g. VS Code). Make sure that application is running as Dapr Sidecar keeps polling for it before starting relevant services/components.

  • Dapr Sidecar also starts metrics server where its own log is visible, below is snapshot,

    Log generation

Dapr and Gen. AI

Dapr is evolving to meet the needs of Agentic workflows. It recently introduced Dapr Agents for it. In this, all the building blocks are extended to serve agentic Workload aimed at data crunching, inferencing with Large Language Models (LLM) and operate them at scale.

Instead of managing complex python scripts for agent coordination, Dapr Agents allows you to:

  • Control Loop Management: It handles the “Reason <–> Act” loop reliably using the underlying Dapr Actor model.
  • Built-in Guardrails: Leverage Dapr bindings to obfuscate PII (Personally Identifiable Information) before it ever hits the LLM provider.
  • Identity: Secure your agents. Since Dapr handles mTLS, your agents can authenticate via SPIFFE, ensuring that “Agent A” is authorized to talk to “Agent B”.

Summary

Dapr helps with

  • Standardized, vendor-neutral approach, eliminating concerns about lock-in or proprietary restrictions.
  • Organizations gain full flexibility and control over their Distributed/microservices based applications.
  • Dapr works both for greenfield projects (built from scratch) and brownfield applications (existing applications being modernized or migrated).
  • A common scenario is migrating a monolithic or traditional service-based app to Kubernetes or another cloud-native infrastructure: Dapr can be introduced to handle inter-service communication, messaging, state, secrets — without rewriting the whole application.
  • Extending dapr , if required ,is supported via pluggable components which are not part of main binary

While this article is meant to provide key aspects, Dapr has much more to offer. Do not forget to go through the documentation here.

Useful References,

Happy Coding !!