Module 6: Discovery, Load Balancing, and Cloud-Native Runtime Patterns

Learning Path

Monday, October 5, 2026, 6:00–8:30 PM. The 20-slide session opens with a short Kubernetes orientation, then follows one marketplace request through discovery, traffic selection, health changes, and orchestration. The final four slides extend these ideas to inference routing, agent capabilities, and recoverable execution. The slide pacing totals 135 minutes; reserve 10 minutes for a break after slide 10 and 5 minutes for setup and transitions.

The marketplace and travel workflow are illustrative designs. Technology references were checked on October 5, 2026. Documentation under latest or stable changes: verify your installed API, controller, and runtime versions before adopting an optional feature. This lesson emphasizes mechanisms and tradeoffs rather than claiming one product is universally best.

Required Preparation

Review Module 3’s deadlines and service identity, Module 4’s failure domains, and Module 5’s repeat-safe processing. Read the Required entries in Additional Resources: Services, probes, Gateway API, gRPC load balancing, and why a service mesh. Focus on the concepts; memorizing Kubernetes commands is not required.

Bring a project architecture diagram and a list of dependencies. For the optional live exercise, use a disposable local Kubernetes cluster with a working network plugin, cluster DNS, kubectl, and permission to create a namespace. An existing Docker Desktop Kubernetes, kind, or minikube environment is sufficient if working; provisioning a cluster is outside this lesson. The paper alternative needs no software. No paid cloud service, mesh installation, or AI account is required.

Mini Kubernetes Orientation

Five-minute walkthrough; no prior Kubernetes experience required. Kubernetes (often abbreviated K8s) coordinates containerized applications across a cluster. It helps keep the declared number of instances running and provides primitives for connecting them. It does not automatically make application state durable or business operations safe to retry.

Terminology: a quick reference

Read the basics for the walkthrough, then return to the networking and scaling terms as they appear in the lesson. Kubernetes is the platform; minikube is one tool for running a local Kubernetes cluster.

Kubernetes basics

Discovery, traffic, and mesh

Health, scaling, and operations

Scaling example: CPU pressure in a synchronous Inventory API may drive HPA. Pending messages in a worker queue may drive KEDA and HPA together. In either case, more replicas help only if the database, downstream tool, or provider quota has spare capacity. See the linked HPA documentation and KEDA scaling concepts for the mechanisms and prerequisites.

flowchart LR
 A["You + kubectl"] -->|"submit YAML"| C["Kubernetes API<br/>desired state"]
 C --> B["Controllers + scheduler"]
 B --> N["Node: kubelet + runtime"]
 N --> P["Pod<br/>application container"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 class C control

Read, apply, inspect, connect

Follow the instructor’s screen or use an existing disposable local cluster. Cluster installation is optional preparation, not part of this short walkthrough. Download the mini tutorial and lab manifest (YAML) as runtime-lab.yaml. The same resources will be reused in the later failure experiment.

1. Read: find kind: Deployment, replicas: 2, the container image, and kind: Service in the YAML. The Service selector matches labels on the inventory Pods. The manifest also creates the namespace, a ConfigMap containing demo code, and a client Pod.
2. Apply: check that your context is the disposable cluster, then submit the desired state.

kubectl config current-context
kubectl apply -f runtime-lab.yaml
kubectl -n cmpe273-module06 rollout status deployment/inventory --timeout=120s
kubectl -n cmpe273-module06 wait --for=condition=Ready pod/client --timeout=120s

3. Inspect: list what Kubernetes created.

kubectl -n cmpe273-module06 get deployments,pods,services

Expect the inventory Deployment to become 2/2 ready, two inventory Pods with generated names, one client Pod, and the inventory Service with a ClusterIP and port 8080. Values and startup time vary. A Service is a network abstraction, so it is not another application Pod. If readiness does not arrive, use kubectl -n cmpe273-module06 describe pod <pod-name> to inspect events and kubectl -n cmpe273-module06 logs <pod-name> for application output, replacing the placeholder with an actual name.

4. Connect: run the provided client inside the cluster.

kubectl -n cmpe273-module06 exec client -- python /lab/client.py

The client sends requests to inventory:8080 and prints counts by responding Pod name. Predict: the name remains stable, but more than one Pod can answer. Exact alternation is not guaranteed. We will next explain how DNS, the Service, and connection selection make this happen. Keep these resources for the later lab; cleanup is documented there. For the paper alternative, draw the two Pods behind one Service and replace a Pod without changing the Service name.

Request Routing and Discovery

A buyer sends a request to create an order. The external gateway selects the Ordering route, and an eligible Ordering instance handles the request. That instance may discover Inventory separately. At every hop, distinguish four questions: where can I connect, which target should receive this work, who is calling, and is that caller allowed to perform this operation? A successful DNS lookup answers only the first part.

flowchart LR
 U["Buyer"] --> G["Gateway<br/>route /orders"]
 G --> S["Ordering service<br/>stable name"]
 S --> A["Replica A<br/>ready"]
 S --> B["Replica B<br/>ready"]
 D["Discovery updates"] -.-> S
 A --> DB[("Shared order state")]
 B --> DB
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class D,G control
 class DB data

DNS returns names and addresses; a registry can supply richer endpoint membership. Clients may cache results or hold connections long after a lookup. In client-side balancing, a library resolves targets and chooses among them. In proxy-side balancing, a client addresses the proxy, which handles backend membership and selection. Either design needs timeouts, refresh behavior, and an owner for unavailable endpoints. A registry or proxy is also a dependency that can fail.

Naming and registration: connect to last semester’s example

A logical name such as payment-service identifies a role; 10.1.4.23:8080 identifies one current endpoint. A registry stores the mapping. The recurring lifecycle is register → discover → call, followed by membership updates as instances start, fail, or leave. Consul and Netflix Eureka illustrate registry-based discovery; Kubernetes controllers maintain Service endpoint membership from selected Pods rather than requiring every application to register itself.

With client-side discovery, the consumer looks up endpoints and chooses one. With server-side discovery, an intermediary locates and selects the backend. The former gives the client control but requires suitable client logic; the latter centralizes decisions but introduces another component to operate. Discovery supplies candidates, and balancing chooses among them. Modern load balancers can be highly available and dynamically updated, so these mechanisms complement each other.

A Service is a stable abstraction

For a normal Kubernetes ClusterIP Service, cluster DNS resolves a name such as inventory.module06.svc.cluster.local to a virtual IP. The actual cluster domain can differ. Service selectors identify matching Pods; EndpointSlices carry endpoint addresses and readiness-related conditions. The data plane uses that information to forward traffic. DNS is not consulted for every HTTP request.

A headless Service has no ClusterIP and can expose individual endpoint addresses through DNS. That can help a client capable of endpoint-aware selection. It does not automatically add balancing, retry safety, or authorization to an arbitrary client.

flowchart LR
 C["Client"] -->|"DNS lookup"| D["Cluster DNS"]
 D -. "Service IP" .-> C
 C -->|"new connection"| V["Service virtual IP<br/>data plane"]
 V --> P["Ready Pod"]
 E["EndpointSlice<br/>addresses + conditions"] -. "updates" .-> V
 K["Pod labels and readiness"] -.-> E
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class D,E,K control

Failure to reason about: a Pod becomes unhealthy after a client resolved its address. DNS caching, health detection, membership updates, and connection state have different lifetimes. “The registry is correct now” does not prove every caller has stopped using the old instance.

Load Balancing and Connection Reuse

L4 means Layer 4, the transport layer; L7 means Layer 7, the application layer in the OSI model. They describe what a networking component understands and the unit at which it makes a forwarding decision. Our example uses TCP and HTTP/2; not every transport protocol is connection-oriented.

First distinguish a connection from a request

A TCP connection is a reusable communication channel between two endpoints. It carries an ordered stream of bytes. A request is an application operation carried over that channel, such as GET /products, POST /orders, or a gRPC ReserveStock call. TCP itself does not interpret those operations.

One connection does not mean one request. HTTP/1.1 can reuse a connection for successive requests. HTTP/2 can multiplex several concurrent request/response streams over one connection. Think of a connection as a phone call and requests as the individual questions asked during that call; several questions do not require several calls.

flowchart LR
 C["Client<br/>requests 1, 2, 3"] -->|"one TCP connection"| L["L4 balancer"]
 L --> A["Backend A<br/>all three requests"]
 D["Client<br/>requests 1, 2, 3"] -->|"one HTTP/2 connection"| P["L7 proxy"]
 P --> B["Backend A: 1, 3<br/>Backend B: 2"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 class L,P control

Follow three requests through both designs

L4 example: a client opens one TCP connection to the load balancer. The balancer selects A. Requests 1, 2, and 3 then use that existing connection, so all reach A. Another new connection might select B. Splitting arbitrary packets from the same connection between A and B would break the connection’s transport state.

L7 example: a client opens one HTTP/2 connection to a proxy. The proxy recognizes three separate requests and may forward 1 to A, 2 to B, and 3 to A using its own backend connections. It tracks the responses and returns each through the correct client stream. This is request forwarding through a proxy, not moving half a TCP connection to another server. The specific distribution depends on its configured policy. See Envoy’s HTTP connection management.

Why this matters for gRPC and agents

Suppose one client sends 100 unary gRPC calls over an existing HTTP/2 connection assigned to A by an L4 balancer. Adding B and C does not redistribute that connection. All three Pods can be healthy while A remains busy and the others receive no work from this client.

The gRPC load-balancing guide describes pick_first and round_robin. Client balancing needs discovery to expose useful backend addresses: rotating over a single ClusterIP does not reveal the Pods behind it. A suitable L7 proxy is another option. Neither approach guarantees perfectly even work, especially when request durations differ.

A streaming model response or long-running gRPC stream remains assigned to its selected backend for that stream’s lifetime. Each token or stream message is not independently load-balanced. Recovering an interrupted stream needs an application policy for cancellation, reconnecting, or resuming.

Check your understanding: if a dashboard shows ten TCP connections evenly spread across two replicas, does that prove equal request load? No. One connection may carry thousands of requests while another carries only a few. Observe requests, active streams, latency, and backend work as well as connection counts.

Choose a policy after choosing the balancing unit

Worked reasoning: A has one 30-second generation in flight; B has three 10-millisecond lookups. An active-request count alone would make A appear less busy. The right signal depends on the work. For remote model APIs you do not control, provider quotas and rate limits may matter more than replica metrics.

Health and Failure Containment

Use different endpoints for different questions. A startup probe protects slow initialization. Readiness represents whether an instance should receive normal Service traffic. Liveness detects a condition for which restarting the local container is a reasonable remedy. A dependency outage that affects all replicas is usually not repaired by restarting all of them.

flowchart LR
 S["Startup passes"] --> R["Readiness fails"]
 S --> L["Liveness fails<br/>past threshold"]
 R --> N["Remove from<br/>normal routing"]
 L --> X["Restart<br/>container"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class N,X warning

Probe thresholds and propagation introduce delay. Design for requests already in flight and connections already open. A readiness check might reasonably exclude a server that cannot serve its core operation, but do not automatically fail every capability because an optional recommendation provider is unavailable.

A retry amplification example

Suppose the gateway, Ordering, and its client library each permit three attempts for the same call, including the original. In a worst case, one user request can produce 3 × 3 × 3 = 27 backend attempts. This is an illustrative upper-bound calculation, not a measured workload. Define retry ownership and an end-to-end deadline, and cap additional attempts. Use jitter to avoid synchronized retry bursts.

A concurrency limit protects active work; a bounded queue limits waiting work. A circuit breaker can stop repeated attempts toward a failing dependency. A bulkhead keeps one dependency from taking all shared capacity. These mechanisms do not determine whether an uncertain payment succeeded. Preserve the same operation identity and reconcile with the authoritative provider.

Runtime Control Loops

A Deployment declares a desired replica count. Controllers maintain the necessary resources, the scheduler selects a node for a pending Pod, and a kubelet starts its containers. A failed Pod can be replaced at a different address. The replacement needs time to start and become ready; it cannot restore an order that existed only in the previous process’s memory.

flowchart LR
 C["Controller<br/>desired 3, sees 2"] --> N["Create Pod"]
 N --> S["Scheduler<br/>selects node"]
 S --> K["Kubelet<br/>starts container"]
 K --> R["Ready<br/>join endpoints"]
 R -. "observed state" .-> C
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class C,S control

Place durable business state outside disposable workers. Use resource requests to express scheduling needs and limits where appropriate to constrain consumption. Confirm that replicas are actually spread across the failure domains you intend to survive: three containers on one host still share a host failure. Managed cloud runtimes can provide these same concepts without exposing every Kubernetes object; choose the platform complexity your team can operate.

Configuration, identity, and secrets

Build one image and supply environment-specific configuration at runtime. Ordinary values may live in a ConfigMap or platform configuration service. Credentials need restricted delivery, rotation, and access controls. Kubernetes Secret encoding is not encryption; review storage protection and permissions. Never put real credentials in this course repository or downloadable examples.

SPIFFE provides a model for verifiable workload identity. Authentication establishes the caller; authorization permits specific actions. An Inventory endpoint should enforce both authorization and stock rules regardless of how the caller found it. The classroom lab deliberately demonstrates routing only and makes no production authentication claim.

Rolling updates and termination

Readiness should keep a new instance from receiving normal traffic until it is usable. Deployment rolling-update settings control how many old instances may be unavailable and how much temporary spare capacity may be created. A canary is a separate exposure policy: direct a limited share of traffic to a new version and observe its outcomes.

flowchart LR
 A["v1 serves"] --> B["v2 starts"]
 B --> C["v2 ready"]
 C --> D["Terminate v1"]
 D --> E["Routing updates<br/>and draining"]
 E --> F["v1 exits"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class D,E warning

Termination and routing updates overlap. The application must handle termination signals, stop admitting new work, finish or checkpoint existing work, and exit within its allowed grace period. Long-lived streams may need a reconnect or resumability design. Do not treat readiness removal as instantaneous revocation of all open connections. A graceful-shutdown experiment should use an application that implements that contract; the minimal lab below tests readiness and replacement, not graceful draining.

Scaling is delayed feedback

The simplified HPA calculation ceil(current replicas × observed metric / target metric) gives four replicas for three instances at 80% average CPU utilization against a 60% target. Real behavior also includes missing metrics, tolerance, readiness treatment, and stabilization. CPU utilization targets depend on configured CPU requests and an available metrics API.

flowchart LR
 W["Work arrives"] --> P["Ready replicas"]
 P --> M["Load metrics"]
 M --> H["Autoscaler<br/>desired count"]
 H --> D["Scheduling<br/>startup delay"]
 D --> P
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class M,H control

For asynchronous workers, consider backlog and the age of unfinished work. KEDA integrates external event metrics with scaling, including activation from zero where supported. Additional workers will not fix an exhausted database pool, a provider quota, or a Kafka partition count that limits useful parallelism. Scaling down must respect in-flight work and repeat handling.

Why Service Mesh and Why Istio?

Suppose Ordering, Inventory, and Payments are maintained by different teams in Java, Python, and Go. Each team needs secure connections, controlled service access, timeouts, and useful network metrics. Implementing these controls independently creates inconsistent behavior and repeated operational work. A service mesh provides shared infrastructure for managing communication among enrolled services, usually through proxies controlled by a common configuration plane.

A Kubernetes Service answers “how do I reach these Pods?” A gateway commonly manages traffic entering the system (north–south). A mesh focuses on communication within the system (east–west). These are useful responsibility boundaries, not absolute product boundaries: one proxy technology can serve both gateway and mesh roles. DNS, a Service, or a gateway alone does not automatically give every internal call mutual authentication, HTTP-aware authorization, and consistent telemetry.

flowchart LR
 A["Ordering app"] --> P["Proxy"] --> Q["Proxy"] --> B["Inventory app"]
 C["Istio control plane"] -. "identity + policy" .-> P
 C -. "identity + policy" .-> Q
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 class C control

Istio is one implementation of a service mesh. Its control plane configures traffic behavior and supports workload identity and certificates; its data-plane proxies process application traffic. For example, configure authenticated Ordering workloads to call Inventory, reject unauthorized callers, and measure errors and tail latency along that path. With the appropriate L7 proxy and policy, route a portion of requests to a new Inventory version. These capabilities require enrollment and configuration; installation alone does not prove coverage.

Sidecar versus ambient: where the work happens

In sidecar mode, an Envoy proxy runs alongside the application in each enrolled Pod. It can inspect supported application protocols and enforce L4 and L7 policy. The extra container consumes resources, and its rollout lifecycle is linked to the application Pod.

In ambient mode, a per-node ztunnel provides the secure L4 transport layer. Waypoint proxies add L7 processing for enrolled traffic that needs it, such as HTTP authorization and routing. Removing per-Pod sidecars does not remove proxies. An L4-only path cannot apply policies requiring HTTP request contents. The next diagram compares the placement choices; consult the linked version-specific documentation for feature support and enrollment details.

When the benefit justifies the cost

A mesh is useful when many services need a common, enforceable identity and traffic policy and the organization can operate that infrastructure. Costs include proxy resources, certificate and policy troubleshooting, upgrades, and new failure paths. A small application with adequate TLS, libraries, and network controls may not need one. Our mini tutorial and lab require no mesh installation.

For an agent runtime, a mesh can authenticate an executor when it calls a tool service. It cannot decide whether a proposed booking matches the user’s approved itinerary, make an external effect exactly once, or make an untrusted tool response safe. Those remain workflow and application responsibilities.

Observe both layers: record per-Pod request counts, active connections, errors, p95/p99 latency, readiness changes, restarts, and desired versus ready replicas. All Pods can be ready while connection reuse concentrates work on one Pod. A fast response can still be a failed business operation.

Contemporary Runtime Patterns

The choices below address different responsibilities. They are optional extensions, not prerequisites for the lab.

flowchart LR
 R["HTTPRoute<br/>matches + weights"] -. "configures" .-> G["Gateway<br/>selected GatewayClass"]
 U["Client"] --> G
 G -->|"90 weight"| A["Stable Service"]
 G -->|"10 weight"| B["Canary Service"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 class R,G control

GatewayClass selects a controller implementation; Gateway represents listeners; routes attach application rules. API resources need a working implementation. A successfully submitted HTTPRoute is not proof of a functioning route: examine its status and exercise the actual request path.

flowchart LR
 A["Sidecar mode<br/>application Pod"] --> B["Local Envoy<br/>L4 + L7"] --> D["Destination proxy + app"]
 E["Ambient mode<br/>application Pod"] --> C["Node ztunnel<br/>L4 secure overlay"] --> W["Waypoint when needed<br/>L7 policy"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 class C control

Cilium’s eBPF forwarding and Istio’s ambient proxies should not be collapsed into one “sidecar-free” promise. Service forwarding, transport identity, and HTTP authorization are separate responsibilities. Evaluate the configuration you will deploy and compare equivalent guarantees before drawing performance conclusions.

Runtime Lab

Goal: send requests through a stable Service name, observe multiple backends, and distinguish an unready Pod from a deleted Pod. Use a disposable local cluster. This example uses synthetic responses only and exposes no external load balancer.

Download the Kubernetes runtime lab (YAML: namespace, demo server, two replicas, Service, and client) and the lab instructions (Markdown: commands, observations, troubleshooting, and cleanup).

The manifest uses python:3.13-slim for a standard-library HTTP server and an in-cluster client. Image retrieval requires registry access; a version-family tag is convenient for teaching but not an immutable production pin. Record the resolved image ID when comparing results. The server returns its Pod name, exposes /live and /ready, and supports readiness failure through a local marker file. It does not implement a real Inventory database, an authentication system, or graceful shutdown.

1. Establish a baseline

Save the download as runtime-lab.yaml, inspect your current Kubernetes context, and apply it only to your disposable cluster:

kubectl config current-context
kubectl apply -f runtime-lab.yaml
kubectl -n cmpe273-module06 rollout status deployment/inventory --timeout=120s
kubectl -n cmpe273-module06 wait --for=condition=Ready pod/client --timeout=120s
kubectl -n cmpe273-module06 get pods -o wide
kubectl -n cmpe273-module06 get endpointslices -l kubernetes.io/service-name=inventory
kubectl -n cmpe273-module06 exec client -- python /lab/client.py

The client makes fresh HTTP connections through inventory:8080 and prints per-Pod counts plus errors. You should normally see both replica names over repeated runs; do not expect exact alternation or a guaranteed 50/50 split. Do not use kubectl port-forward service/inventory to test balancing: port forwarding selects a backing Pod rather than exercising the ordinary Service path.

2. Fail readiness without restarting

LAB_POD=$(kubectl -n cmpe273-module06 get pod -l app=inventory -o jsonpath='{.items[0].metadata.name}')
kubectl -n cmpe273-module06 exec "$LAB_POD" -- touch /tmp/not-ready
kubectl -n cmpe273-module06 wait --for=condition=Ready=false "pod/$LAB_POD" --timeout=60s
kubectl -n cmpe273-module06 get endpointslices -l kubernetes.io/service-name=inventory -o yaml
kubectl -n cmpe273-module06 exec client -- python /lab/client.py
kubectl -n cmpe273-module06 get pods

Compare the endpoint conditions and returned names. After readiness detection and routing propagation, new connections should use the other ready Pod. The unready Pod still exists; its liveness endpoint passes, so the experiment should not increase its restart count. Record timestamps instead of assuming the transition is instantaneous.

3. Restore, then replace

kubectl -n cmpe273-module06 exec "$LAB_POD" -- rm -f /tmp/not-ready
kubectl -n cmpe273-module06 wait --for=condition=Ready "pod/$LAB_POD" --timeout=60s
kubectl -n cmpe273-module06 exec client -- python /lab/client.py
kubectl -n cmpe273-module06 delete pod "$LAB_POD"
kubectl -n cmpe273-module06 rollout status deployment/inventory --timeout=120s
kubectl -n cmpe273-module06 get pods -o wide
kubectl -n cmpe273-module06 exec client -- python /lab/client.py

Deletion is the deliberate fault injection. The Deployment creates a replacement with a new identity. The Service name remains the same. To capture transient errors and recovery time, keep running the client in a second terminal while you delete the Pod. A completed rollout does not measure the interruption window by itself.

Record evidence

For cleanup, verify that the namespace contains only this exercise, then run kubectl delete namespace cmpe273-module06. This deletes all resources in that lab namespace.

Troubleshooting

• ImagePullBackOff: inspect kubectl -n cmpe273-module06 describe pod POD_NAME and confirm registry access.
• No ready endpoints: check Pod readiness and the Service selector with kubectl -n cmpe273-module06 describe service inventory.
• DNS or connection errors: inspect cluster DNS and network policy; run the client inside the lab namespace, where the short name inventory resolves.
• Only one backend appears: first confirm both Pods are ready. Repeat the sample and inspect connection reuse before assuming a broken balancer; a short sample is not a fairness guarantee.

Paper alternative

Sketch two replicas behind one Service. At time 0 both are ready; at time 1 B fails readiness; at time 2 membership updates reach the data plane; at time 3 an old connection to B closes. Mark where a fresh connection and an existing connection may go at each time. Then replace B and show which name remains stable. Use this to distinguish health detection, propagation, and connection lifetime without guessing an exact outage duration.

Agentic Runtime: Locate, Permit, Execute, Recover

A planner may choose a model and discover a booking tool, but those choices still run on ordinary distributed infrastructure. Model serving adds long-lived streams and heterogeneous request cost; endpoint selection should first enforce model compatibility and available capacity. Gateway API Inference Extension is a contemporary project addressing that space, with implementation-specific prerequisites.

flowchart LR
 A["Agent request<br/>model + token budget"] --> G["Compatible inference gateway"]
 G --> P["Endpoint picker<br/>eligible model replicas"]
 M["Serving metrics"] -.-> P
 P --> R1["Model replica A"]
 P --> R2["Model replica B"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class G,P,M control

MCP tool discovery describes callable interfaces and their schemas. A2A discovery exposes Agent Cards for agent interfaces and capabilities. Neither mechanism grants authority to call every advertised capability. Use approved server catalogs, verified identity, operation-level permissions, and controlled egress. Treat retrieved descriptions and tool results as data, not as policy changes.

flowchart LR
 A["Agent runtime"] --> L["Locate server"]
 L --> C["Inspect tools<br/>or Agent Card"]
 C --> P["Verify identity<br/>check permission"]
 P --> T["Invoke tool"]
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class C,P control

Extend the Module 5 travel workflow: a disruption starts planning; a versioned proposal is approved; an executor revalidates constraints and calls a mock booking provider. Store progress durably so any eligible executor can resume. Coordinate concurrent execution and preserve an operation key across retries. If the provider succeeded but its reply vanished, lookup or reconciliation resolves the outcome.

flowchart LR
 P["Disruption<br/>planner reacts"] --> V["Store proposal<br/>exact-version approval"]
 V --> X["Executor<br/>check constraints"]
 X --> T["Booking tool<br/>operation key"]
 T --> R["Record result<br/>or reconcile"]
 R -. "durable progress" .-> V
 classDef control fill:#fff3da,stroke:#9b6b16,color:#35260d
 classDef warning fill:#ffebe7,stroke:#bd5949,color:#542b24
 classDef data fill:#e8eefc,stroke:#556bb4,color:#24345e
 class V,X control
 class R data

Invariant: at most one booking per approved proposal version, and none without approval of that exact version. A healthy executor, a successful model response, and a committed event offset are each useful observations; none alone proves that invariant. Measure tool authorization failures, unknown outcomes, duplicate suppression, and successful workflow completion alongside infrastructure latency.

Architecture Review

Prepare one annotated diagram and a short risk list. Name your discovery mechanism, balancing unit, readiness rule, failure domains, scaling signal, and source of durable business state. Show how configuration and workload credentials reach the service. Include one failure experiment with a prediction, observation, and explanation of any mismatch.

For an optional agent extension, route a synthetic tool call through two executors and interrupt one after mock provider success. Explain why a new executor can recover without a second effect. No real booking, payment, or external account is required. Canvas remains the official source for assessed requirements, deadlines, submissions, feedback, and grades.