AGENTS.md

1# CHTTP2 and PH2 Transports
2 
3*   This directory contains the implementations of two HTTP/2 transports:
4    *   The legacy CHTTP2 transport.
5    *   The newer WIP promise-based HTTP/2 transport (PH2).
6 
7See also: [gRPC Transports overview](../../../transport/AGENTS.md)
8 
9## Overarching Purpose
10 
11This directory houses the C++ implementations of gRPC's HTTP/2 transports.
12These transports manage the low-level details of the HTTP/2 protocol,
13serving as the bridge between gRPC's high-level abstractions (channels, calls)
14and the underlying endpoint.
15 
16**Core Responsibilities:**
17 
18*   **HTTP/2 Framing:** Parsing and serializing HTTP/2 frames (HEADERS, DATA, SETTINGS, PING, GOAWAY, WINDOW_UPDATE, RST_STREAM, CONTINUATION).
19*   **Stream Multiplexing:** Managing multiple concurrent gRPC calls over a single TCP connection using HTTP/2 streams.
20*   **Flow Control:** Implementing HTTP/2 flow control mechanisms (both stream and connection level) to prevent overload.
21*   **Header Compression:** Utilizing HPACK to compress and decompress header fields, reducing bandwidth usage.
22*   **Error Handling:** Detecting and reporting HTTP/2 protocol errors and stream errors.
23 
24**Implementented RFCs:**
25 
26*   [RFC 9113: HTTP/2](https://www.rfc-editor.org/rfc/rfc9113.html)
27*   [RFC 7541: HPACK - Header Compression for HTTP/2](https://www.rfc-editor.org/rfc/rfc7541.html)
28 
29## Subdirectories
30 
31*   **`alpn`**: Contains code for ALPN (Application-Layer Protocol Negotiation), which is used to select the HTTP/2 protocol during the TLS handshake.
32*   **`client`**: Contains the client-side implementation of the CHTTP2 transport.
33*   **`server`**: Contains the server-side implementation of the CHTTP2 transport.
34*   **`transport`**: Contains the core implementation of the CHTTP2 and PH2 transport.
35*   Code in directories [`alpn`](./alpn), [`client`](./client) and [`server`](./server) is shared by PH2 and CHTTP2.
36 
37## 1. CHTTP2 (Legacy)
38 
39*   CHTTP2 is compatible with the Call V1 Stack.
40*   Uses [`combiner`](../../../lib/iomgr/combiner.h) for concurrency.
41*   CHTTP2 was the original default transport.
42*   **Status:** Active and default, but planned for deprecation and removal after PH2 is fully rolled out and stable.
43 
44### CHTTP2 File Structure
45 
46<!--
47TODO(tjagtap) [PH2][CHTTP2] Edit this doc when CHTTP2 is getting deleted.
48-->
49 
50*   General Transport Files:
51    *   `chttp2_transport.{h,cc}`: Core transport logic for both client and server.
52    *   `internal.h`: Internal declarations for CHTTP2.
53    *   `parsing.cc`: HTTP/2 incoming frame parsing and processing.
54    *   `stream_lists.{h,cc}`: Manages stream lists.
55    *   `writing.cc`: Handles writing data to the wire.
56*   Frame Specific Files: `frame_*.h`, `frame_*.cc`
57    *   `frame_data.{h,cc}`
58    *   `frame_goaway.{h,cc}`
59    *   `frame_ping.{h,cc}`
60    *   `frame_rst_stream.{h,cc}`
61    *   `frame_security.{h,cc}`
62    *   `frame_settings.{h,cc}`
63    *   `frame_window_update.{h,cc}`
64    *   `legacy_frame.h`
65 
66## 2. PH2 (Promise-Based HTTP/2)
67 
68*   PH2 is compatible with the Call V3 Stack.
69*   PH2 utilizes the gRPC promise framework (`src/core/lib/promise`) for asynchronous operations.
70*   **Status:** Under Development.
71*   **Rollout:** Expected to begin in November 2026.
72*   **Experiments:**
73    *   Enable Client: `IsPh2ClientEnabled()`.
74    *   Enable Server: `IsPh2ServerEnabled()`.
75    *   Enable both PH2 Client and PH2 Server : `IsPh2ClientServerEnabled()`.
76        *   This is for testing only. This experiment will NEVER be rolled out.
77 
78### PH2 Goals
79 
80*   **Compatibility:** Compatibility with the new efficient Call V3 stack.
81*   **Modernization:** Leverage the gRPC Promise API for cleaner and more maintainable asynchronous code.
82*   **Performance:** Aim for equal or better performance compared to CHTTP2.
83*   **Compliance:** Adhere to HTTP/2 RFC [RFC 9113](https://www.rfc-editor.org/rfc/rfc9113.html).
84*   **Feature Parity:** Support all necessary features currently provided by CHTTP2.
85 
86### PH2 Development Guidelines
87 
88*   **Reference CHTTP2:** When implementing features in PH2, always first check the CHTTP2 implementation in this directory for reference. Use the mapping in the "Key classes in CHTTP2 and their PH2 equivalents" section below to find the PH2 counterparts.
89*   **Asynchronous Operations:** All asynchronous operations *must* use the gRPC Promise library (`src/core/lib/promise`), particularly leveraging `Party` for concurrency. See the `Dependencies for PH2` section.
90*   **Testing:** Any changes to PH2 code should be accompanied by relevant tests. Ensure that existing tests in `test/core/transport/chttp2/http2_client_transport_test.cc` and `test/core/transport/chttp2/http2_server_transport_test.cc` pass. Add new tests as needed to cover new functionality.
91*   **Test Comments:** When writing new tests, include comments within each test explaining its purpose and assertions. For large tests, add comments for each step.
92 
93### Key Differences from CHTTP2
94 
95*   **Concurrency Model:** Uses `Party` promises instead of `combiner`.
96*   **Frame Handling:** New frame parsing and serialization logic in `frame.{h,cc}`.
97*   **Stream Management:** Different data structures and mechanisms.
98 
99### PH2 File Structure
100 
101*   PH2 Client Code:
102    *   `http2_client_transport.{h,cc}`
103*   PH2 Server Code:
104    *   `http2_server_transport.{h,cc}`
105*   Code common to PH2 Client and PH2 Server:
106    *   `http2_transport.{h,cc}`: Common logic.
107*   Frame Parsers and Validators for PH2:
108    *   `frame.{h,cc}`: Newer frame parsing/serialization.
109*   Assemblers:
110    *   `header_assembler.h`: Converts gRPC Initial and Trailing Metadata into HTTP2 HEADER and CONTINUATION Frames and back.
111    *   `message_assembler.h`: Converts gRPC Messages into HTTP2 DATA Frames and back.
112*   Error Handling Classes:
113    *   `http2_status.h`: Custom HTTP/2 error types (Stream vs Connection).
114*   Ping and Keep Alive Helpers:
115    *   `ping_promise.{h,cc}`
116    *   `keepalive.{h,cc}`
117*   Helper classes for PH2 writes:
118    *   `stream_data_queue.h` Stores gRPC messages and Metadata from the CallV3 stack for each stream in a queue.
119    *   `writable_streams.h` Track streams that have some data to send to the peer and have available flow control tokens.
120    *   `write_cycle.{h,cc}` tracks frames and quota for each write cycle.
121*   Settings Helper : `http2_settings_promises.h`
122*   Flow Control Helper : `flow_control_manager.h`
123*   Stream : `stream.h` representation of each HTTP2 stream in the HTTP2 transport.
124*   GoAway : `goaway.{h,cc}` for implementation of HTTP2 GOAWAY
125*   Metadata: `read_context.h`
126*   Security Frame : `security_frame.h`
127 
128## 3. Common Files (Shared by CHTTP2 and PH2)
129 
130*   **`alpn`**: Contains code for ALPN (Application-Layer Protocol Negotiation), which is used to select the HTTP/2 protocol during the TLS handshake.
131*   **`client/chttp2_connector.h`, `client/chttp2_connector.cc`**:
132    *   These files define the client-side connector, which is responsible for creating a new HTTP2 transport.
133    *   This connector creates either PH2 and CHTTP2 transport.
134*   **`server/chttp2_server.h`, `server/chttp2_server.cc`**: These files define the server-side listener, which is responsible for accepting new connections and creating new HTTP2 transports.
135*   Files in `transport/`:
136    *   HPACK implementation:
137        *   `hpack_constants.h`
138        *   `hpack_encoder.{h,cc}`
139        *   `hpack_encoder_table.{h,cc}`
140        *   `hpack_parse_result.{h,cc}`
141        *   `hpack_parser.{h,cc}`
142        *   `hpack_parser_table.{h,cc}`
143    *   Flow Control: `flow_control.{h,cc}`
144    *   Settings: `http2_settings.{h,cc}`, `http2_settings_manager.{h,cc}`
145    *   Ping policies: `ping_abuse_policy.{h,cc}`, `ping_callbacks.{h,cc}`, `ping_rate_policy.{h,cc}`
146    *   Other utilities: `bin_encoder.{h,cc}`, `decode_huff.{h,cc}`, `huffsyms.{h,cc}`, `varint.{h,cc}`
147    *   `transport_common.{h,cc}`
148    *   `internal_channel_arg_names.h`
149    *   `http2_ztrace_collector.h`: Collects events for z-trace debugging.
150    *   `write_size_policy.{h,cc}`
151 
152## 4. Unused or TBD Files
153 
154*   Not used by either transport: `bin_decoder.{h,cc}`
155*   TBD (ETA 2025-10-30):
156    *   `call_tracer_wrapper.{h,cc}`
157    *   `http2_stats_collector.{h,cc}`
158    *   `http2_stats_collector.github.cc`
159 
160## Key classes in CHTTP2 and their PH2 equivalents
161 
162This section maps the core functionalities and classes from the legacy CHTTP2
163transport to their counterparts in the newer Promise-based PH2 transport.
164 
165*   **Transport Object**:
166    *   CHTTP2: `grpc_chttp2_transport` struct defined in `chttp2_transport.cc`.
167    *   PH2: `Http2ClientTransport` class in `http2_client_transport.h` for clients, and `Http2ServerTransport` class in `http2_server_transport.h` for servers.
168 
169*   **Stream Object**:
170    *   CHTTP2: `grpc_chttp2_stream` struct defined in `chttp2_transport.cc`.
171    *   PH2: `Http2ClientTransport::Stream` inner class for clients, and `Http2ServerTransport::Stream` inner class for servers.
172 
173*   **Transport Creation**:
174    *   CHTTP2: `grpc_create_chttp2_transport()` in `chttp2_transport.cc`.
175    *   PH2: Constructors of `Http2ClientTransport` and `Http2ServerTransport`.
176 
177*   **Stream Initiation/Handling**:
178    *   CHTTP2: Functions like `init_stream`, `chttp2_perform_stream_op_locked`, etc., in `chttp2_transport.cc`.
179    *   PH2: Handled within `Http2ClientTransport::StartCall` for clients, and
180        `Http2ServerTransport::IncomingStream` for servers.
181 
182*   **Error Handling**:
183    *   CHTTP2: Error handling with `grpc_error_handle` throughout the code.
184    *   PH2: `http2_status.h` for custom error types, with `HandleError` methods in the transport classes to process stream/connection errors.
185 
186## Test Files
187 
188Key test files include:
189 
190*   **PH2 Specific Tests:**
191    *   `test/core/transport/chttp2/http2_client_transport_test.cc`: Main test suite for the PH2 client transport.
192    *   `test/core/transport/chttp2/http2_server_transport_test.cc`: Main test suite for the PH2 server transport.
193    *   `test/core/transport/chttp2/http2_transport_test.cc`: Common to PH2 Client and Server Transport. Tests code in `http2_transport.{h,cc}`.
194    *   `test/core/transport/chttp2/frame_test.cc`
195    *   `test/core/transport/chttp2/goaway_test.cc`
196    *   `test/core/transport/chttp2/header_assembler_test.cc`
197    *   `test/core/transport/chttp2/http2_status_test.cc`
198    *   `test/core/transport/chttp2/keepalive_test.cc`
199    *   `test/core/transport/chttp2/message_assembler_fuzz_test.cc`
200    *   `test/core/transport/chttp2/message_assembler_test.cc`
201    *   `test/core/transport/chttp2/ping_promise_test.cc`
202    *   `test/core/transport/chttp2/settings_timeout_manager_test.cc`
203    *   `test/core/transport/chttp2/settings_timeout_test.cc`
204    *   `test/core/transport/chttp2/stream_data_queue_fuzz_test.cc`
205    *   `test/core/transport/chttp2/stream_data_queue_test.cc`
206    *   `test/core/transport/chttp2/writable_streams_fuzz_test.cc`
207    *   `test/core/transport/chttp2/writable_streams_test.cc`
208    *   `test/core/transport/chttp2/read_context_test.cc`
209    *   `test/core/transport/chttp2/http2_security_frame_test.cc`
210    *   `test/core/transport/chttp2/write_cycle_test.cc`
211    *   `test/core/transport/chttp2/stream_test.cc`
212 
213*   **Common Component Tests:**
214    *   `test/core/transport/chttp2/flow_control_fuzzer.cc`
215    *   `test/core/transport/chttp2/flow_control_manager_test.cc`
216    *   `test/core/transport/chttp2/flow_control_test.cc`
217    *   `test/core/transport/chttp2/hpack_encoder_test.cc`
218    *   `test/core/transport/chttp2/hpack_parser_test.cc`
219    *   `test/core/transport/chttp2/http2_settings_test.cc`
220    *   `test/core/transport/chttp2/write_size_policy_fuzztest.cc`
221    *   `test/core/transport/chttp2/write_size_policy_test.cc`
222 
223*   **PH2 End-to-end Tests:**
224    *   `test/core/end2end/end2end_ph2_config.cc`: This file defines the test
225    configuration for running end-to-end tests with PH2 transport.
226    It enables/disables specific tests. It tests :
227        1. PH2 client vs CHTTP2 server. (Done)
228        1. CHTTP2 client vs PH2 server. (Coming Soon)
229        1. PH2 client vs PH2 server. (Coming Soon)
230 
231## Dependencies for PH2
232 
233*   **gRPC Promise Library:**
234    *   PH2 heavily relies on the gRPC Promise framework [`src/core/lib/promise/`](../../../lib/promise/AGENTS.md)
235    *   Key components like [`party.h`](../../../lib/promise/party.h) are fundamental to PH2's async model.
236*   **Call Spine:** PH2 interacts with the V3 Call Spine components located in [`src/core/call/`](../../../call/AGENTS.md).
237 
238## Similarities of PH2 and Chaotic Good
239 
240PH2 shares several architectural similarities with the [Chaotic Good transport](../chaotic_good/AGENTS.md) :
241 
242*   **Promise-Based:** Both transports are built upon the gRPC Promise library for managing asynchronous operations. This is a departure from the callback-based system in CHTTP2.
243*   **Call V3 Stack:** Both are designed to work with the newer Call V3 stack.
244*   **Client Transport Implementation:** The client transport implementations in PH2 (`http2_client_transport.h`) and Chaotic Good (`client_transport.h`) show conceptual similarities in how they handle stream creation, frame sending/receiving, and interaction with the promise-based event loop.
245*   **Framing Concepts:** While the specific frame types and serialization differ (HTTP/2 vs. custom proto-based for Chaotic Good), the underlying concepts of defining frame structures (e.g., `frame.h` in both transports) and managing their serialization/deserialization are present in both.
246*   **Endpoint Interaction:** Both transports use the `PromiseEndpoint` abstraction or reading from and writing to the network, making the core transport logic independent of the underlying I/O mechanism.
247*   **Stream Initiation:** In PH2, `Http2ClientTransport::StartCall` initiates a new stream. It acquires a lock, assigns a new stream ID, creates a `Stream` object, and spawns the `CallOutboundLoop` to handle the stream's outgoing messages. Chaotic Good follows a similar pattern in `ChaoticGoodClientTransport::StartCall`, where it calls `StreamDispatch::MakeStream` to allocate a stream ID and create a `Stream` object, and then spawns `CallOutboundLoop` for the stream's lifecycle.
248 
249# PH2 Transport Party Slots
250 
251The HTTP2 transport uses Promise Party internally to manage scheduling of jobs.
252Since the number of slots in a Party is 16, we need to account for all the slots
253that we use in the transport.
254We need to ensure that our slots do not exceed 16.
255 
256## PH2 Common Party Slots Usage
257 
258<!--
259TODO(tjagtap) [PH2][P2] Validate this before roll out begins.
260Last checked on 26-June-2026
261-->
262 
263| Name | Category | Description | Max Spawns at a time | When is it spawned | Max Duration | Resolution |
264|---|---|---|---|---|---|---|
265| SecurityFrameLoop | Loop | Security Frame | 1 | After Constructor | Lifetime of the transport | Transport Close |
266| ReadLoop | Loop | | 1 | After 1st write | Lifetime of the transport | Transport Close |
267| BdpLoop | Loop | BDP Loop | Default - 1 (0 if GRPC_ARG_HTTP2_BDP_PROBE is explicitly set to false) | After Constructor | Lifetime of the transport | Transport Close |
268| MultiplexerLoop | Loop | | 1 | After Constructor | Lifetime of the transport | Transport Close |
269| AddData | Misc | ChannelZ AddData | 1 | On demand | Immediate | Immediate |
270| CloseTransport | Misc | Close transport | 1 | While closing transport. Only once in the life of a transport | As long as it takes to close the transport | Transport Close |
271| WaitForSettingsTimeout | Timeout | Settings Timeout | 1 | When we write SETTINGS | Settings timeout | Settings Ack Received or Settings Timeout |
272| KeepaliveLoop | Loop | Keepalive Loop | 1 | If Keepalive is enabled, after constructor | Lifetime of the transport | Transport Close |
273| Ping | Timeout + Misc | | 4 | Sending a ping request | Timeout or a specific duration | |
274| | | **Total** | 12 | | | |
275 
276## PH2 Client Party Slots Usage
277 
278<!--
279TODO(tjagtap) [PH2][P2] Validate this before roll out begins.
280Last checked on 26-June-2026
281-->
282 
283| Name | Category | Description | Max Spawns at a time | When is it spawned | Max Duration | Resolution |
284|---|---|---|---|---|---|---|
285 
286## PH2 Server Party Slots Usage
287 
288<!--
289TODO(tjagtap) [PH2][P2] Validate this before roll out begins.
290Last checked on 26-June-2026
291-->
292 
293| Name | Category | Description | Max Spawns at a time | When is it spawned | Max Duration | Resolution |
294|---|---|---|---|---|---|---|
295| Graceful Goaway | Misc | | 1 | Sending a graceful Goaway | As long as it takes to complete the graceful goaway process | |
296| | | **Total** | 1 | | | |
297 
298