Batching, A Sophisticated Approach to Network Optimization

Every network request initiated by a frontend application incurs a cost, latency, headers, CPU usage, parsing overhead, and server load. The question is, can we optimize to do more with fewer requests?

The answer lies in the strategy known as batching.

Batching is a sophisticated approach that amalgamates multiple related network operations into a singular request, thereby curtailing the round-trip time and enhancing performance. This strategy is particularly advantageous on high-latency networks or when interacting with restricted APIs.

In this comprehensive guide, we will delve into:

• The fundamentals and significance of batching
• The identification and utilization of batching
• Implementation strategies in REST, GraphQL, and beyond
• Real-world implementation patterns observed in Google, Slack, and GitHub
• A comparative analysis of batching, parallelism, and streaming

The Rationale Behind Batching

Batching is driven by the need to:

• Minimize round-trip latency, especially on mobile networks
• Eliminate redundant headers
• Boost backend efficiency (e.g., prevent N+1 query problems)
• Simplify client-side complexity by grouping similar operations
• Combine multiple read or write operations into atomic actions

This concept often echoes as: “fetch all at once instead of piecemeal”.

Illustrations of Batching

REST Batching

In a typical REST scenario, instead of executing multiple GET requests as shown:

GET /api/user/1
GET /api/user/2
GET /api/user/3

You can consolidate these into a single operation either by POST or GET:

POST /api/users/batch
Body: { ids: [1, 2, 3] }

Or:

GET /api/users?ids=1,2,3

Then, return a single payload:

[
  { "id": 1, "name": "A" },
  { "id": 2, "name": "B" },
  { "id": 3, "name": "C" }
]

This way, you are reducing the network overhead by executing a single HTTP request instead of three.

GraphQL Batching

GraphQL, in its design, inherently supports batching of queries:

[
  { "query": "{ me { name } }" },
  { "query": "{ settings { theme } }" }
]

In the above scenario, the server resolves both queries and returns a combined response.

GraphQL client-side libraries such as Apollo Client, Relay, and urql natively support batching or allow configuration for the same.

UI-Level Batching

In a React application, you can batch dependent requests before rendering:

const user = await fetchUser();
const projects = await fetchProjects(user.id);

Alternatively, batch related queries using useQueries (a feature of React Query):

const results = useQueries([
  { queryKey: ["user", id], queryFn: fetchUser },
  { queryKey: ["settings"], queryFn: fetchSettings }
]);

In this scenario, useQueries provides a streamlined and efficient way to execute multiple asynchronous requests simultaneously, thereby improving application performance.

When Should You Batch?

Batching is an ideal choice when dealing with:

• Lists that contain multiple items (e.g., IDs, filters)
• Dashboard views comprising many sections
• Related mutations (e.g., saving a form group)
• Repeated data fetching on component mount

However, batching might not be the best solution:

• For unrelated operations
• If network latency isn't a significant issue
• For real-time user interfaces where data freshness is more critical than bundling

Implementation Patterns

1. Client-Side Queue with Delay

Buffer multiple requests for a short duration, let's say 5–50ms, and then send them together:

const queue = [];
let timeout;

function batchRequest(request) {
  queue.push(request);

  if (!timeout) {
    timeout = setTimeout(() => {
      sendBatch(queue);
      queue.length = 0;
      timeout = null;
    }, 10);
  }
}

This strategy is implemented in Apollo Link Batching and Google API batching. The delay allows time for additional requests to be added to the batch, thus optimizing the network load.

2. URL Merging

Encode the batch into query parameters:

GET /api/posts?ids=1,2,3,4

In this case, the server reads the encoded parameters and resolves them in a single DB call.

3. Batched Mutations

Group multiple write operations:

POST /api/cart/batch
Body: [
  { action: "add", productId: 1 },
  { action: "remove", productId: 2 }
]

These operations can be executed either transactionally or sequentially. This method is efficient when you need to perform multiple operations at once, like updating a shopping cart.

Real-World Usage

Slack

• Slack's Message history API batches over 100 messages at once.
• Typing indicators are batched and throttled for optimal performance.

GitHub

• GitHub leverages GraphQL to batch repository, PR, and commit information in a single payload.
• This approach helps avoid dozens of REST calls on the repository page, enhancing loading speed and user experience.

Google APIs

• Google APIs offer a batch endpoint used to submit multiple requests in one go.
• This endpoint uses multipart/mixed content type to handle multiple requests.

Anti-Patterns

• Batching unrelated data can hurt cacheability.
• Rebatching already-resolved cache data can lead to unnecessary computational overhead.
• Large batches may cause timeouts or memory pressure.
• Overengineering batching for small payloads can lead to unnecessary complexity.
• Batching mutations that shouldn’t be atomic (like financial operations) can lead to logical inconsistencies.

Conclusion: Apply Batching Intentionally, Not By Default

Batching is a powerful tool, but only when applied judiciously.

It’s not about reducing the lines of code or simplifying the codebase. It's about network efficiency and user-perceived speed.

While one request may be good, bundling five might yield better performance.

However, it's important to ensure that they belong together.

As we conclude, remember to batch intelligently, batch on a small scale, and batch with a clear purpose.