Debugging JSON APIs: Tools and Techniques

Common API Debugging Challenges

After building translation APIs handling thousands of requests, I've seen these issues repeatedly. Recognizing them quickly is half the battle.

The mysterious 500 error

When building AI-powered translation workflows for PIM systems, I encountered a 500 error that only happened for certain product catalogs. The error message was useless: "Internal Server Error". No stack trace, no details.

The culprit: deeply nested JSON exceeding the parser's default recursion limit. The API silently crashed when processing product hierarchies more than 20 levels deep.

Authentication that works in Postman but fails in code

This happens when you copy API calls from Postman without understanding the authentication flow. Postman handles token refresh automatically, your code doesn't.

Response structure changes breaking clients

When managing international expansion across Copenhagen-based teams, I've seen APIs change field names without versioning. Suddenly product_name becomes productName, and your code crashes.

CORS errors that make no sense

The error says "CORS policy blocked the request" but the API has Access-Control-Allow-Origin: *. Turns out the request is failing before CORS checks, usually authentication or malformed JSON.

Timeout issues with large JSON payloads

Translation APIs processing 10,000 product descriptions would randomly timeout. The problem wasn't the API - it was client-side JSON parsing of 50MB responses blocking the event loop.

Browser DevTools Techniques

Chrome and Firefox DevTools are my first debugging tool. Here's how I use them for JSON API debugging.

Network panel essentials

Open DevTools (F12), go to Network tab, filter by "Fetch/XHR":

• Request Headers: Verify authentication tokens, content-type, custom headers
• Request Payload: Inspect exactly what JSON is being sent
• Response Headers: Check status codes, CORS headers, cache settings
• Response Body: View formatted JSON or raw response
• Timing: Identify slow requests (waiting vs download time)

Copy as cURL

Right-click any request → Copy → Copy as cURL. This captures the exact request including headers and auth. Paste into terminal to reproduce outside the browser.

# Copied from DevTools - perfect reproduction
curl 'https://api.example.com/products' \
  -H 'Authorization: Bearer eyJhbGc...' \
  -H 'Content-Type: application/json' \
  --data-raw '{"category":"electronics"}'

Edit and resend requests

In Firefox DevTools, you can right-click a request and choose "Edit and Resend". This lets you modify headers, body, or URL parameters and see what happens.

Preserve log across page reloads

Check "Preserve log" in the Network panel. When debugging authentication flows that redirect, this keeps all requests visible.

Filter by status code

Click "All" filter in Network panel, select status codes to filter. When debugging errors, filter by 4xx and 5xx to ignore successful requests.

Search all network requests

Cmd/Ctrl + F while in Network panel searches across all requests, responses, and headers. Use this to find where a specific field appears.

Console debugging techniques

Log fetch requests and responses:

// Intercept all fetch calls for debugging
const originalFetch = window.fetch;
window.fetch = async (...args) => {
  console.log('Fetch Request:', args[0], args[1]);

  const response = await originalFetch(...args);
  const clone = response.clone();

  try {
    const json = await clone.json();
    console.log('Fetch Response:', json);
  } catch (e) {
    console.log('Fetch Response (not JSON):', await clone.text());
  }

  return response;
};

API Clients: Postman, Insomnia, Thunder Client

When building translation APIs, I use dedicated API clients for testing. Here's my comparison based on daily use.

Postman

The industry standard. I've used it for years.

Strengths:

• Powerful collection organization
• Environment variables for dev/staging/prod
• Pre-request and test scripts (JavaScript)
• Team collaboration features
• Request history and search

Weaknesses:

• Bloated application (memory heavy)
• Requires account for sync
• Complex UI for simple tasks

Example workflow in Postman:

// Pre-request script: Get auth token
pm.sendRequest({
  url: 'https://api.example.com/auth/token',
  method: 'POST',
  header: { 'Content-Type': 'application/json' },
  body: {
    mode: 'raw',
    raw: JSON.stringify({
      username: pm.environment.get('username'),
      password: pm.environment.get('password')
    })
  }
}, (err, res) => {
  const token = res.json().token;
  pm.environment.set('auth_token', token);
});

// Test script: Validate response
pm.test("Response has valid structure", () => {
  const json = pm.response.json();
  pm.expect(json).to.have.property('data');
  pm.expect(json.data).to.be.an('array');
});

Insomnia

My personal favorite for day-to-day API testing.

Strengths:

• Clean, minimal interface
• Fast and lightweight
• Great JSON/GraphQL support
• Template variables with Nunjucks
• No mandatory account

Weaknesses:

• Fewer advanced features than Postman
• Smaller plugin ecosystem

Thunder Client (VS Code extension)

For developers who live in VS Code.

Strengths:

• Integrated into VS Code
• Extremely lightweight
• Collections stored as JSON files in repo
• No separate application needed

Weaknesses:

• Limited features vs Postman/Insomnia
• No scripting support

REST Client (VS Code extension)

My secret weapon for simple API testing. Define requests in .http files:

### Get all products
GET https://api.example.com/products
Authorization: Bearer {{$processEnv AUTH_TOKEN}}

### Create product
POST https://api.example.com/products
Content-Type: application/json

{
  "name": "Wireless Mouse",
  "price": 29.99,
  "category": "electronics"
}

### Variables
@baseUrl = https://api.example.com
@productId = PROD123

### Get specific product
GET {{baseUrl}}/products/{{productId}}

These files live in your repo. Anyone on the team can run requests directly from VS Code. No Postman collections to export and import.

Command-Line Tools: curl, jq, httpie

When debugging production issues via SSH or in CI/CD pipelines, GUI tools aren't available. Command-line tools are essential.

curl: The universal HTTP tool

Every JSON API request I make starts with curl.

Basic GET request:

curl https://api.example.com/products

POST JSON data:

curl -X POST https://api.example.com/products \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{
    "name": "Wireless Mouse",
    "price": 29.99
  }'

Show response headers:

curl -i https://api.example.com/products

Verbose output (debug connection):

curl -v https://api.example.com/products

Follow redirects:

curl -L https://api.example.com/products

Save response to file:

curl https://api.example.com/products -o products.json

Upload file:

curl -X POST https://api.example.com/upload \
  -F "file=@products.json"

jq: JSON processor for command line

When building translation workflows, I use jq constantly to parse and transform JSON responses.

Pretty-print JSON:

curl https://api.example.com/products | jq .

Extract specific field:

curl https://api.example.com/products | jq '.data[0].name'

Filter arrays:

# Get products over $50
curl https://api.example.com/products | jq '.data[] | select(.price > 50)'

Map/transform data:

# Extract just names and prices
curl https://api.example.com/products | jq '.data[] | {name, price}'

Count items:

curl https://api.example.com/products | jq '.data | length'

Find deeply nested values:

# Find all email addresses in nested structure
curl https://api.example.com/users | jq '.. | .email? | select(. != null)'

httpie: User-friendly HTTP client

When I need human-readable output, I reach for httpie.

Installation:

brew install httpie  # macOS
pip install httpie   # Python

Simple GET:

http https://api.example.com/products

POST JSON (automatic Content-Type):

http POST https://api.example.com/products \
  name="Wireless Mouse" \
  price:=29.99 \
  Authorization:"Bearer TOKEN"

Download file:

http --download https://api.example.com/export/products.json

Combining tools for powerful debugging

# Find all products with invalid prices
curl -s https://api.example.com/products | \
  jq '.data[] | select(.price < 0 or .price == null) | {id, name, price}'

# Count products by category
curl -s https://api.example.com/products | \
  jq '.data | group_by(.category) | map({category: .[0].category, count: length})'

# Test API and save only errors
curl -s https://api.example.com/products | \
  jq 'if .error then . else empty end' > errors.json

Debugging Deeply Nested JSON Structures

When processing product catalogs with 10+ levels of nesting, finding the problematic field is challenging.

Pretty-print with path annotations

// Node.js utility to show JSON with paths
function printWithPaths(obj, path = '') {
  if (obj === null || typeof obj !== 'object') {
    console.log(path, '=', obj);
    return;
  }

  if (Array.isArray(obj)) {
    obj.forEach((item, idx) => {
      printWithPaths(item, `${path}[${idx}]`);
    });
  } else {
    Object.keys(obj).forEach(key => {
      printWithPaths(obj[key], path ? `${path}.${key}` : key);
    });
  }
}

// Usage
const response = await fetch('/api/products').then(r => r.json());
printWithPaths(response);

JSONPath for complex queries

# Find all prices in nested structure
echo '{"products": [{"variants": [{"price": 29.99}]}]}' | \
  jq '.. | .price? | select(. != null)'

Validate nested structure

// Check if deeply nested path exists
function hasPath(obj, path) {
  const parts = path.split('.');
  let current = obj;

  for (const part of parts) {
    if (current === null || current === undefined) {
      console.log(`Path breaks at: ${part}`);
      return false;
    }
    current = current[part];
  }

  return true;
}

// Usage
hasPath(apiResponse, 'data.products.0.variants.0.price');

Tracking Down Malformed JSON Sources

When translation APIs return invalid JSON, finding the source is critical. Here's my debugging workflow.

Identify the exact error location

// Better JSON parsing with error context
function parseJSONWithContext(jsonString) {
  try {
    return JSON.parse(jsonString);
  } catch (error) {
    const match = error.message.match(/position (\d+)/);
    if (match) {
      const pos = parseInt(match[1]);
      const start = Math.max(0, pos - 50);
      const end = Math.min(jsonString.length, pos + 50);
      const context = jsonString.substring(start, end);

      console.error('JSON parse error at position', pos);
      console.error('Context:', context);
      console.error(' '.repeat(50) + '^');
    }
    throw error;
  }
}

Common malformation patterns

Trailing commas:

// Invalid - trailing comma
{
  "name": "Product",
  "price": 29.99,
}

Single quotes instead of double:

// Invalid - single quotes
{'name': 'Product'}

Unescaped quotes in strings:

// Invalid
{"description": "The product is 10" tall"}

NaN or Infinity values:

// JavaScript creates invalid JSON
const obj = { value: NaN };
JSON.stringify(obj); // '{"value":null}' - NaN becomes null

Validate JSON before parsing

# Use jq to validate
echo '{"invalid": json}' | jq .
# parse error: Invalid numeric literal at line 1, column 14

Fix common issues automatically

function fixCommonJSONIssues(jsonString) {
  return jsonString
    .replace(/,(\s*[}\]])/g, '$1')  // Remove trailing commas
    .replace(/'/g, '"')              // Replace single quotes
    .replace(/(\w+):/g, '"$1":')     // Quote unquoted keys
    .replace(/\n/g, '\\n')           // Escape newlines
    .replace(/\t/g, '\\t');          // Escape tabs
}

Logging and Monitoring JSON API Responses

When building enterprise translation systems, proper logging saved me countless debugging hours.

Structured logging

const winston = require('winston');

const logger = winston.createLogger({
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'api.log' })
  ]
});

app.use((req, res, next) => {
  const start = Date.now();

  res.on('finish', () => {
    logger.info({
      method: req.method,
      path: req.path,
      status: res.statusCode,
      duration: Date.now() - start,
      requestBody: req.body,
      responseSize: res.get('content-length'),
      userAgent: req.get('user-agent')
    });
  });

  next();
});

Request/response logging middleware

function logAPICall(req, res, next) {
  // Log request
  console.log('→ Request:', {
    method: req.method,
    url: req.url,
    headers: req.headers,
    body: req.body
  });

  // Capture response
  const originalSend = res.send;
  res.send = function(data) {
    console.log('← Response:', {
      status: res.statusCode,
      headers: res.getHeaders(),
      body: data
    });
    originalSend.call(this, data);
  };

  next();
}

Error tracking with context

app.use((err, req, res, next) => {
  const errorContext = {
    error: err.message,
    stack: err.stack,
    request: {
      method: req.method,
      url: req.url,
      headers: req.headers,
      body: req.body,
      query: req.query,
      params: req.params
    },
    user: req.user?.id,
    timestamp: new Date().toISOString()
  };

  // Log to file/service
  logger.error(errorContext);

  // Send generic error to client
  res.status(500).json({ error: 'Internal server error' });
});

API monitoring checklist

• Log all 4xx and 5xx responses with full context
• Track response time percentiles (p50, p95, p99)
• Monitor payload sizes (request and response)
• Alert on error rate spikes
• Track authentication failures
• Monitor rate limit hits

Real Debugging War Stories

Here are actual debugging scenarios I've encountered in production.

The midnight unicode disaster

At 2 AM, translation API started returning corrupted text. Product names like "Café" became "Café". The issue: a load balancer was added that wasn't configured for UTF-8, converting responses to ISO-8859-1.

Diagnosis: curl showed response headers had Content-Type: application/json without charset=utf-8.

Fix: Added explicit charset to API responses.

The case of the invisible character

JSON looked perfect in editors but failed to parse. The culprit: a zero-width non-joiner character (U+200C) copied from a Word document into a product description.

Diagnosis: Opened JSON in hex editor, found E2 80 8C (UTF-8 encoding of ZWNJ).

Fix: Added Unicode normalization to input sanitization.

The mystery of the cached error

API returned 200 OK with error message in body. Why? CDN was caching error responses from a previous outage, serving them with 200 status.

Diagnosis: curl -I showed X-Cache: HIT header even for error responses.

Fix: Added Cache-Control: no-store to error responses.

The 10-hour debugging session

When building content translation workflows in Copenhagen, API randomly returned empty arrays. After 10 hours, discovered the database query used SELECT * FROM products WHERE id IN () - an empty array of IDs produced by upstream filtering.

Diagnosis: Added query logging, saw empty IN clause.

Fix: Validate input arrays aren't empty before querying.

JSON API Debugging Checklist

When an API isn't working, I work through this checklist systematically.

Request validation

• Is the HTTP method correct (GET, POST, PUT, DELETE)?
• Is the URL correct (check for typos, trailing slashes)?
• Are required headers present (Content-Type, Authorization)?
• Is the request body valid JSON?
• Are all required fields included?
• Are data types correct (string vs number)?

Response analysis

• What is the HTTP status code?
• Are response headers present (Content-Type, CORS)?
• Is the response body valid JSON?
• Does the response match expected schema?
• Are error messages helpful?

Authentication issues

• Is the token/API key valid?
• Is the token expired?
• Is the Authorization header formatted correctly?
• Are permissions correct for the endpoint?

Network and connectivity

• Can you reach the API (ping, curl)?
• Are you behind a proxy or VPN?
• Is CORS configured correctly?
• Are there rate limits being hit?
• Is the request timing out?

Data issues

• Is the JSON properly formatted?
• Are there encoding issues (UTF-8)?
• Are there special characters breaking parsing?
• Is the payload too large?
• Is nesting too deep?

I keep this checklist in my notes and reference it when debugging unfamiliar APIs. It prevents me from missing obvious issues when I'm tired or frustrated.