Merge JSON files / Python guide

Merge JSON files with Python

To merge JSON files in Python, load each file with the standard-library json module, then concatenate lists, recursively merge objects, or index records by a shared key. Python is useful when merging is part of a repeatable job: a scheduled export, a repository build, or a local data cleanup. Its standard library can read and write JSON without an extra package. The important decision is not which library to install. It is what “merge” means for your data. Appending two arrays is different from matching records by ID, and replacing an object is different from preserving its nested fields.

If you only need a one-time result, use the browser JSON file merger. It lets you inspect conflicts with their source files before downloading. The scripts below are deliberately smaller: they demonstrate explicit rules, reject unexpected top-level shapes, and do not reproduce the browser tool’s complete conflict report.

Choose your rule before writing files

Start by checking the outermost value in each input. A list of independent events often needs concatenation. Two configuration objects may need a shallow or deep merge. Two customer exports usually need a keyed join so that the same person does not appear twice. File order matters whenever a later value replaces an earlier value. Write down that order instead of relying on whatever order a folder happens to return.

Keep original files and write to a new output path. A syntactically valid result can still be wrong for the application that consumes it. For example, a shallow object merge can silently discard nested settings. Inspect a small fixture with an intentional conflict before running a large batch.

A runnable command-line script

Save this as merge_json.py. It accepts either lists or objects, an output path, and at least two input paths. List mode concatenates items without deduplication. Object mode performs a shallow merge with later files winning. It rejects nonstandard NaN and Infinity values and reports read or parse errors without producing a partial output file during validation.

import json
import sys
from pathlib import Path


def reject_constant(value):
    raise ValueError(f"Nonstandard JSON constant: {value}")


def main(args):
    if len(args) < 4 or args[0] not in ("lists", "objects"):
        raise ValueError("Usage: merge_json.py lists|objects OUT IN1 IN2 [IN...]")
    mode, output, *inputs = args
    destination = Path(output).resolve()
    if any(Path(name).resolve() == destination for name in inputs):
        raise ValueError("Output must not overwrite an input")
    merged = [] if mode == "lists" else {}
    expected = list if mode == "lists" else dict
    for name in inputs:
        with open(name, encoding="utf-8") as source:
            value = json.load(source, parse_constant=reject_constant)
        if not isinstance(value, expected):
            raise ValueError(f"{name}: expected {expected.__name__}")
        if mode == "lists":
            merged.extend(value)
        else:
            merged.update(value)
    encoded = json.dumps(merged, ensure_ascii=False, indent=2, allow_nan=False)
    with open(output, "w", encoding="utf-8") as target:
        target.write(encoded + "\n")


if __name__ == "__main__":
    try:
        main(sys.argv[1:])
    except (OSError, ValueError) as error:
        print(f"Merge failed: {error}", file=sys.stderr)
        sys.exit(1)

These shell commands create tiny fixtures and run both modes. Use a new working directory so that the example output names do not replace existing work. They assume a POSIX shell and an available python3 executable.

printf '%s\n' '[{"id":1}]' > first.json
printf '%s\n' '[{"id":2}]' > second.json
python3 merge_json.py lists combined.json first.json second.json
python3 -m json.tool combined.json
printf '%s\n' '{"region":"west","enabled":false}' > base.json
printf '%s\n' '{"enabled":true}' > override.json
python3 merge_json.py objects settings.json base.json override.json
python3 -m json.tool settings.json

What this script preserves and what it changes

Input list order is preserved. Reversing the filenames reverses which list contributes its items first. Object keys already present in the accumulator receive the later value, including when that value is null. A null is not treated as a deletion instruction. Nested objects are replaced wholesale, not recursively merged. If you need to keep both nested branches, read the deep versus shallow merge guide before changing the implementation.

The script loads each file into memory and retains the merged result, so it is not a streaming solution for arbitrarily large data. Whitespace and indentation from the original files are not preserved. Python integers retain integer precision, unlike JavaScript numbers beyond the safe integer range; decimal values still need deliberate handling if exact decimal arithmetic matters. JSON duplicate object keys are accepted by the default parser with the last occurrence retained. If duplicate keys are unacceptable, validate them with an object-pairs hook before merging.

When you need a keyed join instead

Suppose both files contain a record with ID 1. List concatenation leaves two records with ID 1. Object mode cannot solve that because the top-level inputs are arrays. A keyed join builds an index and combines matching records according to a field-level policy. See the array join guide for a tested Python function and the decisions that go with it.

For a visual review, return to the merge JSON files tool, use advanced mode, and select Join records by key. Try the sample files first. Compare earlier-wins and later-wins results, inspect the conflict sources, then apply the same rules to your own data. Automation is useful only after the desired output is clear.

Make the workflow reproducible

Keep an explicit input manifest when a job grows beyond two files. A shell wildcard can be convenient, but alphabetical order is not necessarily business priority. Name the base file first and list overrides in their intended precedence. Record the selected mode alongside the output so someone else can reproduce the same decision without guessing from the result.

For automated checks, include empty arrays, an empty object, a conflicting scalar, and a wrong-shape input. Check that failure leaves a previous result untouched. This script finishes reading and encoding before opening the output, but a disk failure during writing can still leave a partial file. A production pipeline can write to a temporary file in the destination directory, flush it, and atomically replace the destination when successful. Decide separately whether replacing an existing output is allowed.

Do not send private fixtures to a remote paste service while debugging. Replace names and account numbers with synthetic examples that preserve the relevant structure. When JSON contains credentials, also consider terminal history, build logs, and shared output folders. The browser tool does not log input, but your surrounding workflow can still expose data if you print or publish it.

When not to use this approach

Do not use this in-memory script for exports that exceed available RAM, or when a downstream schema needs transactional validation. Its object mode is shallow, so it is also the wrong choice when nested fields must survive overrides. Use the tested recursive function in the deep-merge guide or a domain-specific pipeline instead. If you need a one-off visual review of conflicting sources, the browser tool is easier to inspect than a script that only prints the final value.