Merge JSON files / Join arrays by ID

Merge JSON arrays by ID

To merge JSON arrays by ID, index each record by its stable ID, combine matching fields, and choose which source wins conflicts. Appending arrays is easy, but it does not reconcile records. If two exports contain the same product, appending creates two entries. A keyed join treats the product ID as an identity and combines matching records into one. That can preserve a description from one export while adding a price from another. It also creates a decision when both records contain different prices: which source should win?

The JSON file merger makes that decision explicit. Advanced mode lets you select an ID key, reorder files, choose earlier or later values, and inspect conflicts by source. Quick mode infers a likely ID for two lists of records. Inference is a convenience, not proof of identity. Always read the chosen-method sentence before trusting the result.

Pick a stable, meaningful key

Use an identifier that means the same thing in every input. A database ID from one system may not match an ID from another system. A display name can change, and an email may not be unique. Missing IDs need a separate rule: keep unmatched records as independent entries or discard them. Do not accidentally join all missing values into one record. The browser tool keeps them independently by default.

In advanced mode, user.id selects a nested key. If the list lives inside an API response, set Records path to something like data.items. The output is the merged array, not a reconstructed API response. Dot paths cannot address property names containing literal dots. Numeric IDs and string IDs remain distinct: the number 7 is not the string "7". Case-insensitive matching applies only to string IDs and should be enabled only when the source system treats case as insignificant.

A small, explicit Python join

This example accepts arrays of objects with nonempty string IDs. It preserves first-seen record order, adds new fields, and lets the later record replace a conflicting top-level field. It is a shallow record merge: nested objects are replaced, unlike the browser tool’s recursive merge. That difference is intentional and is tested by keeping this example focused on flat records.

from copy import deepcopy


def join_by_id(*groups, key="id"):
    output = []
    index = {}
    for records in groups:
        if not isinstance(records, list):
            raise ValueError("Each group must be a list")
        for record in records:
            if not isinstance(record, dict):
                raise ValueError("Each record must be an object")
            identity = record.get(key)
            if not isinstance(identity, str) or not identity:
                raise ValueError("Each record needs a nonempty string ID")
            if identity in index:
                output[index[identity]].update(deepcopy(record))
            else:
                index[identity] = len(output)
                output.append(deepcopy(record))
    return output


first = [{"id": "p1", "name": "Notebook", "price": 5}]
second = [{"id": "p1", "price": 6}, {"id": "p2", "name": "Pen"}]
result = join_by_id(first, second)
assert result == [
    {"id": "p1", "name": "Notebook", "price": 6},
    {"id": "p2", "name": "Pen"},
]
assert first[0]["price"] == 5  # Inputs were not mutated.
print(result)

Save the block as join_by_id.py and run this command in the same directory. The assertions fail if the example stops preserving its documented behavior.

python3 join_by_id.py

Decide what to do with nested lists

A record may contain tags, addresses, or transactions. Joining the parent record does not automatically establish how to merge those child lists. Concatenation preserves every item, including duplicates. Unique combination removes structurally equal items, not items sharing a business ID. Replacement chooses one entire list using the earlier/later rule. The browser tool exposes all three options and applies the selection wherever two lists meet.

Two address objects with the same address ID but different text are not structurally equal, so unique combination retains both. If child records also require keyed joins, normalize those child lists separately or write a domain-specific recursive join. The deep merge guide explains why recursion alone does not answer array identity questions.

Review conflicts, not just the record count

A stable output count does not prove correctness. A later file could overwrite a meaningful value with an empty string or null. In this tool those are explicit JSON values, not deletion commands. Earlier-wins keeps the previous conflicting value but still accepts fields that were not present before. Duplicate IDs inside one file are also joined, so inspect source data if fewer records appear than expected.

Before a production merge, test one matching ID, one new ID, one missing ID, and one deliberate conflict. Download the conflict CSV and keep it alongside the result if you need a review trail. The CSV reports sources and encountered differences; it is not an immutable audit log or a substitute for retaining original files. For scheduled workflows, adapt the Python file-processing guide. For a local, visual workflow, join your JSON files in the browser without uploading them.

Check identity before resolving values

Before joining a full export, examine whether each candidate key is unique within its source. Repeated IDs may be legitimate revisions, or they may indicate the key is not an entity identifier. A join can reduce those repetitions without telling you which interpretation was intended. Keep a small fixture where two different entities deliberately share a display name to confirm that you are not joining by appearance.

String normalization deserves the same care. Trimming whitespace or removing leading zeros can turn two different codes into one. This tool does not silently apply those transformations. Its optional case-insensitive rule lowercases string IDs for comparison only; it does not establish equivalence across different alphabets or external systems. A system-specific identity mapping should happen before the merge and should itself be reviewed.

Finally, verify stable order in the downloaded result. Matching records stay at their first-seen position, while new IDs append as they arrive. This makes the result easier to compare, but it is not a sort by ID or timestamp. If a downstream consumer requires sorted output, perform that as a separate explicit step after checking the join and conflicts.

When not to use this approach

Do not join records when the candidate key is not a stable identity shared by every source. A display name, row position, or ID allocated independently by another system can combine unrelated people or products. Append independent events instead. The flat Python example is not suitable for recursively merging nested fields or generating a source-aware conflict report. Use a reviewed identity mapping before joining, and keep unmatched records until you can explain what each missing key means.