How To Merge Multiple Json Files Into One In Python

Merging multiple JSON files into one is a common task in Python when working with large datasets or different data sources. By combining data from multiple files into a single file, you can better manage your data and analyze it for insights. In this tutorial, we will learn how to merge multiple JSON files into one using Python.

Step 1: Create a list of JSON files

First, we need to create a list of JSON file names that we want to merge. For demonstration purposes, let’s consider we have two JSON files named file1.json and file2.json.

1
json_files = ["file1.json", "file2.json"]

Step 2: Read and parse JSON files

Next, we need to read the JSON files, parse them using Python’s built-in JSON library, and store the parsed data in a dictionary.

1
2
3
4
5
6
7
8
import json
 
json_data = []
 
for file_name in json_files:
    with open(file_name, "r") as file:
        data = json.load(file)
        json_data.append(data)

Step 3: Merge JSON files

Now, we can merge the data in the json_data list into a single JSON object.

1
2
3
4
5
6
7
8
merged_data = {}
 
for data in json_data:
    for key, value in data.items():
        if key not in merged_data:
            merged_data[key] = value
        else:
            merged_data[key].extend(value)

This code snippet assumes that each JSON file has a dictionary with unique keys that store a list of values. If there’s a key present in both files, their values get combined into one list.

Step 4: Save the merged data to a new JSON file

Finally, we can save the merged data to a new JSON file.

1
2
3
4
output_file_name = "merged_data.json"
 
with open(output_file_name, "w") as output_file:
    json.dump(merged_data, output_file)

Full Code

The following is the complete code for merging multiple JSON files into one in Python.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import json
 
json_files = ["file1.json", "file2.json"]
 
json_data = []
 
for file_name in json_files:
    with open(file_name, "r") as file:
        data = json.load(file)
        json_data.append(data)
 
merged_data = {}
 
for data in json_data:
    for key, value in data.items():
        if key not in merged_data:
            merged_data[key] = value
        else:
            merged_data[key].extend(value)
 
output_file_name = "merged_data.json"
 
with open(output_file_name, "w") as output_file:
    json.dump(merged_data, output_file)

For the above code, let’s assume the content of the two JSON files is as follows:

file1.json:

1
2
3
4
{
  "data1": [1, 2, 3],
  "data2": [4, 5, 6]
}

file2.json:

1
2
3
4
{
  "data1": [7, 8, 9],
  "data2": [10, 11, 12]
}

Output

Upon running this script, a new file called merged_data.json will be created with the following content:

{
    "data1": [1, 2, 3, 7, 8, 9],
    "data2": [4, 5, 6, 10, 11, 12]
}

Conclusion

In this tutorial, we have learned how to merge multiple JSON files into one in Python using the built-in JSON library. By merging JSON files, you can efficiently manage your data and perform analysis on the consolidated data.