How to Compare Multiple CSV Files in Python

In this tutorial, we will learn how to compare multiple CSV files in Python. CSV (Comma Separated Values) files are commonly used for storing and transferring data in a tabular format, and it’s a scenario commonly encountered when dealing with data analysis or data science tasks.

Python, being a powerful programming language with vast libraries, offers an easy approach to dealing with such tasks.

Step 1: Importing Necessary Libraries

The very first step is to import the necessary Python libraries. For this task, we will be using the Pandas library, a flexible and powerful data analysis library.

1
import pandas as pd

Step 2: Load CSV Files

After importing the necessary libraries, we will load the CSV files we want to compare. Here we’re assuming that we have two CSV files, ‘file1.csv’ and ‘file2.csv’. You would have to replace these with the actual paths of your CSV files.

1
2
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')

file1.csv:

Name,Age,City
Alice,25,New York
Bob,30,Los Angeles
Charlie,35,Chicago

file2.csv:

Name,Age,City
David,28,Houston
Eve,22,Miami
Frank,40,Denver

Step 3: Compare the CSV Files

Next, we’ll use pandas to compare the files. We will find all the differences between the two dataframes and save them in a third dataframe.

1
df3 = df1.compare(df2)

Step 4: Display the Differences

Finally, we’ll display the differences between the two CSV files.

1
print(df3)

The full code

Putting it all together, here’s the complete Python source code:

1
2
3
4
5
6
7
8
import pandas as pd
 
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')
 
df3 = df1.compare(df2)
 
print(df3)
 
      Name         Age               City         
      self  other self other         self    other
0    Alice  David   25    28     New York  Houston
1      Bob    Eve   30    22  Los Angeles    Miami
2  Charlie  Frank   35    40      Chicago   Denver

Conclusion

Comparing multiple CSV files in Python is very straightforward thanks to pandas. With just a few lines of code, you can load your CSV files, compare them, and display any differences.

Understanding how to compare CSV files can be incredibly useful, especially when dealing with large datasets in data analysis and data science.

For more in-depth details, you can refer to the official pandas documentation.