How to Append CSV Files using Python Pandas

This tutorial will guide you through the process of appending CSV files using Python’s Pandas.

This is a common practice in data manipulation and can help you manage massive datasets by combining smaller ones. It’s a critical skill for anyone dealing with data analysis or data science.

Prerequisites

To follow this tutorial, you need to have Python installed on your computer. You also need the Pandas library, which can be installed using the package manager pip. Open your terminal and run the following command:

1
pip install pandas

Step 1: Importing the necessary libraries

First, we need to import pandas into our script. We can do this using the following line of code:

1
import pandas as pd

Step 2: Reading the CSV files

Before appending the CSV files, we need to read them using pandas. Let’s assume we have two CSV files: ‘file1.csv’ and ‘file2.csv’. To read these files, we use pandas’ read_csv function:

1
2
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')

file1.csv:

Name,Age,City
Alice,25,New York
Bob,30,Los Angeles
Charlie,35,Chicago

file2.csv:

Name,Age,City
David,28,Houston
Eve,22,Miami
Frank,40,Denver

Step 3: Appending the CSV files

Once we’ve read the CSV files, appending them is a simple process. We’ll use the Pandas append function to do that:

1
combined_data = df1.append(df2)

Step 4: Saving the new DataFrame to a CSV file

After appending the CSV files, the result is stored in the ‘combined_data’ DataFrame. To save this DataFrame into a new CSV file, we use pandas to_csv function:

1
combined_data.to_csv('combined_data.csv', index=False)

Now, ‘combined_data.csv’ contains the combined data from ‘file1.csv’ and ‘file2.csv’.

Full code

1
2
3
4
5
6
7
import pandas as pd
 
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')
 
combined_data = df1.append(df2)
combined_data.to_csv('combined_data.csv', index=False)

combined_data.csv:

Name,Age,City
Alice,25,New York
Bob,30,Los Angeles
Charlie,35,Chicago
David,28,Houston
Eve,22,Miami
Frank,40,Denver

Conclusion

By following this tutorial, you learned how to append CSV files using Python’s pandas library. This skill is very useful for data manipulation and analysis. Pandas provides a host of other features for data manipulation that you can explore using their official documentation.