How To Read Multiple Files In A Loop In Python

Reading multiple files in a loop is a common task in data science and programming. In this tutorial, we will learn how to read multiple files in a loop using Python.

Steps:

1. put three files into the my_dir directory:

filename1.csv

Date,Value
2022-01-01,100
2022-02-01,200

filename2.csv

Name, Age, Location, Description
Alice, 30, New York, "Software developer, likes hiking"
Bob, 25, San Francisco, "Data scientist, enjoys cooking"
Cathy, 28, Los Angeles, "UX designer, loves traveling"
David, 32, Boston, "Product manager, passionate about photography"

filename3.csv

Name	Age	Occupation
John	Doe	aaa
Jeo	Duck	bbb

2. import the necessary library modules that will enable us to handle files. We will use the os module to get a list of files in a directory, and the pandas module to read files.

3. The next step is to use the os module to get a list of files in the directory. We will create a function that will take a directory path as an argument, and return a list of file paths.

4. Now, let’s use the pandas module to read each file in the directory. We will create a loop that will iterate through the list of file paths and read each file using the read_csv function from the pandas module. In this example, we will assume that all files are in CSV format.

5. You can now do whatever you want with the data. In this example, we are just printing the first 5 rows of each file.

Output:

         Date  Value
0  2022-01-01    100
1  2022-02-01    200
       Name  ...                     Description
Alice    30  ...                   likes hiking"
Bob      25  ...                 enjoys cooking"
Cathy    28  ...                loves traveling"
David    32  ...   passionate about photography"

[4 rows x 4 columns]
  Name\tAge\tOccupation
0        John\tDoe\taaa
1        Jeo\tDuck\tbbb

Conclusion:

In this tutorial, we have learned how to read multiple files in a loop using Python. We used the os module to get a list of file paths, and the pandas module to read each file. This method can be used to preprocess data or to analyze data in bulk.