Loop to create Multiple JSON files from the CSVs saved in multiple folder

Hi all can help to enhanced my python code for creating JSON files. I want to create loop that creates json (different json for each of the folder) from the csv files save in different folder and and then save all the json in the one common folder. Right now I am using below code and changing the path last two digits manually to create json one by one that is very tedious task.

import csv
import json
import glob
import os

class csv2jsonindirectory():
    def Python_trial(self):
        # Update the following variable with the path in windows and replace
        # every "\" with "/".
        path_to_folder = "C:\\Users\\CSVs\\AO"
        csv_files_in_folder = path_to_folder + '/*.csv'
        csvfilenames = []
        i = 1
        mydict = {}
        for filename in glob.glob(csv_files_in_folder):
            csvfilenames.append(os.path.splitext(filename)[0])
            rows = []
        for i in range(len(csvfilenames)):
            with open(csvfilenames[i] + ".csv", "r") as f:
                csvreader = csv.DictReader(f)
                rows = list(csvreader)
                mydict["chartdiv" + str(i + 1)] = rows

        print(mydict)

        with open(csvfilenames[0] + ".json", 'w') as f:
            json.dump(mydict, f, indent= 4)


dd = csv2jsonindirectory()
dd.Python_trial()

You should make the path a parameter of the function. That way you can call it multiple times for all the folders you want to process. The list of folders could come from command line parameters, a glob, or a built-in list, depending on what you need.

As a side note: Why are you using a class there? It doesn’t seem to do anything.

If you want to handle all CSV files under that folder, then you could scan the entire folder-tree using os.walk.

import os
...
path_to_folder="C:\\Users\\CSVs"
for root,dir,files in os.walk(path_to_folder):
  for file in files:
    if file.endswith('.csv'):
        {process this file to json}
1 Like

Thansk alot for the solution. I tried accomodate but I thing i did it incorerct . can you plese help me whrere add this chunk.

My folder structure is :-
Main project foder > CSV > Sub-Folders (Containng CSVs)

This is script is craeted by one of the senior who left the organization long ago. I need work in this script by I am new learner python. can you please help me how I can di the same.

Youe help is really apprciated!!

I’m not sure what you need help with. If you’re not sure how to define a function with parameters, the Python tutorial chapter about that is here: 4. More Control Flow Tools — Python 3.9.5 documentation

At a high level, here is what you currently have:

class A:
   function B:
       path_to_folder = <hard-coded path>
make A
A.B()

What you want to do is to change B so that it can take a path parameter:

class A:
   function B(path_to_folder):
       # not using hard-coded path_to_folder
make A

path_to_folder="<main proj folder>\\CSVs"
for root,dirs,files in os.walk(path_to_folder):
  for dir in dirs:
    A.B(dir)

Note that I’ve changed the os.walk code so that it iterates through folders, not files. Each folder path is passed to the changed routine which then processes each csv file in that folder.

You’ll want to follow @airtower-luna 's links on passing parameters, I don’t remember the syntax off-hand.

1 Like

Yeah just to add to @fire-eggs ’ answer, syntax-wise it’d be like this:

class A:

    def b(self, path_to_folder):
        pass # not using hard-coded path_to_folder


a_instance = A()
path_to_folder = "<main proj folder>\\CSVs"
for root, dirs, files in os.walk(path_to_folder):
    for dir in dirs:
        a_instance.b(dir)

However, you could make your life a bit simpler as you don’t need the wrapper class:

def python_trial(path):
        pass  # Code here, use the path variable

path_to_folder="<main proj folder>\\CSVs"
for root, dirs, files in os.walk(path_to_folder):
    for dir in dirs:
        python_trial(dir)
1 Like