Difficulties in extracting data from CSV and matching it with Product model, please advise

Hi I’m stuck with a Django Python project where I’m trying to build an application where you can upload a csv file to then extract its values to generate a sales report pdf. Products can be added via the admin console and if those items appear (at the right place for now, eg item 4 in each row gets checked on) in the csv file they get extracted and added to the report (calculating the sum, storing the date of each purchase).

For completeness I will add the entire csv_upload_view function to then mention the specific parts I'm having difficulties with:

def csv_upload_view(request):
    print('file is being uploaded')

    if request.method == 'POST':
        csv_file_name = request.FILES.get('file').name
        csv_file = request.FILES.get('file')
        obj, created = CSV.objects.get_or_create(file_name=csv_file)
        #result = []

        if created:
            obj.csv_file = csv_file
            obj.save()
            with open(obj.file_name.path, 'r') as f:
                reader = csv.reader(f)
                reader.__next__()
                for row in reader:
                    print(row,type(row))
                    data = " ".join(row)
                    data = data.split(";")
                    print(data, type(data))
                # print(data[2])
                #data = data.split(';')
                #data.pop()
                #print(data)

                    transaction_id = data[1]
                    product = data[2]
                    quantity = int(data[3])
                    customer = data[4]
                    date = parse_date(data[5])

                    print(transaction_id, product, quantity, customer, date)

                try:
                    product_obj = Product.objects.get(name__iexact=product)
                except Product.DoesNotExist:
                    product_obj = None


        return HttpResponse()

The csv file is as follows:

POS,Transaction id,Product,Quantity,Customer,Date
1,E100,TV,1,Test Customer,2022-09-19
2,E100,Laptop,3,Test Customer,2022-09-20
3,E200,TV,1,Test Customer,2022-09-21
4,E300,Smartphone,2,Test Customer,2022-09-22
5,E300,Laptop,5,New Customer,2022-09-23
6,E300,TV,1,New Customer,2022-09-23
7,E400,TV,2,ABC,2022-09-24
8,E500,Smartwatch,4,ABC,2022-09-25

I’m having 2 main problems, the first is that following a tutorial with someone using a Mac (I’m on Windows but also saving the csv file in Macintosh format didn’t fix this) the code he uses just doesn’t work for me. It literally returns an empty string:

with open(obj.csv_file.path, 'r') as f:
                reader = csv.reader(f)
                reader.__next__()
                for row in reader:
                    data = "".join(row)
                    data = data.split(';')
                    data.pop()

My workaround here is then to write the following code which generates a sting separated by ‘;’:

for row in reader:
                    print(row,type(row))
                    data = " ".join(row)
                    data = data.split(";")

As part of this first problem I’m currently unable to grab the elements. I’m thinking that I probably need to convert the values into a list, but there’s a problem to that which is my main problem:
Going further in the code

transaction_id = data[1]
                    product = data[2]
                    quantity = int(data[3])
                    customer = data[4]
                    date = parse_date(data[5])

                    print(transaction_id, product, quantity, customer, date)

                try:
                    product_obj = Product.objects.get(name__iexact=product)
                except Product.DoesNotExist:
                    product_obj = None
                
        return HttpResponse()

The terminal prints out:

Quit the server with CONTROL-C.
[26/Sep/2022 19:15:25] "GET /reports/from-file/ HTTP/1.1" 200 11717
file is being uploaded
['1', 'E100', 'TV', '1', 'Test Customer', '9/19/2022'] <class 'list'>
1;E100;TV;1;Test;Customer;9/19/2022 <class 'str'>
; E 1 0 None
(…)

It turns out the product_obj always has the value None which it also had when I was playing around with the iteration where I occasionally could grad elements but never product_obj which is constantly set to None. I really appreciate if you can have a look at this!

using csv.reader - no need to join and then split back row, etc.

with open(obj.csv_file.path, 'r') as f:
    rdr = csv.reader(f)
    next(rdr) # skip the header row

    # alternative 1
    for row in rdr:
        pos, transaction_id, product, quantity, customer, transaction_date = row
        # here work with product to check if exists

    # alternative 2
    for pos, transaction_id, product, quantity, customer, transaction_date in rdr:
        # here work with product to check if exists

Using csv.DictReader:

with open(obj.csv_file.path, 'r') as f:
    rdr = csv.DictReader(f)
    for row in rdr:
        product = row['Product'] # this will work even when column order/number has changed
        # here work with product to check if exists

Now I see this is your third question in a row on the same problem. You've got some good working code and some ill advise. Anyway, it doesn't look like you actually try to understand the code you are given.

Back to Top