Click here to Skip to main content
14,240,716 members
Rate this:
Please Sign up or sign in to vote.
Hello,
Basically currently my program reads the Data file (electric info), sums the values up, and after summing the values, it changes all the negative numbers to 0, and keeps the positive numbers as they are. The program does this perfectly. This is the code I currently have:
import csv
from datetime import timedelta
from collections import defaultdict

def convert(item): 
    try:
        return float(item)
    except ValueError:
        return 0

sums = defaultdict(list)

def daily():
    lista = []
    with open('Data.csv', 'r') as inp:
        reader = csv.reader(inp, delimiter = ';')
        headers = next(reader)
        for line in reader: 
            mittaus = max(0,sum([convert(i) for i in line[1:-2]])) 
            lista.append()
            #print(line[0],mittaus) ('#'only to check that it works ok)
daily()


My question is: How can I save the data to lists, so I can add all the values per day, so should look something like this
1.1.2016;358006
2.1.2016;39
3.1.2016;0 ...
8.1.2016;239143

And After had having these in a list (to save later on to a new data file), it should calculate the cumulative values, which would look like this:
1.1.2016;358006
2.1.2016;358045
3.1.2016;358045...
8.1.2016;597188 

These should be in a new subprogram

Small peak what's behind the Data file: https://pastebin.com/9HxwcixZ [It's actually divided with ';' , not with ' ' as in the pastebin]
The data file: https://files.fm/u/yuf4bbuk

What I have tried:

def analysointi():
    lista = []
    reader = csv.reader(inp, delimiter = ';')
    for line in reader:
        lista.append(line)
    print(line[2])
analysointi()

Thinking about this way, it shouldn't work, as the values should already be in a list, so this way can be dumped.

I have tried to search for results, asked people, but I haven't got any solution. I just can't solve it myself, so I'd appreciate it so much if someone could help me
Posted
Updated 19-Nov-18 22:29pm
v3
Comments
Member 14060132 21-Nov-18 20:43pm
   
Still can't figure out how to save the data that in the first column is the date and in the second is the value. I have tried working with the datetime but it hasn't worked out.

1 solution

Rate this:
Please Sign up or sign in to vote.

Solution 1

Not sure I understand the issue. But once you have done the first set of calculations you can read the datafile and do the cumulative calculations. Once you have those you can then write out the new data.
   
Comments
Member 14060132 20-Nov-18 7:15am
   
The idea is to
1. Read the Data file
-> In this part it sums all the values, and after it's done, if a value is negative, it returns 0. If the calculation is positive, it keeps the default value. I have done this, and it works fine. It saves the values as:
2016-01-01 00:00:00 0
2016-01-01 08:00:00 356303.0
2016-01-01 16:00:00 0
etc.

2. Analyze the read data (I can't solve how to do this part)
-> In this part it should sum one day values to one row, so it should look like this:
1.1.2016 358006
2.1.2016 39
3.1.2016 0 etc.

3. The last step is saving the 2. values to a new csv file
This part is just saving the new data from step 2, which is pretty straight forward.
Richard MacCutchan 20-Nov-18 8:53am
   
As you read the data file you need to create a list of the input values. Either a list of lists, or a list of objects of a class. As you create the list you need to convert the text from each line into proper values: DateTime from the date and time, and integer from the total value. You can then add the new information into the totals for each list item (or add new list items). When that is complete you just write out the updated information.
Member 14060132 20-Nov-18 9:05am
   
I have tried to save the values to a list, but I don't know why haven't it worked out. I have the right idea in my head, but can't work aound it. Any tips how to start editing the code please?
Thank you
Richard MacCutchan 20-Nov-18 9:39am
   
Define a class that has a date (or datetime) attribute and an integer attribute. Do not use float types unless you are trying to do complex calculations. The class constructor should convert the first string to a date(time): see 8.1. datetime — Basic date and time types — Python 2.7.15 documentation[^]. It should convert the second number to an integer (use the int() builtin. You then create objects of this class for every record that you read in, and add them to a list, or dictonary. You can then update the values from any other data that you process by searching for an entry with the specified date (and time) and adding to the value. Once you have completed all the updates you can write the data to the new file in .csv (or any other) format, depending on requirements. Don't forget to add a ToString method to your class to make it easy to write the updated data.
Member 14060132 20-Nov-18 17:19pm
   
I edited my code a bit now it calculates the sum and cumulative values. Is it possible to save these values to a new csv file now? The date should be in the first cell and the values in the second

def daily():    with open('MonthData1.csv', 'r') as file1:        read_file = csv.reader(file1, delimiter=';')        delheader = next(read_file)        data = defaultdict(int)        for line in read_file:            valuedata = max(0, sum([convert(i) for i in line[1:5]]))            data[line[0].split()[0]] += valuedata        for key in OrderedDict(sorted(data.items())):            print('{} {}'.format(key, data[key]))        previous_values = []        for key, value in OrderedDict(sorted(data.items())).items():            print('{} {}'.format(key, value + sum(previous_values)))            previous_values.append(value)            daily()
Richard MacCutchan 21-Nov-18 3:31am
   
You write the new data with a csv.writer: see csv — CSV File Reading and Writing — Python 3.7.1 documentation[^].

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100