Login Register






Checking for Duplicates Across Multiple Files filter_list
Author
Message
RE: Checking for Duplicates Across Multiple Files #11
(01-26-2015, 03:46 PM)Eclipse Wrote: That's reassuring.


I was thinking of doing this; writing the lists into .txt files and then iterating through them.

If you are iterating though anything, you are using a list/array. Don't store anything to the HDD... It's pointless and unnecessary. Try my suggested solution first, I'm almost entirely positive it will work perfectly fine. Explore different solutions after testing the first; you're thinking into this too much.

Reply

RE: Checking for Duplicates Across Multiple Files #12
(01-26-2015, 04:03 PM)Dyme Wrote: If you are iterating though anything, you are using a list/array. Don't store anything to the HDD... It's pointless and unnecessary. Try my suggested solution first, I'm almost entirely positive it will work perfectly fine. Explore different solutions after testing the first; you're thinking into this too much.

Code:
import os import os.path files = [] flines = [] sep = input() for fname in os.listdir('.'): if fname[-3:] == 'txt': files.append(fname) for f in files: for line in open(f, 'r').readlines(): if line not in flines: flines.append(line) def splns(lines, n): for i in xrange(0, len(lines), n): yield lines[i: i + n] for index, lines in enumerate(splns(flines, sep)): with open('new/' + str(index) + '.txt', 'w') as f: f.write(''.join(lines))

OffTopic: What do you use to create gifs?

Reply