+- Sinisterly (https://sinister.li)
+-- Forum: Coding (https://sinister.li/Forum-Coding)
+--- Forum: Python (https://sinister.li/Forum-Python)
+--- Thread: Checking for Duplicates Across Multiple Files (/Thread-Checking-for-Duplicates-Across-Multiple-Files)
Pages:
1
2
RE: Checking for Duplicates Across Multiple Files - Dyme - 01-26-2015
(01-26-2015, 03:46 PM)Eclipse Wrote: That's reassuring.
I was thinking of doing this; writing the lists into .txt files and then iterating through them.
If you are iterating though anything, you are using a list/array. Don't store anything to the HDD... It's pointless and unnecessary. Try my suggested solution first, I'm almost entirely positive it will work perfectly fine. Explore different solutions after testing the first; you're thinking into this too much.
RE: Checking for Duplicates Across Multiple Files - Eclipse - 02-01-2015
(01-26-2015, 04:03 PM)Dyme Wrote: If you are iterating though anything, you are using a list/array. Don't store anything to the HDD... It's pointless and unnecessary. Try my suggested solution first, I'm almost entirely positive it will work perfectly fine. Explore different solutions after testing the first; you're thinking into this too much.
Code:
import os
import os.path
files = []
flines = []
sep = input()
for fname in os.listdir('.'):
if fname[-3:] == 'txt':
files.append(fname)
for f in files:
for line in open(f, 'r').readlines():
if line not in flines:
flines.append(line)
def splns(lines, n):
for i in xrange(0, len(lines), n):
yield lines[i: i + n]
for index, lines in enumerate(splns(flines, sep)):
with open('new/' + str(index) + '.txt', 'w') as f:
f.write(''.join(lines))OffTopic: What do you use to create gifs?