[sunjester tuts] Combolist Management in C#: Part 1

[sunjester tuts] Combolist Management in C#: Part 1 - Printable Version

+- Sinisterly (https://sinister.li)
+-- Forum: Coding (https://sinister.li/Forum-Coding)
+--- Forum: Visual Basic & .NET Framework (https://sinister.li/Forum-Visual-Basic-NET-Framework)
+--- Thread: [sunjester tuts] Combolist Management in C#: Part 1 (/Thread-sunjester-tuts-Combolist-Management-in-C-Part-1)

[sunjester tuts] Combolist Management in C#: Part 1 - sunjester - 03-12-2019

Introduction
You can follow along by reading the first tutorial about loading/running C# on Linux here. If you have already read that one or know how to do it, please feel free to continue. You can read my other combolist management thread (in bash/shell script) here. All of my tutorials are written under Linux, and I will not pander to the Windows kiddies. I have been developing software since before .NET was around and normally don't even use an IDE, I either use SublimeText, vi, or nano. Nothing wrong with using a full IDE, especially if you're new.

Combo Lists
You can get a shit ton of combo lists from my old github repo (https://github.com/therealsunjester/combos) that has 17 million combos. You can use any of those to practice your skills and use the code presented here. Most of the combo lists people are using are huge. When you are dealing with a large amount of data, reading line by line into a string[] array is just stupid, it will slow your computer down and probably crash your software. The lists in my repo shouldn't have duplicates in them and they should all be a colon delimited format.

We can read a list (the list im using has 8,000 lines) into a generic list:

Code:
public void RemoveDupes(string fileName)
{
    var file = File.ReadAllLines(fileName);
    var lines = new List<string>(file);

In the code above we are reading a file named fileName into a generic string list named lines. Two libraries are needed for this: System.Collections.Generic and System.IO.

Removing Duplicates
When removing duplicates you can use Linq's Distinct method since we are using a generic list to hold the combos. It requires a simple addition to one of our lines of code.

Code:
var lines = new List<string>(file).Distinct();

That's it, you have removed all the duplicates from that generic list. So now that we have remove the duplicates we can write it back out to a new file named "no_dupes". So let's put it all together.

Code:
public void RemoveDupes(string fileName)
{
    var file = File.ReadAllLines(fileName);
    var lines = new List<string>(file).Distinct();
    TextWriter tw = new StreamWriter("no_dupes");

    foreach(string line in lines)
    {
            tw.WriteLine(line);
    }

    tw.Close();
}

Dealing with HUGE Combo Lists
When dealing with large combo lists the best thing to do is read a bit at a time. There is no reason to ever store that much in a single file but if you do (and you know who you are) we can use a temporary file and read a little bit at a time. This will keep your software from locking up and your computer free from dying. This is a 24mb flat file with 800k+ combos https://github.com/therealsunjester/combos/blob/master/815K.txt. This is not something you should load into a string[] or even a generic list all at once.

So what do we do? Chunk it up into smaller sections.

Code:
public void ReadLargeCombo(string fileName, int start, int howMany)
{
    List<string> lines = new List<string>(File.ReadLines(fileName).Skip(start).Take(howMany));
    foreach(string line in lines)
    {
            Console.WriteLine(line);
    }
}

So to use the method we would supply the name of the file, which line to start with and then how many to return from the line we started at, for example:

Code:
combos.ReadLargeCombo("combolist1", 50, 100);

We are starting at line 50 in the file and returning 100 lines.

Conclusion
I am stopping here and going to provide you with the class I've written so far. I changed the ReadLargeCombo method to return the array of lines instead of printing them out on the spot (since that would be bad OOP practice). If you have suggestions for the next tutorials, let me know. If you have problems with your combo lists and would like to have a solution to the problem, let me know, I will add it to the next tutorial.

Code:
class ComboSun
{
        public ComboSun() {}

        public void RemoveDupes(string fileName)
        {
                var file = File.ReadAllLines(fileName);
                var lines = new List<string>(file).Distinct();

                TextWriter tw = new StreamWriter("no_dupes");

                foreach(string line in lines)
                {
                        tw.WriteLine(line);
                }

                tw.Close();
        }

        public List<string> ReadLargeCombo(string fileName, int start, int howMany)
        {
                List<string> lines = new List<string>(File.ReadLines(fileName).Skip(start).Take(howMany));
                return lines;
        }
}

RE: [sunjester tuts] Combolist Management in C#: Part 1 - Vultra - 03-13-2019

Thanks for sharing. I've been personally interested in this so, I'll come back to it once I'm home.

RE: [sunjester tuts] Combolist Management in C#: Part 1 - mothered - 03-13-2019

In terms of removing dupes, I simply use a tool (example, D3vLeecher) which does the job In literally a few seconds.

Excellent guide though, well done. I'll be delving Into the "Dealing with huge combo lists" section shortly.

RE: [sunjester tuts] Combolist Management in C#: Part 1 - sunjester - 03-22-2019

(03-13-2019, 03:07 AM)mothered Wrote: In terms of removing dupes, I simply use a tool (example, D3vLeecher) which does the job In literally a few seconds.

Excellent guide though, well done. I'll be delving Into the "Dealing with huge combo lists" section shortly.

yea this is for people who dont want to keep using other peoples tools Wink

they can start building their own

RE: [sunjester tuts] Combolist Management in C#: Part 1 - mothered - 03-22-2019

(03-22-2019, 04:18 PM)sunjester Wrote:
(03-13-2019, 03:07 AM)mothered Wrote: In terms of removing dupes, I simply use a tool (example, D3vLeecher) which does the job In literally a few seconds.

Excellent guide though, well done. I'll be delving Into the "Dealing with huge combo lists" section shortly.

yea this is for people who dont want to keep using other peoples tools they can start building their own

Agree.

Coding your own tools not only provides a sense of satisfaction, but also the liberty to customize to your heart's content.

RE: [sunjester tuts] Combolist Management in C#: Part 1 - darkninja1980 - 03-24-2019

nice looking tutorial. Smile

thank you for the information.

RE: [sunjester tuts] Combolist Management in C#: Part 1 - Pinkz0rd - 05-06-2019

nice and clean Smile

, do you recommend any cracker source code? Im being interested on learning how to make a cracker/checker!

RE: [sunjester tuts] Combolist Management in C#: Part 1 - sunjester - 05-06-2019

(05-06-2019, 06:57 PM)Pinkz0rd Wrote: nice and clean , do you recommend any cracker source code? Im being interested on learning how to make a cracker/checker!

https://sinister.ly/Thread-Tutorial-sunjester-tuts-Creating-a-GPT-Checker

RE: [sunjester tuts] Combolist Management in C#: Part 1 - Pinkz0rd - 05-07-2019

(05-06-2019, 07:59 PM)sunjester Wrote:
(05-06-2019, 06:57 PM)Pinkz0rd Wrote: nice and clean , do you recommend any cracker source code? Im being interested on learning how to make a cracker/checker!

https://sinister.ly/Thread-Tutorial-sunjester-tuts-Creating-a-GPT-Checker

thanks very much mate Smile