Word List Creator - Printable Version

Word List Creator - Printable Version

+- Sinisterly (https://sinister.li)
+-- Forum: Coding (https://sinister.li/Forum-Coding)
+--- Forum: Java, JVM, & JRE (https://sinister.li/Forum-Java-JVM-JRE)
+--- Thread: Word List Creator (/Thread-Word-List-Creator)

Word List Creator - Solixious - 06-22-2013

I tried making a simple word list generator in java today and came up with this thing. It reads a URL and writes unique words to a file in separate lines if its length is more than or equal to 4.
WLC.java

Code:
/* Made by Solixious Klein */
import javax.swing.*;
import java.awt.event.*;
import java.io.File;
public class WLC extends JFrame implements ActionListener
{
	private static final long serialVersionUID = 1L;
	private JLabel urlLabel,outputFileLabel;
	private JTextField urlField,fileField;
	private JButton browseButton,generateButton,closeButton;
	private JFileChooser saveFileDialog;
	public WLC()
	{
		setLayout(null);
		setTitle("Word List Creator");
		setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
		urlLabel=new JLabel("URL to Scan");
		outputFileLabel=new JLabel("Output File");
		urlField=new JTextField();
		fileField=new JTextField();
		browseButton=new JButton("Browse");
		generateButton=new JButton("Start");
		closeButton=new JButton("Close");
		browseButton.addActionListener(this);
		generateButton.addActionListener(this);
		closeButton.addActionListener(this);
		urlLabel.setBounds(30,30,100,20);
		outputFileLabel.setBounds(30, 60, 100, 20);
		urlField.setBounds(150, 30, 200, 20);
		fileField.setBounds(150, 60, 200, 20);
		browseButton.setBounds(370,60,80,20);
		generateButton.setBounds(150,100,80,20);
		closeButton.setBounds(250,100,80,20);
		add(urlLabel);
		add(outputFileLabel);
		add(urlField);
		add(fileField);
		add(browseButton);
		add(generateButton);
		add(closeButton);
		setBounds(200,200,500,200);
		setVisible(true);
	}
	public void actionPerformed(ActionEvent e)
	{
		Object event=e.getSource();
		if(event==browseButton)
		{
			saveFileDialog=new JFileChooser();
			saveFileDialog.setAcceptAllFileFilterUsed(false);
			saveFileDialog.setFileFilter(new TextFilter());
			int ret=saveFileDialog.showDialog(this,"Save To Text File");
			if(ret==JFileChooser.APPROVE_OPTION)
			{
				String path=saveFileDialog.getSelectedFile().getPath();
				path=path.toLowerCase();
				if(path.endsWith("txt"))
					fileField.setText(path);
				else
					fileField.setText(path+".txt");
			}
		}
		else if(event==generateButton)
		{
			ReadURL r=new ReadURL();
			String urlString=urlField.getText();
			String fileName=fileField.getText();
			if(!urlString.startsWith("http://"))
				urlString="http://"+urlString;
			try
			{
				r.read(urlString, fileName);
			}
			catch(Exception e1)
			{
				e1.printStackTrace();
			}
		}
		else if(event==closeButton)
		{
			dispose();
			System.exit(0);
		}
	}
	public static void main(String[] args)
	{
		new WLC();
	}
	class TextFilter extends javax.swing.filechooser.FileFilter
	{
		public boolean accept(File f)
		{
		    if (f.isDirectory())
		      return true;
		    String s = f.getName();
		    s=s.toLowerCase();
		    if (s.endsWith(".txt"))
		        return true;

		    return false;
		}
		public String getDescription()
		{
		    return "Text Files(*.txt)";
		}
	}
}

ReadURL.java

Code:
import java.net.*;
import java.io.*;
public class ReadURL
{
    public void read(String urlString,String fileName) throws Exception
    {
        URL url = new URL(urlString);
        BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
        String inputLine;
        File f=new File(fileName);
        if(!f.exists())
        	f.createNewFile();
        BufferedWriter bw=new BufferedWriter(new FileWriter(f,true));
        while ((inputLine = in.readLine()) != null)
        {
        	inputLine=strip(inputLine);
        	String st[]=inputLine.split(" ");
        	for(int i=0;i<st.length;i++)
        	{
        		if(st[i].length()>=4) //length should be more than 3
        		{
        			int flag=0;
        			for(int j=0;j<i;j++)
        			{
        				if(st[j].equals(st[i]))
        				{
        					flag=1;
        					break;
        				}
        			}
        			if(flag==0)
        			{
        				bw.write(st[i]);
        				bw.newLine();
        			}
        		}
        	}
        }
        in.close();
        bw.close();
        optimize(fileName); //sort the words in file and also remove the duplicate words
    }
    public void optimize(String fileName)throws Exception
    {
    	BufferedReader br=new BufferedReader(new FileReader(fileName));
    	int ctr=0;
    	while(br.readLine()!=null)
    		ctr++;
    	br.close();
    	br=new BufferedReader(new FileReader(fileName));
    	String words[]=new String[ctr];
    	for(int i=0;i<ctr;i++)
    	{
    		words[i]=br.readLine();
    	}
    	//sorting
    	for(int i=0;i<ctr;i++)	
    	{
    		int min=i;
    		for(int j=i+1;j<ctr-1;j++)
    		{
    			if(words[min].compareTo(words[j])<0)
    				min=j;
    		}
    		if(min!=i)
    		{
    			String temp=words[min];
    			words[min]=words[i];
    			words[i]=temp;
    		}
    	}
    	br.close();
    	BufferedWriter bw=new BufferedWriter(new FileWriter(fileName,false));
    	for(int i=0;i<ctr;i++)
    	{
    		if(i>0)
    		if(words[i].equals(words[i-1]))
    			continue;
    		bw.write(words[i]);
    		bw.newLine();
    	}
    	bw.close();
    }
    public String strip(String str)
    {
    	String st="";
    	for(int i=0;i<str.length();i++)
    	{
    		char c=str.charAt(i);
    		if((c>='a' && c<='z') || (c>='A' && c<='Z'))//consider alphabets only
    			st=st+String.valueOf(c);
    		else if(!st.endsWith(" "))//separate various words with a single blank space
    		{
    			st=st+" ";
    		}
    	}
    	return st;
    }
}

Screen Shot :

Spoiler:

Cheers

RE: Word List Creator - Psycho_Coder - 06-22-2013

Hello Solixious,

That's a nice code. I have tested it and i works quite well. I have some suggestions or better say I have some queries which I think you will help me to clear it out. (Note: I haven't studied much about URL class in Java though the code is pretty understandable)

1. You have used BufferedReader , you would also have have used Scanner class as this takes less space than BufferedReader class, but the reason for which you have used BufferedReader is because Scanner is not thread safe.

2. You have have method optimize which sorts and removes the duplicates from the file, instead of that long method you could have stored the words in a HashSet as Set class doesn't stores the duplicates.

Code:
Set<String> st = new HashSet<>();

Also you have used selection sort to sort the words in the file but you could have used the Collections class as :

Code:
Set<String> st=new HashSet<String>();
Collections.sort(new ArrayList<String>(st));

I think the above would have made your method really optimized and fast than the present

Also in you WLC.java you should add @Override to accept() and getDescription() method.

Do clear out my doubts!

Thank you,
Sincerely,
Psycho_Coder

RE: Word List Creator - Solixious - 06-22-2013

(06-22-2013, 05:46 PM)Psycho_Coder Wrote: 1. You have used BufferedReader , you would also have have used Scanner class as this takes less space than BufferedReader class, but the reason for which you have used BufferedReader is because Scanner is not thread safe.

It is just a personal decision. Their are more classes I could use, but I'm more comfortable with this one.

(06-22-2013, 05:46 PM)Psycho_Coder Wrote: 2. You have have method optimize which sorts and removes the duplicates from the file, instead of that long method you could have stored the words in a HashSet as Set class doesn't stores the duplicates.

Code:
Set<String> st = new HashSet<>();

Now that you mentioned it, I guess I could have used it to make it a little easy for me. But in the end, it would mean about the same since a similar processing is done in that very class to remove duplicates. Still, I think that would be a lot easier for me Biggrin

(06-22-2013, 05:46 PM)Psycho_Coder Wrote: Also you have used selection sort to sort the words in the file but you could have used the Collections class as :

Code:
Set<String> st=new HashSet<String>(); Collections.sort(new ArrayList<String>(st));

I think the above would have made your method really optimized and fast than the present

In the present case, I should say that using a Bubble Sort would make it a little faster since it would work mainly on an already sorted array with a few additions. using it the way you suggested would nearly mean the same thing except for the reduction of lines of codes..

(06-22-2013, 05:46 PM)Psycho_Coder Wrote: Also in you WLC.java you should add @Override to accept() and getDescription() method.

Why would I do that? Isn't it unnecessary?

RE: Word List Creator - Psycho_Coder - 06-22-2013

@Solixious I think using the in-built classes would be more efficient as for those class the memory management has been done in an efficient manner by the compiler already actually its predefined. Also since the words are not very large and they are almost in a sorted manner (considering avearge case), insertion sort would be far more effective and efficient.

I think using Set would make the code better optimized however it your code and as long as you enjoy it its good. But I would have gone for the things that I told as I am too lazy to write a lot when things can be done with a little ease :tongue:

Also if this code is taken from the point of view that you will get experience if you write the methods by yourself then I think you are experienced enough for these. If those long codes are taken with the POV of a reader who is a beginner in Java then I think its better to write the complete working methods.

A suggestion if possible give a snapshot of your GUI and output side by side.

Thank you,
Sincerely,
Psycho_Coder

RE: Word List Creator - Deque - 06-22-2013

That's a nice and useful idea you implemented here.

Quote:1. You have used BufferedReader , you would also have have used Scanner class as this takes less space than BufferedReader class, but the reason for which you have used BufferedReader is because Scanner is not thread safe.

A scanner is used to create tokens from a text. It's not mainly used for reading Files (but it has the functionality). The scanner has more functionality than you need there, so I would prefer the BufferedReader. It makes your intentions (you don't want to tokenize) more clear.

Quote:Now that you mentioned it, I guess I could have used it to make it a little easy for me. But in the end, it would mean about the same since a similar processing is done in that very class to remove duplicates. Still, I think that would be a lot easier for me

The second suggestion of @Psycho_Coder is a very good one and I would have done the same.
No, it is not the same process and also not a similar one that is used in a set. A HashSet has a much better performance. It uses hashing to store the values, therefor it can immediately (O(1) for the average case) determine if a value has been stored already.
You on the other hand read the values, write the values, read the values, sort the values and write them again. The sorting is O(n^2) which is much worse. When it comes to reading large files, you might get into trouble soon.

Quote:Why would I do that? Isn't it unnecessary?

Actually it should be mandatory like it is in C#, but Java is older and they build in the @Override annotations later, while preserving the backward compatibilty, so they couldn't make it mandatory.
It is a bad habit not using these annotations, they prevent bugs (i.e. spelling error when overriding methods) and they make the code more readable. Btw: If you use Eclipse you can tell it to add the override annotation on save operations.

Other than that:

Close your streams within a finally block.

Start your GUI (WLC) like this or you get into trouble, because Swing isn't thread save:

Code:
SwingUtilities.invokeLater(new Runnable() {
    public void run() {
        new WLC();
    }
});

RE: Word List Creator - Solixious - 06-22-2013

@Deque : Thank you for taking time to go over my code and suggesting things for it. I'll keep that in mind. Smile

@Psycho_Coder : I've added a screen shot of the running code and output. Even though it isn't very pretty to look at Biggrin

RE: Word List Creator - Psycho_Coder - 06-22-2013

Oops Sorry Solixious I forgot to answer that override question which Deque had answered, I just over looked that part.

@Deque I had made a similar mistake in my Pong game in java where I forgot to use the annotations and then you said to include them and I remembered and hence suggested him to do so. Thanks you for teaching to me then Confused

miling:

(06-22-2013, 07:07 PM)Solixious Wrote: @Deque : Thank you for taking time to go over my code and suggesting things for it. I'll keep that in mind.

@Psycho_Coder : I've added a screen shot of the running code and output. Even though it isn't very pretty to look at

Thank you for accepting my suggestion. Would you mind if you put it in a spoiler as it take a big part of the screen.

Looking pretty is not an issue as those who want to learn they won't care for design it should be readable enough for the reader to understand.

Thank you,
Sincerely,
Psycho_Coder