输入URL 将整个html内容保存到指定文件
package parser;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
/**
* 基本能实现网页抓取，不过要手动输入URL 将整个html内容保存到指定文件
*
*@author chenguoyong
*
*/
public class ScrubSelectedWeb {
privatefinal static String CRLF = System.getProperty("line.separator");
/**
* @param args
*/
publicstatic void main(String[] args) {
try{
URLur = newURL("http://10.249.187.199:8083/injs100/");
InputStreaminstr = ur.openStream();
Strings, str;
BufferedReaderin = new BufferedReader(new InputStreamReader(instr));
StringBuffersb = new StringBuffer();
BufferedWriterout = new BufferedWriter(new FileWriter(
"D:/outPut.txt"));
while((s = in.readLine()) != null) {
sb.append(s+ CRLF);
}
System.out.println(sb);
str= new String(sb);
out.write(str);
out.close();
in.close();
}catch (MalformedURLException e) {
e.printStackTrace();
}catch (IOException e) {
e.printStackTrace();
}
}
}
基本能实现网页抓取，不过要手动输入URL，此外没有重构。只是一个简单的思路。
1.htmlparser 使用
htmlparser是一个纯的java写的html解析的库，htmlparser不依赖于其它的java库，htmlparser主要用于改造或提取html。htmlparser能超高速解析html，而且不会出错。毫不夸张地说，htmlparser就是目前最好的html解析和分析的工具。无论你是想抓取网页数据还是改造html的内容，用了htmlparser绝对会忍不住称赞。由于htmlparser 结构设计精良，所以扩展htmlparser 非常便利。
http://c.tieba.baidu.com/p/3316726163
http://c.tieba.baidu.com/p/3316723845
http://c.tieba.baidu.com/p/3316722567
http://c.tieba.baidu.com/p/3316721327
http://c.tieba.baidu.com/p/3316717504
http://c.tieba.baidu.com/p/3316714975
http://c.tieba.baidu.com/p/3316710876
http://c.tieba.baidu.com/p/3316692502
http://c.tieba.baidu.com/p/3316689008
http://c.tieba.baidu.com/p/3316687706
http://c.tieba.baidu.com/p/3316750701
http://c.tieba.baidu.com/p/3316760692
http://c.tieba.baidu.com/p/3316762691
http://c.tieba.baidu.com/p/3316780765
http://c.tieba.baidu.com/p/3316781850
http://c.tieba.baidu.com/p/3316787592
http://c.tieba.baidu.com/p/3316798631
http://c.tieba.baidu.com/p/3316804467
http://c.tieba.baidu.com/p/3316806665
http://c.tieba.baidu.com/p/3316811332
http://c.tieba.baidu.com/p/3316828201
http://c.tieba.baidu.com/p/3316826791
http://c.tieba.baidu.com/p/3311944721
http://c.tieba.baidu.com/p/3311943490
http://c.tieba.baidu.com/p/3311943062
http://c.tieba.baidu.com/p/3305095344
http://c.tieba.baidu.com/p/3305097954
http://c.tieba.baidu.com/p/3305100697
http://c.tieba.baidu.com/p/3305103600
http://c.tieba.baidu.com/p/3305105795
http://c.tieba.baidu.com/p/3305110305
http://c.tieba.baidu.com/p/3305112079
http://c.tieba.baidu.com/p/3305115018
http://c.tieba.baidu.com/p/3305117117
http://c.tieba.baidu.com/p/3305118990
http://c.tieba.baidu.com/p/3305123204
http://c.tieba.baidu.com/p/3305123924
http://c.tieba.baidu.com/p/3305124673
http://c.tieba.baidu.com/p/3305130305
http://c.tieba.baidu.com/p/3305136460
http://c.tieba.baidu.com/p/3305140204
http://c.tieba.baidu.com/p/3316925465
http://c.tieba.baidu.com/p/3317149335
http://c.tieba.baidu.com/p/3317148112
http://c.tieba.baidu.com/p/3317146582
http://c.tieba.baidu.com/p/3317151995
http://c.tieba.baidu.com/p/3287967193
http://c.tieba.baidu.com/p/3317242653
http://c.tieba.baidu.com/p/3317244575
http://c.tieba.baidu.com/p/3317242653
http://c.tieba.baidu.com/p/3317247843
http://c.tieba.baidu.com/p/3317248495
http://c.tieba.baidu.com/p/3317251825
http://c.tieba.baidu.com/p/3317253337
http://c.tieba.baidu.com/p/3317253840
http://c.tieba.baidu.com/p/3317146582
http://c.tieba.baidu.com/p/3317148112
http://c.tieba.baidu.com/p/3317149335
http://c.tieba.baidu.com/p/3317151995
http://c.tieba.baidu.com/p/3317176379
http://c.tieba.baidu.com/p/3317177568
http://c.tieba.baidu.com/p/3317178811
http://c.tieba.baidu.com/p/3317192065
http://c.tieba.baidu.com/p/3317193734
http://c.tieba.baidu.com/p/3317195526
http://c.tieba.baidu.com/p/3317213453
http://c.tieba.baidu.com/p/3317218881
http://c.tieba.baidu.com/p/3317220460
http://c.tieba.baidu.com/p/3317221802
http://c.tieba.baidu.com/p/3317264965
http://c.tieba.baidu.com/p/3317266739
http://c.tieba.baidu.com/p/3317292343
http://c.tieba.baidu.com/p/3317302135
http://c.tieba.baidu.com/p/3317301165
http://c.tieba.baidu.com/p/3316890365
http://c.tieba.baidu.com/p/3305128709
http://c.tieba.baidu.com/p/3305138616
http://c.tieba.baidu.com/p/3305142732
http://c.tieba.baidu.com/p/3317222864
http://c.tieba.baidu.com/p/3317334635
http://c.tieba.baidu.com/p/3317335086
http://c.tieba.baidu.com/p/3317335730
http://c.tieba.baidu.com/p/3317336186
http://c.tieba.baidu.com/p/3317337213
http://c.tieba.baidu.com/p/3317337665
http://c.tieba.baidu.com/p/3317338031
http://c.tieba.baidu.com/p/3317338563
http://c.tieba.baidu.com/p/3317339537
http://c.tieba.baidu.com/p/3317341138
http://c.tieba.baidu.com/p/3317341879
http://c.tieba.baidu.com/p/3317342204
http://c.tieba.baidu.com/p/3317342973
http://c.tieba.baidu.com/p/3317343387
http://c.tieba.baidu.com/p/3317345377
http://c.tieba.baidu.com/p/3317345704
http://c.tieba.baidu.com/p/3317346163
http://c.tieba.baidu.com/p/3317347026
http://c.tieba.baidu.com/p/3317347678
http://c.tieba.baidu.com/p/3317351042
http://c.tieba.baidu.com/p/3317351639
http://c.tieba.baidu.com/p/3317351921
http://c.tieba.baidu.com/p/3317352257
http://c.tieba.baidu.com/p/3317355970
http://c.tieba.baidu.com/p/3317356202
http://c.tieba.baidu.com/p/3317356465
http://c.tieba.baidu.com/p/3317356701
http://c.tieba.baidu.com/p/3317356930
http://c.tieba.baidu.com/p/3317358153
http://c.tieba.baidu.com/p/3317358297
http://c.tieba.baidu.com/p/3317358544
http://c.tieba.baidu.com/p/3317358737
http://c.tieba.baidu.com/p/3317358972
http://c.tieba.baidu.com/p/3317352535
http://c.tieba.baidu.com/p/3317384138
http://c.tieba.baidu.com/p/3317384839
http://c.tieba.baidu.com/p/3317385738
http://c.tieba.baidu.com/p/3317386332
http://c.tieba.baidu.com/p/3317387505
http://c.tieba.baidu.com/p/3317386978
http://c.tieba.baidu.com/p/3317387505
http://c.tieba.baidu.com/p/3317388068
http://c.tieba.baidu.com/p/3317389331
http://c.tieba.baidu.com/p/3317389987
http://c.tieba.baidu.com/p/3317390834
http://c.tieba.baidu.com/p/3317392303
http://c.tieba.baidu.com/p/3317392786
http://c.tieba.baidu.com/p/3317396474
http://c.tieba.baidu.com/p/3317407676
http://c.tieba.baidu.com/p/3317409934
http://c.tieba.baidu.com/p/3317419426
http://c.tieba.baidu.com/p/3317429015
http://c.tieba.baidu.com/p/3317422129
http://c.tieba.baidu.com/p/3317416171
http://c.tieba.baidu.com/p/3317414267
http://c.tieba.baidu.com/p/3317412472
http://c.tieba.baidu.com/p/3317411779
郑重声明：本站内容如果来自互联网及其他传播媒体，其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享，并不代表本站赞同其观点和对其真实性负责，也不构成任何其他建议。
