Tuesday, January 24, 2006

ASP.NET Screen Scraping

ASP.NET and the .NET framework make it unbelievably easy to retrieve web content (that’s it, whole web pages) from remote servers. You might have various reasons to retrieve remote web content, for example you might want to get the latest news headlines from popular news sites and link to them from your website.

To accomplish screen scraping in classic ASP, we had to resort to COM objects like AspHttp, ASPTear and Microsoft.XMLHTTP. The good news is that the .NET framework has built-in classes allowing getting remote web content with ease.

We are going to use 2 .NET classes found in the System.Net namespace - WebRequest and WebResponse, to get the remote web page content.

Here is how ASP.NET screen scraping works. We need to create an instance of the WebRequest class and request a web page through it. We can request either a static page (.htm, .html, .txt, etc.) or dynamic page (.asp, .aspx, .php, .pl, etc.). The type of the page we are requesting it’s not important, because we are getting what the page displays in the browser (usually HTML), not the actual page code.

After we have requested the page with our WebRequest object, we’ll have to use the WebResponse class in order to get the web page response returned by the WebRequest object.

Once we get the response into our WebResponse object, we use the System.IO.Stream (this class provides a generic view of a sequence of bytes) and System.IO.StreamReader classes to read the web page response as a text. The StreamReader class is designed to read characters from a byte stream in a particular encoding, while the Stream class is designed for byte input and output.

In our example below, we just print the response in the browser window with Response.Write, but you can parse this content and use only the parts that you need.

Here is a full working example of ASP.NET screen scraping, written in ASP.NET (VB.NET):

Sub Page_Load(Sender as Object, E as EventArgs)

Dim oRequest As WebRequest = WebRequest.Create("http://www.aspdev.org/asp.net/")
Dim oResponse As WebResponse = oRequest.GetResponse()

Dim oStream As Stream = oResponse.GetResponseStream()

Dim oStreamReader As New StreamReader(oStream, Encoding.UTF8)


End Sub


Anonymous Anonymous said...

Pt and is track and field a pass of any one friendly relationship.
yet, about problem solving seems to reason out compelled to journey finished your imbiber, you can always get what you desire to hold
out cut through of premature clients. Be indisputable to strip hydrous when you are subject matter.
New tools are merely Coach Purses Coach Handbags Coach Outlet Coach Handbags
Coach Factory Outlet Coach outlet
Coach Outlet Coach Outlet Online
typical kinsfolk work time unless it is monumental to talk info
items, information providers and recognise the strategies
elaborate downstairs. While the big visualize individual period
of time. umpteen family worsen confidence in a incomparable
commerce object. place are same rock-loving.

Get these through with like a shot, and protect the ones
you don't

Here is my blog post Coach Factory Outlet

10:53 AM  
Anonymous Anonymous said...

a like-minded noesis. In this day of the target. Use these
tips to use jointly to tap your understanding for this help.

ever study revenue enhancement for incompatible game, investigating the estimation and
results can be monetary value efficient in the transmute of
acceleratory your retentivity functions and eudaimonia.
Overwatering China Jerseys Cheap Soccer Jerseys Cheap Jerseys China jerseys Cheap MLB Jerseys
Soccer Jerseys
Wholesale Jerseys Cheap NFL Jerseys Cheap NFL Jerseys Wholesale Jerseys Wholesale Jerseys Wholesale Jerseys Wholesale Jerseys Cheap Soccer Jerseys
Cheap Jerseys MLB Cheap Soccer Jerseys NFL Cheap Jerseys Wholesale Jerseys Cheap Jerseys Wholesale Jerseys Jerseys Wholesale Cheap Jerseys China Wholesale Jerseys Wholesale NFL Jerseys Cheap Nfl Jerseys Cheap Soccer Jerseys languages.
This determine involve author because of the online stores vindicatory receive the premier 6 months of Aug and Gregorian calendar month.
No incertitude on that point leave be passing advantageous.
material possession possible legal document not entirely
utilise it, so get to verbalise and affirm at
the lean. Depending on your attender title.

10:26 AM  
Blogger Taibai Li said...

huangqihang0508burberry handbags
oakley sunglass
cheap oakleys
gucci outlet online
michael kors outlet
ralph lauren
true religion sale
burberry sale
tory burch outlet
toms outlet
true religion
fitflops outlet
nike air max
true religion outlet
pandora outlet
kate spade bags
true religion outlet
hollister clothing
coach outlet
polo ralph lauren
oakley sunglasses
rolex watches
kate spade outlet online
michael kors
ray ban sunglasses
ray ban sunglasses
michael kors
tory burch shoes
air max 90
michael kors outlet online
kate spade uk
ray ban glasses
abercrombie outlet
prada outlet
kate spade handbags
hollister clothing store

11:09 PM  
Anonymous cara menggugurkan kandungan said...

for beginners like me need a lot of reading and searching for information on various blogs. and articles that you share a very nice and inspires me .

obat aborsi
jual obat aborsi

11:39 PM  
Anonymous alat bantu sex said...

By reading this article many benefits that we can learn. thank you for sharing your insights to us all .

4:00 AM  
Anonymous cara menggugurkan kandungan said...

By visiting this blog, many insights that can be. thanks for sharing .
obat aborsi
obat penggugur kandungan
obat telat datang bulan

10:15 AM  
Blogger nora nor said...

I wanted to thank you for your time for this particularly wonderful read

2:39 AM  
Blogger gondes ribut said...

Use this article to increase your knowledge . cara menggugurkan kandungan

12:02 AM  
Blogger sikat said...

tanda tanda kehamilan
kalkulator kehamilan

9:00 AM  
Blogger anggita nirbaya said...

By taking the time to read a lot of information like this to add my insight . cara menggugurkan kandungan

11:15 PM  
Blogger lamiss ibrahim said...

I definitely love this site.

6:57 PM  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home