• Interpret HTTP requests
• Use the .NET Framework to make HTTP requests without a browser
• Interpret HTML
• Work with IIS
• Produce dynamic Web content without using ASP.NET yet

This tutorial covers the fundamentals of building a Web-based application. Unlike the development of most desktop applications, in which most of the parts are available locally (as components on the user's hard disk drive), developing a Web application requires getting software parts to work over a widely distributed network using a disconnected protocol. The technologies underlying ASP.NET have been around for a long time. Of course ASP.NET makes use of this technology underneath, while making it very approachable at the same time.

After completing this tutorial, you will be able to

• Interpret HTTP requests

• Use the .NET Framework to make HTTP requests without a browser

• Interpret HTML

• Work with IIS

• Produce dynamic Web content without using ASP.NET yet

This tutorial covers the fundamentals of building a Web-based application. Unlike the development of most desktop applications, in which most of the parts are available locally (as components on the user's hard disk drive), developing a Web application requires getting software parts to work over a widely distributed network using a disconnected protocol. The technologies underlying ASP.NET have been around for a long time. Of course ASP.NET makes use of this technology underneath, while making it very approachable at the same time.

Although ASP.NET makes developing Web applications far easier than ever before, you must have a solid understanding of how the plumbing is actually working during the development of an ASP.NET application. A good example of such a time might be when you're tracking down a stray HTTP request or trying to figure out why a section of your page is appearing in the wrong font within a client's browser. Another such time might occur while you're writing a custom control for a Web page. Custom controls often require that the rendering code be written manually. That is, you must carefully ensure that the HTML tags emitted by your control occur in exactly the right order. For that, you need to understand HTML.

This tutorial covers three things necessary to allow you to work with ASP.NET:

• How HTTP requests work

• How HTML works

• How HTTP requests are handled on the Microsoft platform (IIS)

Understanding these three technologies underlying ASP.NET frames the rest of the system. As you study ASP.NET these pieces will undoubtedly fall into place.

The communication mechanism by which Web browsers talk to Web sites is named the HyperText Transfer Protocol (HTTP). The World Wide Web as we know it today began as a research project at CERN in Switzerland. In those days, the notion of hypertext¡ªdocuments linked together arbitrarily¡ªwas becoming increasingly popular. Applications such as Hypercard from Apple Computer Inc. introduced hypertext applications to a wider audience. Now, if documents could be linked over a network, that would revolutionize publishing. That's the reason for the HyperText Transfer Protocol, which lies on top of TCP/IP as an application layer.

In its original form, HTTP was meant to transfer hypertext documents. That is, it was originally intended simply to link documents together without consideration for anything like the Web-based user interfaces that are the staple of modern Web sites. The earliest versions of HTTP supported a single GET request to fetch the named resource. It then became the server's job to send the file as a stream of text. After the response arrived at the client's browser, the connection terminated. The earliest versions of HTTP supported only transfer of text streams and did not support any other sort of data transfer.

The first formal specification for HTTP found itself in version 1.0 and was published in the mid-1990s. HTTP 1.0 added support for more complex messaging beyond a simple text transfer protocol. HTTP grew to support different media (specified by the Multipurpose Internet Mail Extensions). The current version of HTTP is version 1.1.

As a connection protocol, HTTP is built around several basic commands. The most important ones we'll see in developing ASP.NET applications are GET, HEAD, and POST.

GET retrieves the information identified by the Uniform Resource Identifier (URI) specified by the request. The HEAD command retrieves only the header information identified by the Uniform Resource Identifier specified by the request (that is, it does not return a message body). You use the POST method to make a request to the server that may cause side effects. You make most initial contacts to a page using a GET command, and you handle subsequent interactions with POST commands.

HTTP Requests from a Browser

As an example, look at the request that is sent from a browser to fetch the helloworld.htm resource from the virtual directory ASPNETStepByStep running on localhost. Listing 1-1 shows the text of the request that is sent to the server. If you would like to see the data going back and forth, there are several TCP monitors available. In addition, you may use TELNET to send GET Requests to the server. Just look for some online. Most are very easy to use.

To send an HTTP request to a server using TELNET, follow these steps:

1. Open the Visual Studio command prompt to connect to your own PC over port 80.

2. At the prompt, type the following:

   C:\>TELNET localhost 80

3. After the TELNET client connects, type the following GET command (assuming you have a virtual directory named ASPNETStepByStep on you machine, containing a file named HelloWorld.HTM):

   C:/> GET /ASPNETStepByStep/helloworld.htm

4. You should see the file's contents returned to the command line.

Listing 1-1

GET /ASPNETStepByStep/helloworld.htm HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, 
application/vnd.ms-powerpoint, application/vnd.ms-excel, 
application/msword, application/x-shockwave-flash, */*
Accept-Language: en-us 
Accept-Encoding: gzip, deflate 
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 
2.0.50215)
Host: localhost:80
Connection: Keep-Alive

When a browser wants to make an HTTP request, it needs to cook up the HTTP request including the URI along with other information. The header information in the request includes details about the operating environment of the browser and some other information that is often useful to the server. When the server receives this request, it returns the requested resource as a text stream. The browser then parses it and formats the contents. Listing 1-2 shows the response provided by the server when asked for the HelloWorld.htm file. Normally, you don't see all the header information when viewing the resource through a browser. A good TCP tracing utility will show it to you. When we look at ASP.NET's tracing facilities later on, this header information will be visible.

Listing 1-2

HTTP/1.1 200 OK
Server: Microsoft-IIS/5.1
X-Powered-By: ASP.NET 
Date: Wed, 01 Jun 2005 23:44:04 GMT
Content-Type: text/html
Accept-Ranges: bytes
Last-Modified: Sun, 22 May 2005 21:54:20 GMT
ETag: "04e9ace185fc51:bb6"
Content-Length: 130
<html>
   <body>
      <h1> Hello World
</h1>
      Nothing really showing here yet, except some HTML...
   </body>
</html>

The first line indicates the protocol and the return code. The rest of the response (until the first <html> tag) is information about the time of the request, the last time the file was modified, and what kind of content is provided. This information will be useful later when we examine such issues as page caching and detecting browser capabilities. The content following the response header information is literally the HTML file sent back by the server.

Making HTTP Requests Without a Browser

In addition to being a framework for building Web applications, the .NET development environment includes classes for making HTTP requests in the raw. The WebRequest class includes a member named GetResponse that will send a request to the address specified by the Uniform Resource Locater. To get a flavor as to how to make direct requests to a Web server without a browser, try compiling and then running this short program that fetches the home page for Microsoft.com.

Build a Simple HTTP Requestor

1. Start Visual Studio.NET. Select New | Project from the main menu. In the New Project dialog box, select a Console application and name it WebRequestorApp, as shown below.

   

   Visual Studio will generate a blank Console program for you.

2. Add the code necessary to make a Web request to the program. Visual Studio places the entry point of the Console application into a file named Program.cs. (This file is the code that shows up in the code window by default.) The code for making a Web request is shown in bold in the following lines of code:

   using System;
   using System.Collections.Generic;
   using System.Text;
   using System.Net;
   using System.IO;
   namespace WebRequestorApp
   {
      class Program
      {
         static void Main(string[] args)
         {
         {
            WebRequest req = 
                  WebRequest.Create
                     ("http://www.microsoft.com");
            WebResponse resp = req.GetResponse();
            StreamReader reader =
               new StreamReader(resp.GetResponseStream(), 
                    Encoding.ASCII);
            Console.WriteLine(reader.ReadToEnd());
        }
     }
   }

3. Run the application. You may do this by choosing Debug | Start Without Debugging from the main menu. Visual Studio will start up a Console for you and run the program. After a couple of moments, you'll see the following HTML spewed to your screen.

Of course, the HTML isn't meant for human consumption. That's what a browser is for. However, this example does show the fundamentals of making a Web request¡ªand you can see exactly what comes back in the response.

In this case, the request sent to the server is much smaller. WebRequest.GetResponse doesn't include as much information in the request¡ªjust the requisite GET followed by the URI, host information, and connection type:

GET /ASPNETStepByStep/helloworld.htm HTTP/1.1
Host: localhost:80
Connection: Keep-Alive

The fundamental jobs of most browsers are (1) to package a request and send it to the server represented in the URI and (2) to receive the response from the server and render it in a useful way. The response usually comes back as a text stream marked up using HTML tags.

In the course of looking at ASP.NET, we'll see quite a bit of HTML. Most of it will be generated by the ASP.NET server-side controls. However, it's important to understand HTML because you may want to write your own server-side control from scratch, and at times you may need to tweak or debug the output of your ASP.NET application.

Most HTTP requests result in a stream of text coming back to the caller. The world has pretty much agreed that HTML is the language for formatting documents, and most browsers understand HTML.

The first release of HTML worth using was version 2.0. Version 3.2 included many new features, such as tables, applets, text flow around images, and superscripts and subscripts, while providing backwards compatibility with the existing HTML 2.0 Standard.

The bottom line is that given a competent browser and well-structured HTML, you had the beginnings of a user interface development technology. And because HTML was understood by browsers running on a variety of platforms, the door was open for implementing a worldwide interactive computing platform. The other key that made this happen (besides a mature version of HTML) was the ability of servers to adapt their output to accommodate the requests of specific users at runtime.

For example, the HTML stream shown in Listing 1-3 will render an HTML page containing a button and a combo box filled with options. (This file is named SelectNoForm.htm in the collection of examples for this tutorial.)

Listing 1-3

<html> 
 <body>
   <h2>Hello there. What's your favorite .NET feature?</h2>
   <select name='Feature'>
    <option> Type-Safety</option>
    <option> Garbage collection</option>
    <option> Multiple syntaxes</option>
    <option> Code Access Security</option>
    <option> Simpler threading</option>
    <option> Versioning purgatory</option>
   </select>
   </br>
   <input type=submit name='Lookup' value='Lookup'></input>
  </br>
 </body>
</html>

See Figure 1-1 for an example of how the page looks when rendered by the browser.

Figure 1-1 A simple HTML page showing a selection tag (rendered here as a Windows combo box) and a Submission button.

This is a static page. Even though it has a combo box and a button, they don't do anything worthwhile. You can pull down the combo box and work with it inside the browser. You can push the button, but all the action happens locally. That's because the server on the other end needs to support dynamic content.

The earliest Web sites were built primarily using static HTML pages. That is, you could surf to some page somewhere and read the HTML document living there. While at that time being able to do this was pretty amazing, HTML eventually evolved to be capable of much more than simply formatting text.

For example, HTML includes tags such as <select></select> that most browsers interpret as a combo box. Various attributes applied to the <input></input> tag causes browsers to draw text boxes and buttons.

HTML Forms

HTML includes a <form></form> tag for notifying the browser that a section of HTML includes tags representing controls. This is how you specify a Web document will be handling input from the end user (not just output).

The <form> tag usually sandwiches a set of tags specifying controls. Listing 1-4 shows the feature selection page, but with the form tag added (the file name is SelectFeature2.htm in the accompanying examples):

Listing 1-4

<html>
<body>
<form action="http://localhost/HttpHandlers/selectfeature.htm" 
     method="get">
<h2>Hello there. What's your favorite .NET feature?</h2>
<select name='Feature'>
<option> Type-Safety</option>
<option> Garbage collection</option>
<option> Multiple syntaxes</option>
<option> Code Access Security</option>
<option> Simpler threading</option>
<option> Versioning purgatory</option>
</select>
</br>
<input type=submit name='Lookup' value='Lookup'></input>
</br>
</form>
</body>
</html>

The form tag includes several attributes that you may set to control how the page behaves. In the above example, notice the <form> tag sets the ACTION attribute, which points back to the server that will receive the form's contents. In its absence, the current document URL will be used.

The other attribute used in the Listing 1-4 is the method attribute. The method attribute specifies the HTTP method used when submitting the form. The method employed in the example is GET because the software on the server doesn't understand POST yet. GET causes the form contents to be appended to the URL. POST causes the form contents to be sent to the server in a data body.

Adding the form tag to the body of the document gets us part of the way to having a workable HTTP application. Now we need a little more support on the server end. When you click the Lookup button, the browser will actually force another round-trip back to the server (though it will only perform an HTTP GET command to refetch the document).

At this point, a normal HTTP GET command will only return the document. For a truly interactive environment, the server on the other end needs to modify the content as requests go back and forth between the browser and the server.

For example, imagine the user does an initial GET for the resource, then selects a features from the combo box and clicks the Submit button. For an interactive application to work, the browser will need to make a second round-trip to the server with a new request. The server will need to examine the request coming from the browser and figure out what to do about it.

Common Gateway Interface

The earliest Web servers supporting ¡°dynamic Web content¡± did so through the Common Gateway Interface (CGI). CGI was the earliest standard for building Web servers. CGI programs execute in real time and change their output based on the state of the application and the requests coming in. Each request coming into a Web server running CGI runs a separate instance of a program to respond. The application can run any sort of operation, including looking up data in a database, accepting credit card numbers, and sending out formatted information.

The Microsoft Platform as a Web Server

On the Microsoft platform, it's too expensive to start up a new process for each request (¨¤ la CGI). Microsoft's solution is to have a single daemon process watch port 80 and load DLLs to handle separate requests when the content needs to change. Microsoft's standard Web platform is based on the Internet Information Services (IIS).

All Web application environments work fundamentally the same way. No matter what hardware/software platform you use, some piece of software is required on the server to watch port 80 for incoming HTTP requests. When a request arrives, it's the server's job to somehow respond to the request in a meaningful way. On the Microsoft platform, Internet Information Services is the watchdog intercepting HTTP requests from port 80¡ªthe normal inbound port for HTTP requests. Internet servers use other ports as well. For example, HTTPS (Secure HTTP) happens over port 443. However, right now we're mostly interested in normal Internet traffic over port 80.

When a browser makes a call to a server running on the Microsoft platform, IIS intercepts that request and searches for the resource identified by the URL. IIS divides its directory space into manageable chunks called virtual directories. For example, imagine someone tries to get to a resource on your server using this URL:

http://www.aspnetstepbystep.com/examples/showfeatures.htm

The domain ¡°aspnetstepbystep¡± is fictitious and used here for illustration. However, if there were a server registered using this name, the URL would identify the entire resource. Within this URL, http://www.aspnetstepbystep.com identifies the server and will direct the request through a maze of routers. Once the request reaches the server, the server will look for the showfeatures.htm resource in some directory-type entity named examples. If the server is running IIS, examples refers to a virtual directory.

IIS divides its working space into multiple virtual directories. Each virtual directory typically refers to a single application. That way, IIS can serve multiple applications. Each virtual directory includes various configuration properties, including such things as security options, error handling redirections, and application isolation options (among others). The configuration parameters also include mappings between file extensions and ISAPI DLLs.

Internet Services Application Programming Interface DLLs

On the Microsoft platform, creating a process space is an expensive proposition (in terms of system resources and clock cycles). Imagine trying to write a server that responds to each request by starting a separate program. The poor server would be bogged down very quickly, and your e-commerce site would stop making money.

Microsoft's architecture prefers using DLLs to respond to requests. DLLs are relatively inexpensive to load, and running code within a DLL executes very quickly. The DLLs handling Web requests are named ISAPI DLLs (ISAPI stands for Internet Services Application Programming Interface).

While we won't dive all the way into the inner workings of ISAPI DLLs, we'll take a cursory look at their architecture so you can see how they relate to ASP.NET.

ISAPI DLLs handling normal HTTP requests define an entry point named HttpExtensionProc. Although ISAPI extension DLLs define more entry points than HttpExtentsionProc, it is by far the most important method in an ISAPI DLL. The important point to realize about ISAPI extension DLLs is that they all implement this singular function when responding to HTTP requests. However, they may all respond differently.

• The HttpExtensionProc method takes a single parameter¡ªan EXTENSION_CONTROL_BLOCK structure. EXTENSION_CONTROL_BLOCK includes the entire context of a request. We don't need to see the whole structure here. However, we will see the managed equivalent in ASP.NET when we look at the HttpContext class.

Upon receiving a request, IIS packages the information into the EXTENSION_CONTROL_BLOCK. IIS then passes the structure into the ISAPI DLL through the HttpExtensionProc entry point. The ISAPI extnsion DLL is responsible for parsing the incoming request and doing something with it. The ISAPI extension DLL is completely at liberty to do whatever it wants to with the request. For example, the client might make a request that includes parameters in the query string (perhaps the client is looking for a customer lookup or something similar). The ISAPI extensio DLL uses those query string parameters to create a database query specific to the site. If the site is a commerce site, the database query might be for the current inventory. After processing the request, the ISAPI DLL streams any results back to the client.

You may have had some experience working with classic ASP, in which case much of this structure will appear familiar to you. For example, calling Write through ASP's intrinsic Response object eventually ends up executing the method pointed to by WriteClient.

We've explored the inner structure of an ISAPI DLL. Let's see how these DLLs fit into IIS.

Internet Information Services

The user interface to IIS is available through the Control Panel. To get a feel for how to administer IIS 5.1, let's take a short tour. It's important to have some facility with IIS because ASP.NET relies on it to service Web requests. IIS 5.x and 6.0 work similarly as far as dividing the server's application space into virtual directories. IIS 6.0 includes many other features such as application isolation and recycling, which is out of the scope of this discussion.

Running IIS

1. Run IIS.

   To get to IIS, first go to Administrative Tools. On Windows XP Professional, you can do this through the Control Panel. Run Internet Information Services and you should see the IIS user interface on your screen:

   
2. View configuration for a specific virtual directory.

   On the left-hand side of the screen is an expandable tree showing the Web sites and virtual directories available through IIS on your machine. To find out more details about how the directory is configured, right-click on the directory and select Properties from the context menu. You'll see the Properties dialog box:

   

   As you can see, the Properties dialog box is fairly extensive, covering all aspects of how the directory is accessed from the outside world. We won't spend a lot of time here because ASP.NET takes care of most of these issues (rather than leaving them up to IIS).

3. View file mappings for a virtual directory.

   Click the Configuration button to see the file mappings. IIS responds by showing you another dialog box listing the file mappings for the directory:

   

   These mappings tell IIS which DLL should handle the request. Static file types such as HTM are transmitted directly back to the client. However, dynamic pages whose contents can change between posts require further processing, so they are assigned to specific ISAPI DLLs. There's not a lot of variety in the kinds of DLLs listed above. In fact, most of the file types in this directory are handled by a file named ASPNET_ISAPI.DLL. We'll talk about that DLL in much greater detail soon. Notice the other DLL in the list: ASP.DLL. This is the DLL for handling classic Active Server Pages requests.

Microsoft originally developed Active Server Pages to encourage a larger number of developers than just those using C++ to undertake Web development. When IIS came out, it was certainly a feasible environment for developing Web sites on the Microsoft platform. In fact, you can still see some sites today deployed as pure ISAPI DLL sites; just look in the query strings going between the browser and the server for clues. For example, you might see a file name such as ACMEISAPI.DLL embedded within the query string.

However, writing an entire site using ISAPI DLLs can be daunting. Writing ISAPI DLLs in C or C++ gives you complete control over how your site will perform and makes the site work. However, along with that control comes an equal amount of responsibility, because developing software using C or C++ presents numerous challenges.

So in delivering ASP, Microsoft provided a single IASPI DLL named ASP.DLL. ASP Web developers write their code into files tagged with the extension .asp (for example, foo.asp). ASP files often contain a mixture of static HTML and executable sections (usually written in a scripting language) that emit output at runtime. For example, the code in Listing 1-5 shows an ASP program that spits out the HelloWorld page, which contains both static HTML and text generated at runtime. (The file name is HelloWorld.asp in the accompanying examples.)

Listing 1-5

<%@ Language="javascript" %>
<html><body><form>
    
    <h3>Hello world!!! This is an ASP page.</h3>
   
    <% Response.Write("This content was generated ");%> 
    <% Response.Write("as part of an execution block");%> 
  </form>
</body></html>

The code shown in Listing 1-5 renders the following page. IIS watched port 80 for requests. When a request for the file Helloworld.asp came through, IIS saw the .asp file extension and asked ASP.DLL to handle the request (that's how the file mapping was set up). ASP.DLL simply rendered the static HTML as the string ¡°Hello world!!! This is an ASP page.¡± Then when ASP.DLL encountered the funny-looking execution tags (<% and %>), it executed those blocks by running them through a JavaScript parser. Figure 1-2 shows how the page renders in Internet Explorer.

Figure 1-2 The results of a request made to the ASP program from Listing 1-5.

This tutorial is about developing ASP.NET software, so we'll focus most of the attention there. However, before leaving the topic of classic ASP, Listing 1-6 shows the SelectFeature.htm page rewritten as a classic ASP page. Looking at this simple ASP application presents some of the core issues in Web development and illustrates why Microsoft rewrote its Web server technology as ASP.NET. (The accompanying file name is SelectFeature.asp.)

Listing 1-6

<%@ Language="javascript" %>
<html><body><form>
    
    <h2>HelloWorld<h2>
   
   <h3>What's your favorite .NET feature?</h3>
   <select name='Feature'>
      <option> Type-Safety</option>
      <option> Garbage collection</option>
      <option> Multiple syntaxes</option>
      <option> Code Access Security</option>
      <option> Simpler threading</option>
      <option> Versioning purgatory</option>
    </select>
    </br>
    <input type=submit name="Submit" value="Submit"></input>
    <p>
        Hi, you selected <%=Request("Feature") %>
    </p>
  </form>
</body></html>

Much of the text in SelectFeature.asp looks very similar to SelectFeature.htm, doesn't it? The differences lie mainly in the first line (that now specifies a syntax for executable blocks) and the executable block marked by <% and %>. The rest of the static HTML renders a selection control within a form.

Take note of the executable blocks and how the blocks use the Response object (managed by the ASP infrastructure) to push text out to the browser. The executable block examines the Feature control (specified by the <select> tag) and prints out the value selected by the user.

Figure 1-3 shows how SelectFeature.asp renders in Internet Explorer.

Figure 1-3 The code from Listing 1-6 as viewed using Internet Explorer.

The screen in Figure 1-3 may look a bit odd because the drop-down list box shows ¡°Type-Safety¡± while the rendered text shows ¡°Multiple syntaxes.¡± Without doing anything extra, the drop-down list box will always re-render with the first element as the selected element. We'll see how ASP.NET fixes this later when we look at server-side controls. That's enough background information to help you understand the core issues associated with developing Web applications.

In the end, developing Web applications forces you to deal with two significant issues, which are managing UI using HTML over a disconnected protocol and managing the state of your application. These fundamental activities separate Web development from other types of application development.

In many ways, the programming model has gone back to the model that dominated the mid-1970s, when large mainframes served output to terminals connected directly to them. Users would submit jobs to the mainframe and get output to their terminals. So, what's changed here? First, the terminal is a lot fancier¡ªit's a powerful PC running a browser that interprets HTML. The end point to which the browser connects is a Web server (or perhaps a server farm). Finally, the connection protocol used by the client and the server is indirect (and a request can quite literally cross the globe before the user sees a result).

In Web application development, the program's primary job is to receive requests from ¡°out there¡± and to provide meaningful responses to the requestors. That often means generating complex HTML that renders in a form humans can read on the client's browser. That can be fairly involved, for example, in a modern commercial Web site supporting commerce. Customers will undoubtedly ask about current pricing, request inventory levels, and perhaps even order items or services from the Web site. The process of ¡°generating meaningful HTML for the client¡± suddenly means doing things such as making database accesses, authenticating the identity of the client, and keeping track of the client's order. Imagine doing all this from scratch!

While frameworks such as classic ASP go a long way toward making Web development more approachable, many features are still left for developers to create on their own (mostly related to the two issues mentioned at the beginning of this section). For example, building a secure but manageable Web site in classic ASP usually meant writing your own security subsystem (or buying one). Managing the state of the UI emitted by your Web site was often a tedious chore as well.

All of this brings us to ASP.NET. A common theme you'll see throughout this tutorial is that ASP.NET takes features usually implemented (over and over again) by developers and rolls them into the ASP.NET framework.

ASP.NET 2.0 takes ASP.NET 1.1 to the next level and pushes even more commonly implemented features into the framework. An example of how ASP.NET 2.0 improves upon ASP.NET 1.1 is the authentication and authorization services provided by ASP.NET 1.1; ASP.NET 1.1 included a reasonable and easy-to-manage authentication model. However, developers were often left with the task of rolling their own authentication systems into their Web sites. ASP.NET 2.0 adds an authorization subsystem. We'll cover ASP.NET Forms Authentication and other security features in depth during tutorial 10.

In the following tutorials, we'll cover the most important ASP.NET features. By the end of the last tutorial, you'll be well equipped to develop a Web site based on ASP.NET.

discuss this topic to forum