�� Rustican.com - ��α׷��

�Ž��(Mashup) ��

�� — May, 13 2008 01:11:01 GMT

Yahoo Pipe

MS Popfly

http://www.popfly.com/
�� : http://en.wikipedia.org/wiki/Popfly
Silverlight ��
�ÿ�� : Three Minut Mashup - Microsoft Popfly

IBM lotus mashup

http://www.ibm.com/lotus/mashups
2008�� ߹ݿ� ��

Google Mashup Editor

Intel Mash maker

http://mashmaker.intel.com/web/
�� Ȯ�� ϴ� ��
�ÿ�� : http://blip.tv/file/get/Intel_SW-IntelMashMakerOverviewVideo814.wmv

SQL 2005 �� Ʃ�� ڷ�

�� — Apr, 23 2008 09:12:11 GMT

file link 1 : 1024HOL.zip
1024HOL.zip ��
�� ũ��Ʈ
Performance Dashboard ��ġ ��
DMVStats ��ġ��
ClearTrace ��ġ��
��ڷ�(Wait and Queue ��)

Work Items and Undo Support

�� — Sep, 04 2007 06:59:12 GMT

Work Items and Undo Support
��
��

��

Brian A. Randell

more
...

	Print
	E-mail
	Add to Favorites
	Rate
	RSS (Team System)
	Add RSS to Any
	Related Articles
	Live Spaces
	Digg This
	BlogThis!
	Slashdot
	del.icio.us
	Technorati
	Explore the code. Download the code.

Get
the sample code for this article.

NEW:
Explore the sample code online!

- or -

Code download available at: TeamSystem2007_09.exe (197KB)

��

Contents
��

Changes
Work Item Support
Returning Work Item
Queries
Updating the Check-In
Dialog
Final Details
��

��

n the January 2007 installment of this column (msdn.microsoft.com/msdnmag/issues/07/01/TeamSystem),
I described how to build a Microsoft^�� Word 2003 add-in to work with the Team Foundation Server
version control subsystem. In the April 2007 column (msdn.microsoft.com/msdnmag/issues/07/04/TeamSystem),
I drilled down into the work item tracking subsystem. In
this month's column, I'll describe how you can add support
for work items to the add-in. In addition, you'll learn how
to add a feature that should have been in the first version
of the add-in—undo support.

��

Changes
��

In the first column describing the
add-in, I used the beta release of Microsoft Visual Studio^�� 2005 Tools for the 2007 Microsoft Office system (Visual
Studio 2005 Tools for Office Second Edition, or VSTO 2005 SE
for short). Since that time, Microsoft has released the
final version, which supports building application add-ins
for both Office 2003 and the 2007 Office system. Thus, if
you're working along with the article, you need to upgrade
the first "release" of the add-in to the RTM version. To do
this, simply open the solution and recompile on a machine
with VSTO 2005 SE installed. Once you've verified the new
version works, the next step is to add undo pending changes
support. This feature requires you modify the tfsvcUtil
class and the ThisAddin class.

In tfsvcUtil, add
a new shared method UndoPendingChanges that accepts the full
path to the document currently being modified, and have it
return a Boolean. It turns out the core functionality of
this new method mimics the existing CheckInDocument method.
The method checks to ensure a valid connection has been made
to the Team Foundation Server and that a valid workspace is
loaded. Once this is accomplished, it gets an array of
PendingChanges objects for the passed-in document. Assuming
a pending change is returned, the Undo method of the user's
workspace is called, passing in the PendingChanges array.
This method should return an integer value of 1. If it does,
the method returns true, otherwise a tfsUtilException is
thrown using the newly defined MSG_UNDO_NOT_ZERO constant.
Figure 1 provides the full method listing.

The changes you
need to make to the ThisAddIn class are pretty
straightforward. As before, due to space restrictions, I
won't go into any great detail on the Word add-in specific
code (you can compare this column's download to the previous
version to see the changes). Needless to say, you will have
to modify the class to add a new Undo Pending Changes button
to the toolbar, add a click handler for the new button, and
add additional code to handle state changes to control
whether the button is enabled. Basically, the Undo Pending
Changes button's enabled state should mirror that of the
existing Check-in button.

Work Item Support
��

Adding support for associating work
items with a check-in requires three major sets of changes
to take place. First, you need to modify the tfsvcUtil class
to support connecting to the work item store. Second, you
need to modify the CheckInDocument method to do the actual
association of work items. Finally, you need to modify the
existing frmCheckIn dialog to support listing work items for
the user to select as part of check-in.

To program work
items, you add a reference to the
Microsoft.TeamFoundation.WorkItemTracking.Client.dll
assembly from the TFSUtil project. As mentioned in earlier
columns, Team Explorer installs its supporting assemblies in
the Global Assembly Cache (GAC) by default. However, the
installer does not register them to show up in the Visual
Studio 2005 Add References dialog. You'll need to either
modify your registry so the Add References dialog sees the
assemblies or manually browse for the assemblies. You'll
find the assemblies at %Program Files%Microsoft Visual
Studio 8Common7IDEPrivateAssemblies.

Once you've added
the reference, you need to modify the tfsvcUtil class and
add an imports statement at the top of the source file as
follows:

��

Imports Microsoft.TeamFoundation.WorkItemTracking.Client

��

Currently when the add-in connects to the Team Foundation
Server box, it only connects to the version control service.
You could modify the existing connect method to also connect
to the work item store. However, there might be times when
the user of the add-in is not going to associate work items,
and thus this would add unnecessary overhead to the connect
method. Instead, you'll add a new method that is called
separately to connect to the work item store. You'll make
this method public; however, most of the time existing
methods will call the method before doing any work
item-related operations. If the connection to the work item
store is already in place, then it effectively becomes a
no-op.

You need to add a
new shared method ConnectWIS to the tfsvcUtil class. This
method accepts no parameters and returns nothing. The method
body is simple. First, it checks to make sure a valid Team
Foundation Server instance exists. If not, it throws an
exception since the user needs to initiate the connection
before this method is called. That said, you could modify
the architecture of the add-in to support attempting to
login to the user's default Team Foundation Server
installation. If the server is available, a connection is
made to the work item store using the GetServiceMethod of
the m_tfs instance, typed as a TeamFoundationServer object:

��

Public Shared Sub ConnectWIS()

  If Not m_tfs Is Nothing Then

    If tfsvcUtil.m_tfsWIS Is Nothing Then

      tfsvcUtil.m_tfsWIS = CType( _

        m_tfs.GetService(GetType(WorkItemStore)), WorkItemStore)

    End If

  Else

    Throw New tfsUtilException(MSG_SERVER_NOT_VALIDATED)

  End If

End Sub

��

You'll note the
work item store reference is cached in a class-level
variable m_tfsWIS, typed as a WorkItemStore object. You'll
need to add this variable to the class like this:

��

Private Shared m_tfsWIS As WorkItemStore = Nothing

��

Once you've done this, you need to create an overloaded
version of the CheckInDocument method so that it matches the
existing version with an additional input parameter that
accepts an array of WorkItemCheckinInfo instances. Next, cut
the body from the existing method and paste it into your new
method. Modify the original method so that it calls the new
version, passing Nothing for the WorkItemCheckinInfo array.
The modified version is now one line of code:

��

Return tfsvcUtil.CheckInDocument(docPath, comment, Nothing)

��

Now you need to
modify the code in CheckInDocument to see if any
WorkItemCheckinInfo objects were passed into the method. If
this is the case, the code calls the ConnectWIS method to
ensure a connection to the work item store has been made.
Once you've done that, you call the overloaded version of
CheckIn that accepts an array of WorkItemCheckinInfo
instances, passing Nothing for check-in notes and policy
overrides.
Figure 2 provides the complete listing.

Returning Work Item Queries
��

At this point, you've added work
item association support to the check-in process. However,
you need to enhance the tfsvcUtil class to support returning
a list of work item queries for the current Team Project. In
addition, you'll need a method that runs a query and returns
the work items to the caller so that they can be displayed
to the user as part of the integrated check-in experience.

To do this, add
two methods. The first method, GetWIQ, returns a
StoredQueriesCollection. The second method, RunWIQ, accepts
a query name as string parameter and returns a
WorkItemsCollection object. I detailed how to do this in the
April 2007 edition of this column (msdn.microsoft.com/msdnmag/issues/07/04/TeamSystem).
Figure 3 provides the code you need.

Updating the Check-In Dialog
��

Now that you've got the main code
written to handle associating work items at check-in, you
need to modify the existing check-in dialog to support work
item association. Start by making the existing form larger,
adjusting the size property to something like 640��480 to
give you room to adjust the layout. Next, add a panel
control to the form and name it pnlComments. Then cut the
existing textbox txtComment from the form and paste it into
the new panel, setting the Dock property to Fill. You'll
need to re-add a handles clause to the
txtComment_TextChanged event handler because, when you cut
the textbox, the existing clause is removed.

You'll need a
GroupBox control called grpOptions added to the form on the
left. Add two RadioButton controls into the group box and
name them rdoComments and rdoWorkItems. Change their
AutoSize property to False and their Appearance property to
Button. Set the rdoComments radio button's Checked property
to True. Finally, rearrange the form so that it resembles
Figure 4.

Figure 4 Modified
Check-In Dialog with Comments Page (Click the image for
a larger view)

With the basic UI
changes in place, you need to add support for showing work
items in a grid when the Work Items toggle button is
clicked. Add a panel control to the form called
pnlWorkItems. Inside that panel, add two panels: pnlSQCombo
and pnlGridHolder. Set pnlSQCombo's Dock property to Top and
pnlGridHolder's to Fill. You then need to add a label and a
combobox control to pnlSQCombo and a DataGridView control to
pnlGridHolder. Name the combobox cboStoredQueries and the
grid dgvWorkItems. Dock the grid within the panel control
using the smart tag. Adjust the size and location of
pnlWorkItems to match pnlComments and set its Visible
property to False.

When the user
clicks the Work Items toggle button, the click event handler
will perform the work necessary to make the work items
available. There's no point in adding the extra overhead to
the form load time unless you know you'll always want work
items associated with a document check-in. There's quite a
bit of code that makes things happen, much of which is
generic Windows^�� Forms code
necessary to set the UI correctly.

The order of
operations when the Work Items toggle button is clicked is
as follows: load the check-in action combobox data sources,
define the grid's columns, load the stored queries, and then
run the My Work Items query, binding the results of the
query to the grid control.
Figure 5 lists the code that gets called when the user
clicks the Work Items toggle button.

The
rdoWorkItems_Click event handler calls a number of
supporting procedures to do its work.
LoadCheckInActionComboBox initializes the data sources for
the check-in action combobox that will appear in the grid.
If you examine the Microsoft check-in dialog, you'll note
that, depending upon the type of work item being selected,
the check-in action combobox will either show only the word
Associate or it will show Associate and Resolve. Mimicking
this behavior requires a bit of behind-the-scenes work in
the form.

In order to make
this work, your first step is to define two class-level
arrays:

��

Private listAssociate(0) As String

Private listAssociateResolve(1) As String

��

The next thing you'll do is swap the check-in action
combobox's data source between these two arrays depending
upon the type of work item selected.
LoadCheckInActionComboBox loads the string values into the
arrays.

The event handler
then runs DefineWorkItemGrid to initialize the grid control.
This method adds an unbound checkbox control, four
data-bound text columns (Work Item Type, Work Item ID,
Title, and State), and an unbound combobox control for the
check-in action. Next, rdoWorkItems_Click calls
LoadStoredQueries. This method calls the GetStoredQueries
method you created earlier. It then data binds the results
to the cboStoredQueries combobox control.

Back in the
rdoWorkItems_Click method, the code walks the list of stored
queries until it finds the My Work Items query. Once it
finds this, the code sets that to be the currently selected
query in the cboStoredQueries combobox control. Then the
method executes LoadGridData, which retrieves the work items
returned by the My Work Items query and binds the results to
the grid. Finally, rdoWorkItems_Click hides the comments
panel and shows the work items panel.

To make the grid
work in a similar fashion to the Microsoft version, you need
to write event handlers for two grid-related events. The
first event is the CellFormatting. One of the columns
displayed by the grid is the Type property from the WorkItem
object. This property is typed as WorkItemType object. You
want to display the Name property of this object in the
grid. Unfortunately, when the ToString method is executed,
the object returns its fully qualified type name, not its
Name property. To display the Name property, you need to
change the cell's value in the CellFormatting event handler.
This code checks to see whether the column to format is the
Work Item Type column:

��

Private Sub dgvWorkItems_CellFormatting(ByVal sender As Object, _

  ByVal e As DataGridViewCellFormattingEventArgs) _

  Handles dgvWorkItems.CellFormatting


  If e.ColumnIndex = typeColumnIndex Then

    If e.Value IsNot Nothing Then

      Dim wit As WorkItemType = CType(e.Value, WorkItemType)

      e.Value = wit.Name

      e.FormattingApplied = True

    End If

  End If

End Sub

��

If it is and the cell's value is not null, the code
retrieves the WorkItemType instance from the cell and
changes the value of the cell to the value of the Name
property.

Next, you need to
implement an event handler for the
Current-CellDirtyStateChanged event. In this handler you're
changing the data source of the check-in action combobox
based upon the type of work item selected. Write the code so
that it only executes when the checkbox column's state is
changed.

To know what the
data source should be, you need to access the WorkItem
object that is bound to the current row. Once you have it,
you call its GetNextState method passing in a string value
of Microsoft.VSTS.Actions.Checkin. The code is checking to
see if there's a valid state transition from a check-in. If
there is, the code receives a valid string back that tells
it that the combobox should display both Associate and
Resolve. If a valid string is not retrieved, then only
Associate should be available. In this way, the code sets
the correct data source and then commits the current edit,
changing the state of the checkbox.

The code then
checks the value of the checkbox and, if it has been set to
checked (True), the check-in action combobox is enabled and
made the active control, and an edit is started. If the
checkbox is returning to an unchecked state, the code resets
the combobox.
Figure 6 provides the details.

The last bit of
code you need to add to process work items is developed by
creating an array of WorkItemCheckinInfo objects in the
FormClosing event (see
Figure 7). You create this array by walking the rows of
the grid, creating a WorkItemCheckinInfo instance if the
checkbox column is checked. When you create a
WorkItemCheckinInfo instance, you specify the check-in
action. In addition to this method, you need to add a public
property, WorkItems, to the form so that the array can be
retrieved once the dialog is closed. You set this property
at the end of the FormClosing method.

Final Details
��

You'll find some additional code in
the form to handle the stored queries combobox's
SelectedValueChanged event. The only thing left to do for
the add-in to support work items is to modify the
cbbCheckInDoc_Click method of the ThisAddIn class in the
Word add-in. The modified version simply checks to see if
the new WorkItems property of the dialog has data. If it
does, the new version of the CheckInDocument method is
called. Otherwise, the old version is executed.
Figure 8 displays the completed check-in experience
working with work items.

Figure 8 Completed
Check-In Dialog with Work Item Support (Click the image
for a larger view)

At this point, if
you run the add-in, you can add a document to source
control, check it in with comments and associate work items,
check it out, and even undo pending changes. By this time, a
lot of work has been accomplished, but there's even more
that can be done. In the next edition of this column, I'll
look at adding check-in notes, policy support, and possibly
other interesting features.

Send your questions and comments to mmvsts@microsoft.com.

��

NEW:
Explore the sample code online! - or - Code download available at: TeamSystem2007_09.exe (197KB)

Brian A. Randell is a senior
consultant with MCW Technologies LLC. Brian spends his time
speaking, teaching, and writing about Microsoft technologies. He
is the author of Pluralsight's Applied Team System course and is
a Microsoft MVP. Contact Brian via his blog at
mcwtech.com/cs/blogs/brianr.

Exploring Claims-Based Identity

�� — Sep, 04 2007 06:57:54 GMT

Exploring Claims-Based Identity
��
��

��

Keith Brown

more
...

	Print
	E-mail
	Add to Favorites
	Rate
	RSS (Security Briefs)
	Add RSS to Any
	Related Articles
	Live Spaces
	Digg This
	BlogThis!
	Slashdot
	del.icio.us
	Technorati
	Explore the code. Download the code.

Get
the sample code for this article.

NEW:
Explore the sample code online!

- or -

Code download available at: SecurityBriefs2007_09.exe (206KB)

��

Contents
��

Finding Common Ground
Traditional
Representation of Client Identity
ClaimSets and Claims
Claims Transformation
and IAuthorizationPolicy
Security Token Service
Trust and Federated
Identity
Until Next Time
��

��

ost enterprise applications need some basic user security
features. At a minimum, they need to authenticate their
users, and many also need to authorize access to certain
features so that only privileged users can get to them. Some
apps must go further and audit what the user does. On
Windows^��, these features are
built into the operating system and are usually quite easy
to integrate into an application. By taking advantage of
Windows integrated authentication, you don't have to invent
your own authentication protocol or manage a user database.
By using access control lists (ACLs), impersonation, and
features such as groups, you can implement authorization
with very little code. Indeed, this advice applies no matter
which OS you are using. It's almost always a better idea to
integrate closely with the security features in your OS
rather than reinventing those features yourself.

��

But what
happens when you want to extend reach to users who don't
happen to have Windows accounts? What about users who aren't
running Windows at all? More and more applications need this
type of reach, which seems to fly in the face of traditional
advice. This column will introduce you to the new identity
model in the Microsoft^�� .NET
Framework 3.0, which is designed to help address these and
other problems.

Finding Common Ground
��

On Windows, the most common type
of credential used to access an enterprise application is
simply the user's domain account. An application that uses
Windows integrated authentication receives a Kerberos ticket
to represent a client. An application using SSL might
instead receive an X.509 certificate for the client. A
Kerberos ticket and an X.509 certificate are two very
different beasts, and the code in the OS that parses,
validates, and ultimately presents the data inside is very
different. But if you take a step back and think about what
they really represent, you'll see that these two credentials
have a lot in common.

To make this
discussion more concrete, and to help introduce what might
be new terminology for some readers, let's imagine that
Alice is a user who wants to access a purchasing service
using her Windows domain account. Her domain controller
authenticates her and creates a Kerberos ticket with a bunch
of security identifiers (SIDs) in it. These SIDs represent
Alice's user account as well as the domain groups in which
she's a member, and they are embedded in the ticket along
with a signature from the domain controller. In
identity-speak, an issuer (the domain controller) has given
a subject (Alice) a security token that she can use in order
to prove her identity.

These same
ideas apply when Alice uses a certificate instead. A
certificate is just another type of security token. The
issuer in this case is a certificate authority (CA) and the
subject is Alice. Both the Kerberos ticket and the
certificate are, in essence, signed statements by an issuer
about a subject. These are just two different ways that a
trusted authority can vouch for one of its subjects. Each
signed statement can be thought of as a collection of
claims. You should think of it this way: the domain
controller is making claims about Alice's identity when it
signs the list of SIDs in her ticket. Each SID becomes a
claim. The certificate authority makes claims about Alice's
identity when it signs her name and public key. The name and
public key are examples of claims in the certificate.

If the
terminology I'm using sounds a little academic, bear with me
because these abstractions turn out to be incredibly useful
in practice. In fact, the entire goal of this new identity
model is to abstract identity in a way that reduces your
dependency on specific types of credentials without
compromising the security of your application. By coding to
the identity model in the .NET Framework 3.0, you'll be able
to process not only Kerberos tickets and certificates, but
also Security Assertion Markup Language (SAML) tokens, which
opens the door to some really interesting identity
architectures, including federated identity.

Traditional Representation of
Client Identity
��

That's enough abstract talk;
let's see this stuff in action. I'll start with a simple
Windows Communication Foundation (WCF) service that accepts
either Windows credentials or X.509 certificates from
clients. It exposes a single method that takes no arguments,
and when called by a client, it prints out details about the
client's identity.
Figure 1 shows this method, called Hello. In this
version, it uses the traditional IIdentity interface
introduced in the .NET Framework 1.0 to inspect the client's
identity.

The first step
here is to grab the ServiceSecurityContext for the call,
which is how WCF communicates details of the client's
identity to a service. I'm careful to check for null, which
would indicate an anonymous client. Then I grab the
PrimaryIdentity property and dump out its contents. Here's
what the output looks like when I configure the sample to
use Windows credentials:

��

Primary identity type: WindowsIdentity

AuthenticationType: NTLM

IsAuthenticated: True

Name: GROMITAlice

��

The concrete
type in this case is WindowsIdentity, which exposes a lot
more information than IIdentity. By downcasting to
WindowsIdentity, I'd be able to get a list of the groups of
which Alice is a member, get her user SID and use that to
look up her user record in Active Directory^��,
and I might even be able to impersonate her in some
circumstances.

Now let's run
this same example with a different WCF configuration. This
time I'll have the client submit an X.509 certificate.

��

Primary identity type: X509Identity

AuthenticationType: X509

IsAuthenticated: True

Name: CN=SampleClient; 33BB8518E4B7��

��

The Name
property in this case is a combination of the common name in
the client certificate and its thumbprint. If you wanted
further information, such as the issuer, expiration date,
and so on, you might try downcasting to X509Identity, but
you'd quickly find that this class is marked internal. But
regardless of this limitation, even if you could downcast,
you'd be dealing with an entirely different programming
model from that in WindowsIdentity. This is one problem that
the new identity model solves: it gives you a single
programming model no matter what shape of client credential
you get. In the next section, I've rewritten the Hello
sample to be claims-aware. Read on and see what identity
looks like through a claims-based lens.

ClaimSets and Claims
��

In
Figure 2, you'll see a new version of Hello that's been
rewritten using classes in System.IdentityModel, the new
identity model introduced by the .NET Framework 3.0. This
model steps back and treats identity using the abstractions
of subject, issuer, and claim. Issuers make claims about
their subjects and sign those claims to create security
tokens. Each statement from any given issuer is modeled as a
ClaimSet, which is a collection of Claim objects coupled
with a property called Issuer, which is simply another
ClaimSet.

Before
drilling straight into the code, I want you to see some
output so you can get a feeling for what a ClaimSet
represents. Have a look at
Figure 3, which shows the output of the new Hello method
when the client uses Windows credentials. The
displayClaimSet helper method in
Figure 2 pretty-prints the list of claims in the
ClaimSet, and then recursively walks up the chain of
issuers, printing their ClaimSets in turn. Not surprisingly,
with Windows credentials, the ClaimSet mainly consists of
SIDs. Looking a bit more closely at this output, we can see
how System.IdentityModel represents claims.

The Claim
class has three key properties: ClaimType, Right, and
Resource. In
Figure 3, you can see two types of claims: a SID and a
Name. The ClaimTypes for these are URIs, but I've
abbreviated them in the displayClaim helper method by
removing the first part of the URI, otherwise the output
would be pretty tough to read.

While there is
no centralized organization that defines all possible types
of claims, we really don't need that. All we need is for the
issuer to use claim types understood by the consumer (in
this case, our Hello method). In practice, you'll see
different claim types defined for use in different contexts.
As an example, if you read the Information Card Profile,
you'll see a dozen or so claim types defined for use with
information cards.

Besides
ClaimType, the Claim class also exposes a Right property
that can often help you find a claim you need. Look once
again at
Figure 3 and note that of all the SIDs shown, only the
first SID represents the subject's identity (Alice). The
rest of them represent groups. The RIGHT column in the
output shows the value of Claim.Right, which is technically
a URI that is also abbreviated in the displayClaim routine.
System.IdentityModel only defines two rights today: Identity
and PossessProperty. You can see that WCF thinks that the
user's SID in this example should be used as an identity
claim. Of all the claims presented, it's the one you'd want
to use to identify the user. All the other claims are
PossessProperty claims.

Claim.Resource
is the value of the claim. My claim dump code in
Figure 2 has a helper method, stringizeResource, which
pretty-prints these values along with their type names. You
see, the Resource property is of type object, and can
contain just about anything, depending on the type of the
claim. For example, the SID claims you see in
Figure 3 are represented by instances of the
SecurityIdentifier class introduced in the .NET Framework
2.0.

In a nutshell,
the Claim class tells you the type and value of the claim,
and often gives you a hint as to whether the claim can be
used to uniquely identify a subject or issuer. The ClaimSet
is a collection of these Claim objects, coupled with a
reference to the issuer's ClaimSet. In
Figure 3, note that the issuer is represented with a SID
of S-1-5. That SID represents the "NT Authority," which
basically means the OS. Alice is using a local account in
this example; otherwise the issuer would be identified by
the SID of Alice's domain.

As you
traverse up the chain of issuers, you'll eventually find one
that is self-referential, that is, it points to its own
ClaimSet. The displayClaimSet method in
Figure 2 shows how you can check for this using
Object.ReferenceEquals. This happens when you reach a root
authority such as a domain controller or certificate
authority. It's important to watch for this. Otherwise, if
you are traversing up the issuer chain, you might just end
up in an infinite loop when you reach the root issuer!

Now have a
look at
Figure 4, which shows the output of Hello when Alice
sends a certificate. In this case, WCF uses Claim.Right to
hint that the certificate's thumbprint can be used to
identify the client. And since I'm cheating by using
self-signed certificates for my sample code, the Issuer
property points to the subject's claim set, indicating that
it was self-issued. Note that different classes are used to
represent the claims, but the overall structure is very
similar to the previous example.

Finally, if
you look at
Figure 5, you'll see the output of Hello when the client
uses an information card to authenticate with the service.
In this case, the client's e-mail address and first name
were sent as claims, and because a personal card was used,
the issuer is identified with an RSA public key. If you want
to know more about information cards and what these
particular ClaimSets mean, read my two recent columns on
Windows CardSpace™ in the October 2006 and May 2006 issues
of MSDN^�� Magazine (msdn.microsoft.com/msdnmag/issues/06/10/SecurityBriefs
and
msdn.microsoft.com/msdnmag/issues/06/05/SecurityBriefs).

There are
several advantages to building systems that use a
claims-based lens to look at client identity. One obvious
benefit is that you get a single programming model to
inspect any kind of client identity. Just program against
ClaimSet and you're all done, right? Well, in practice, if
you're expecting a certificate, you're going to be looking
for different types of claims than if you were expecting
Windows credentials or a personal information card. In fact,
there are classes that derive from ClaimSet, such as
X509CertificateClaimSet and WindowsClaimSet, that can make
things a little easier when you know what type of credential
you are expecting. Having a single programming model is a
good thing. But you'll discover the real power of claims
once you realize that you can model just about any sort of
authentication and authorization system using these concepts
of subjects, issuers, and claims. And there's no better way
to show this than through claims transformation.

Claims Transformation and
IAuthorizationPolicy
��

Imagine that you need to build a
WCF service that accepts lots of different types of
credentials, and you want to use roles to authorize access
to features. If the user is in the Manager role, she will
have access to more features than if she is only in the
Clerk role, for example. You can model roles using claims,
and you can use claims transformation to deal with the
multitude of different credential formats you might accept.

System.IdentityModel has an interface that allows you to
perform arbitrary claims transformation. It's called
IAuthorizationPolicy, and its one key method is called
Evaluate, which takes an instance of the following class as
an argument:

��

public abstract class EvaluationContext {

    // Methods

    protected EvaluationContext();

    public abstract void AddClaimSet(

        IAuthorizationPolicy policy, ClaimSet claimSet);

    public abstract void RecordExpirationTime(DateTime expirationTime);


    // Properties

    public abstract ReadOnlyCollection<ClaimSet> ClaimSets { get; }

    public abstract int Generation { get; }

    public abstract IDictionary<string, object> Properties { get; }

}

��

This context
allows you to do a number of things, but what I care about
here is that it allows you to examine the claim sets
presented by a client, do some sort of mapping and then add
your own claim set. Say, for example, the client submitted a
certificate. Your authorization policy could map the subject
name in the certificate onto a set of roles, and expose
those roles as claims in a new ClaimSet that indicates your
application as the issuer. If instead the user supplied a
Windows credential, you could map her groups onto those same
roles using a similar ClaimSet. Once this transformation
occurs, your service simply needs to look for the ClaimSet
issued locally, and then read the user's roles from there.

WCF has
excellent support for this model. In your service's
configuration file, you can wire up a ServiceAuthorization
behavior that tells WCF to call your implementation of
IAuthorizationPolicy before it calls your service's method
(Hello, for example). This allows you to centralize the code
that deals with the myriad of client credentials that your
service accepts. Note that if you're planning to use
role-based security, you can take this one step further and
set the PrincipalPermissionMode attribute on the
ServiceAuthorization behavior to Custom, and then supply an
instance of IPrincipal that contains your roles via the
evaluation context. This will allow you to use the
PrincipalPermission attribute on your service's methods to
grant access based on roles. Here's what this behavior looks
like in config:

��

<serviceAuthorization principalPermissionMode="Custom">

  <authorizationPolicies>

    <add policyType="MyAuthorizationPolicy, MyAssembly"/>

  </authorizationPolicies>

</serviceAuthorization>

��

And here's how
you communicate a custom principal to WCF using the
evaluation context:

��

string[] roles = lookupRolesForUser(evaluationContext);

GenericIdentity id = new GenericIdentity(clientName);

evaluationContext.Properties["Principal"] =

    new GenericPrincipal(id, roles);

��

WCF will then
set up Thread.CurrentPrincipal with the supplied principal
before calling into the service's method. I only mention
this custom principal trick because a lot of people like
using PrincipalPermission and they will find this to be
handy. But let's get back to claims again, because that's
really where the power lies. So, how would you go about
creating your own ClaimSet to represent a set of roles?
While a full example can be found in the sample code
available with this column on the MSDN Magazine Web
site, here's an excerpt that shows how to do this:

��

string clientName = getClientNameFromClaims(EvaluationContext);

Claim c1 = new Claim(ClaimTypes.Name, clientName, Rights.Identity);

Claim c2 = new Claim("urn:role", "foo", Rights.PossessProperty);

Claim c2 = new Claim("urn:role", "bar", Rights.PossessProperty);

ClaimSet newClaimSet = new DefaultClaimSet(

    DefaultClaimSet.System, c1, c2, c3);

��

I used a
well-known claim set called System to indicate that this is
an internal claim set issued by my own code. The System
claim set has nothing to do with the SYSTEM account in
Windows. It's just a simple claim set that is easy to search
for. Note that I add roles as PossessProperty claims, with a
custom URI. You'll probably want to come up with something
more elaborate than "urn:role" for the claim type URI, but
since System.IdentityModel doesn't define a URI for a role
claim, you can pick one that makes sense for your app.

Once you've
created the new claim set, you can add it to the evaluation
context via the AddClaimSet method. Now the code in your
application simply needs to look for this ClaimSet you've
issued, and it will get a list of roles for the user,
regardless of whether she used a certificate or a Windows
account to authenticate.

Security Token Service
��

Imagine for a moment that you've
taken the time to build a great implementation of
IAuthorizationPolicy that maps different credential types
onto a simple set of role-based claims as I described above.
You may have lots of different services that need this
feature. One approach to reuse this code would be to drop
the class into its own assembly and wire it into every
service that needs it.

A more
interesting approach would be to issue your own security
tokens by using your authorization policy as the basis for
implementing a Security Token Service (STS), which issues
security tokens via a Web service protocol known as
WS-Trust. An STS can accept a security token of one type and
issue a token of another type, and can deal with whatever
claims transforms are necessary, such as mapping a client
certificate onto a SAML token that contains a set of roles
that an application understands. In this scenario, the
applications in your enterprise would look for a ClaimSet
issued (and signed) by your own STS.

As I am
writing this in June 2007, implementing an STS is possible,
but not very easy. (Microsoft is working on making this
easier.) Regardless of whether or not you happen to
implement your own STS, building a claims-aware application
will make it easier for you to leverage one should it ever
become available to you.

Trust and Federated Identity
��

WCF and other communication
frameworks use cryptography to ensure that the sender of a
security token is indeed the subject and that the claims in
the token were signed by the issuer named in the token. But
all of this fancy plumbing doesn't have any idea how much
you trust the issuer. If you don't trust him, you're not
going to trust the claims he makes about his subjects!
That's why the issuer is always identified when you receive
a claim set, and it's the first thing you should look at
when processing a claim set.

It's easy to
write code that accepts tokens from a single trusted issuer.
Just make sure the claim set you received was issued by the
one authority you trust, and then you can use those claims
to make security decisions. You've essentially delegated
responsibility to the STS for doing the heavy lifting such
as mapping users onto roles and dealing with different types
of security tokens.

Now imagine
you wanted to take this one step further. Instead of only
accepting Windows credentials and X.509 certificates, what
if your STS also accepted signed SAML tokens issued by an
STS at a trusted partner? This leads to the realm of
federated identity, which is very powerful. Instead of
having to worry about managing user accounts for external
users from partner companies, you can instead accept signed
statements from those partners in the form of SAML tokens. I
discuss the many benefits of automating these sorts of
business-to-business trust relationships in my article on
Active Directory Federation Services (msdn.microsoft.com/msdnmag/issues/06/11/SingleSignOn).

Federated
identity ultimately boils down to claims transformation, if
you think about it. The partner's STS makes the client's
life easy by accepting as input whatever credential is most
natural for her, given her operating system and platform.
For example, if the client is running Windows, the STS could
use Kerberos to automatically authenticate her and issue a
SAML token. Another partner company might run a completely
different OS that uses other strong authentication
protocols. But the STS at that company would use those
protocols to seamlessly authenticate the user and issue a
SAML token. Meanwhile, the user enjoys the benefits of
single-sign on, even when using applications like yours from
federated partner companies.

The STS in
your organization makes life easy for your applications.
Your STS will ensure that only SAML tokens from trusted
partners are accepted, and will then issue a new token for
use with your application. Your application can live a long
and happy life simply looking for ClaimSets issued by your
company's STS.
Figure 6 shows the flow of messages in this federated
system. The client first goes to her company's account STS
to obtain a SAML token. She sends this token to the resource
STS to obtain a new SAML token that the service will
consume.

Figure 6 Federated
Authentication Message Flow

Security token
services are designed to bridge the gap between security or
technology realms. By transforming one token type into
another, and at the same time transforming the claims inside
those tokens, they are a great way to centralize
authentication and authorization policies.

I plan on
drilling deeper into these federated identity models in the
future. For now I hope you'll trust me (pun intended) when I
say that federated identity solutions are becoming more and
more common, and you'll be able to fit into this model if
you're building claims-aware applications today. Expect
further innovation in this space in the future.

Until Next Time
��

Building claims-aware
applications will prepare you for the future of identity on
the Windows platform. For example, you'll be able to accept
not only traditional token formats such as Windows
credentials or X.509 certificates, but you'll also be ready
to accept information cards. You'll also be better prepared
to implement federated identity when the time comes for
that.

The first step
to learning about claims is to spend some time exploring
System.IdentityModel, which is the new claims-based
programming model introduced in the .NET Framework 3.0. Use
the sample code with this column to get started and see how
you can apply the power of claims-based identity in your own
applications today.

Send your questions and comments for
Keith to briefs@microsoft.com.

��

NEW:
Explore the sample code online! - or - Code download available at: SecurityBriefs2007_09.exe (206KB)

Keith Brown is a co-founder of
Pluralsight, a premier Microsoft .NET training provider. Keith
is the author of Pluralsight's Applied .NET Security course as
well as several books, including The .NET Developer's Guide to
Windows Security, which is available both in print and on the
Web. Learn more at www.pluralsight.com/keith
��

AJAX Application Architecture, Part 1

�� — Sep, 04 2007 06:54:39 GMT

AJAX Application Architecture, Part 1
��
��

��

Dino Esposito

more
...

	Print
	E-mail
	Add to Favorites
	Rate
	RSS (Cutting Edge)
	Add RSS to Any
	Related Articles
	Live Spaces
	Digg This
	BlogThis!
	Slashdot
	del.icio.us
	Technorati
	Explore the code. Download the code.

Get
the sample code for this article.

NEW:
Explore the sample code online!

- or -

Code download available at: CuttingEdge2007_09.exe (333KB)

��

Contents
��

AJAX to the Rescue
The AJAX Architecture
What's an AJAX
Framework?
ASP.NET AJAX Extensions
A Look at Partial
Rendering
Anatomy of an AJAX
Postback
Optimization Techniques
for Partial Rendering
Weighing Partial
Rendering
��

��

hether you're a system administrator, a designer, or a
developer, your job will be significantly affected by AJAX.
Administrators have to ensure that the security bar is still
high enough to face new types of possible attacks. Intranet
administrators have to guarantee that JavaScript is not
disabled on any browsers. Web designers have new challenges
to pursue due to features attainable with AJAX that were
once impossible or impractical. And developers have a new
API and a new overall approach to programming to become
familiar with. All that said, what does AJAX mean for
architects?

��

AJAX applications
are challenging because they introduce brand-new concepts
and a new foundation. The role of the architect is essential
because the AJAX paradigm straddles both the client and the
server environments. A clear architectural vision is
critical for determining what logic and processing happens
on the client and what remains on the server, as well as
what data objects the client and the server are able to
exchange.

AJAX to the Rescue
��

The Web was originally used to
share static pages of information. Over just a few years, it
evolved into a far more dynamic medium. Today, the Web is
made up of extremely interactive pages; however, it still
relies heavily on a paradigm that is based on full-page
transitions. The more the user interacts with a page, the
more frequently pages must be created and served back to the
user's browser. There are many negative consequences that
result from serving interactive pages in this manner, such
as slow page refreshes and flickering. It's an inelegant
solution that frequently leads to a poor user experience.
AJAX brings about a much-desired shift in this paradigm that
allows for cleaner Web applications and it lays the
groundwork for a new generation of applications. AJAX is
really about the end user. With AJAX features, a Web
application is much more interactive, more responsive. It's
faster and friendlier. The user is encouraged to do more
with the page as interaction and responses (and feedback in
case of slow operations) are all handled more elegantly.
Overall, the user has a much better experience.

With AJAX, more
code is executed on the browser and this requires more
script being given to the client pages. But this leaves some
big questions to be answered. What kind of script? Who will
write this script? And what architectural principles and
patterns will this script adhere to?

In this column,
I'll explain AJAX from an architectural standpoint and help
you, the developers and architects, make thoughtful choices.
At the same time, I'll give designers and administrators
enough context information to help make their jobs easier.
This is a pretty big topic, so I'm dividing it into two
parts. Don't miss next month's column when I'll continue
this discussion.

The AJAX Architecture
��

If you're considering AJAX, take a
look at
Figure 1, which illustrates the architectural
implications of such a move. A traditional Web application
does everything on the server. Only occasionally does the
application emit script code to run a task on the client—for
example, to handle data validation.

Figure 1 Classic
Web Paradigm versus AJAX Paradigm (Click the image for a
larger view)

An AJAX
application uses a client-side framework that takes care of
issuing calls to the Web server. An AJAX server-side
framework then takes care of the request and returns a data
feed to the client. This will often be a JavaScript Object
Notation (JSON) data stream, but other formats—such as XML,
RSS, and CSV—can be used.

The client
receives data feeds and updates the UI using JavaScript. The
server responds to requests by returning raw data, encoded
in a given format. Bandwidth consumption is minimized, the
speed of the app increases since requests take less time to
complete, and UI updates are able to take effect with no
visible postback. But while this solves many problems, the
increase in action on the client also brings about new
issues, such as new coding practices, new security hazards,
accessibility concerns, and so on.

What's an AJAX Framework?
��

To keep AJAX alive in a Web page, a
few conditions must be met. First, you need JavaScript
support from the browser. A full-time Internet connection is
also required—an AJAX application can't work offline. When
all the requests are made at the page level, as they are
with non-AJAX applications, the browser can offer to
navigate through a cached set of these pages, and thus can
proceed offline. This same behavior would have to be coded
explicitly in the script-based framework that partners with
the browser in an AJAX application. There's a lot going on
in this area and some new tools to help are being developed.
The ability to take AJAX applications offline is a challenge
for many software vendors.

A special
framework is required by any serious AJAX application.
Usually the framework of choice is articulated in separate
client and server parts.

There are two
main types of script code in an AJAX page: system-level code
from the framework of choice and user-level code that
implements the page's expected behavior. The system-level
code provides the engine used to send asynchronous requests
to the Web server and process any responses. The user-level
code updates the UI of the page, essentially using Document
Object Model (DOM) scripting. So here's one of those
questions I mentioned earlier: Who writes which? To discuss
this, I'll focus on a particular AJAX framework—ASP.NET AJAX
Extensions.

ASP.NET AJAX Extensions
��

An extension to ASP.NET 2.0,
ASP.NET AJAX Extensions provides AJAX capabilities to new
and existing Web sites. There are two programming models for
ASP.NET AJAX: partial rendering and remote services. They
assume two radically different architectures in the
application and consequently have different pros and cons.
(You've probably already figured out that most features in
AJAX involve trade-offs.)

In brief, partial
rendering allows you to maintain an architecture similar to
a classic ASP.NET 2.0 application. It just provides you with
a set of new server-side tools you can use to implement
flicker-free page updates.

Remote services,
on the other hand, involve a service-oriented back end
invoked by a relatively script-heavy AJAX front end. Nearly
all fundamental application processes—including
authentication, data paging, and sorting—must be redesigned.
Server-side code must be factored out to
application-specific services and a format (such as JSON)
must be selected for data exchange. Finally, a front end
must be arranged, paying due attention to limit your coding
to UI-level tasks and keeping most business logic off the
client.

The remote
services approach provides a more complete AJAX experience,
while partial rendering offers a smoother transition with a
progressive enhancement of the features already in your
existing application.

The two
approaches can be mixed to some extent. You can, for
instance, keep the same traditional Web architecture, add
some flicker-free updates here and there, and then take time
to refactor some key features in a service-oriented manner.
ASP.NET AJAX Extensions provides several key ingredients,
but significant pieces of the work for creating great AJAX
applications is still up to you.
Figure 2 shows the client and server components of the
ASP.NET AJAX Extensions framework.

A Look at Partial Rendering
��

Jeff Prosise's June 2007 Wicked
Code column (msdn.microsoft.com/msdnmag/issues/07/06/WickedCode)
offers an excellent discussion about partial rendering with
ASP.NET AJAX. From an architectural viewpoint, partial
rendering doesn't add anything new. It's just a cool way of
enhancing legacy applications with some AJAX
capabilities—the most important of which is flicker-free
page updates.

Partial rendering
doesn't require learning a lot of new skills, and it has a
very limited impact on existing code. It also offers a
straightforward fallback mechanism for aspects of an
application that may suffer from the adoption of
AJAX—security and accessibility, for example. Basically, if
in the upgrade process you get into trouble when turning
certain parts of existing code into AJAX, you can just leave
those code blocks as they were and move on.

A partial
rendering request is often referred to as an AJAX postback.
The term is quite appropriate, perfectly representing what
really happens. An AJAX postback is like a classic ASP.NET
postback, except that it is carried out by a piece of script
code defined in the client-side ASP.NET AJAX library.
Although dressed like a pure AJAX remote call, an AJAX
postback looks like a regular postback request to the
ASP.NET runtime components. Once on the server, the request
goes through the typical lifecycle of postback requests and
raises such events as Init, Load, and PreRender. On the
server, an AJAX postback differs from a classic ASP.NET
postback only in the algorithm it uses to render the final
markup. The different algorithm is key to the improved
performance and absence of page flickering. However, the
application model remains the same as in ASP.NET.
Figure 3 shows the modified lifecycle of an AJAX
postback request.

Figure 3 Lifecycle
of an AJAX Postback

Everything is the
same in the AJAX and ASP.NET postbacks, except the
implementation of the rendering stage. An AJAX postback
still fires notification events, such as Load and PreRender,
and it still processes viewstate information and raises
state changed and postback events, such as TextChanged and
Click.

An ASP.NET AJAX
page must include one instance of the ScriptManager control.
This control is the real nerve center of ASP.NET AJAX pages.
It takes care of linking the page to any required framework
script files and orchestrates partial rendering if it
detects that an AJAX postback is occurring. The
ScriptManager control checks an HTTP header in the request
to determine whether the request is an AJAX postback. The
following is an excerpt from the MicrosoftAjaxWebForms.js
file that sets the HTTP header before triggering the AJAX
request:

��

   request.get_headers()['X-MicrosoftAjax'] = 

       'Delta=true';

��

To see what makes
the rendering stage different for an AJAX postback, take a
close look at the following excerpt from the
System.Web.Extensions assembly:

��

   protected override void OnPreRender(EventArgs e) {

       base.OnPreRender(e);

       if (IsInAsyncPostBack) 

           PageRequestManager.OnPreRender();

   }

��

In particular, the code snippet shown above comes from the
ScriptManager control. As you can see, the script manager
registers a pre-rendering event handler. When fired, the
handler detects whether it is engaged in an AJAX postback
and, if so, invokes a method from the PageRequestManager
internal class. The OnPreRender method on the
PageRequestManager class does the following:

��

   internal void OnPreRender()

   {

       _owner.SetRenderMethodDelegate(

          new RenderMethod(this.RenderPageCallback));

   }

��

Quite simply, the
method sets a specific subroutine to render the markup for
the current request. SetRenderMethodDelegate is a method
defined on the System.Web.UI.Control class. Not intended for
public use, this method basically assigns an event handler
that the page or the control will use to render its
contents. In the preceding code snippet, the method is
invoked on the current page and ends up overriding the whole
rendering process for the current request. The actual code
responsible for the markup of an AJAX postback is therefore
defined in the RenderPageCallback method—an internal method
in the PageRequestManager class. (For more information on
the ScriptManager control, see the article by Ben Rush in
this issue of MSDN^�� Magazine.)

Basically the
modified rendering process for an AJAX postback loops
through all updatable panels in the page that are involved
with the postback and accumulates the markup of each. All
markup chunks, along with hidden fields and any error
information, are packed into a response stream and passed to
the client. Now let's dissect a typical partial rendering
call in the context of a sample page.

Anatomy of an AJAX Postback
��

To make an ASP.NET page a partially
rendered page, you must first add a script manager to the
page and then define independently updatable regions by
wrapping them with an UpdatePanel control. Here's an
example:

��



   
      <%-- Markup of the region goes here --%>

��

The UpdatePanel control doesn't alter in any way the visible
markup generated for the region. It simply adds a
surrounding

tag to the original markup:

��


      <%-- Markup of the region goes here --%>

��

What triggers an
AJAX postback? How is it managed? And by whom? Whenever the
script manager detects one or more UpdatePanel controls in
the page, it emits the block of script as in the following:

��

The _initialize method is a static method on the client-side
PageRequestManager object (see MicrosoftAjaxWebForms.js). It
creates a global instance of the PageRequestManager class
and initializes it. The class acts as a singleton and the
only available instance can be retrieved later through the
getInstance method. The second statement of the preceding
snippet registers an array of UpdatePanel controls with the
client framework. Each server-side UpdatePanel control is
referenced through its ID.

The key action
happening here is inside the _initialize method. As I said,
after creating the singleton instance of the class, the code
initializes it. At this time, among other things, a handler
is registered for the submit event of the DOM form object.
This means that whenever the page submits a form, the AJAX
script kicks in and places the request using XMLHttpRequest
instead of letting the request go through a normal browser
postback. The original set of form fields is maintained and
some extra information is appended for the convenience of
the server-side script manager. Hence, an AJAX postback
uploads a bit more information than a regular ASP.NET
postback.

The viewstate, as
well as any other hidden fields, are carried out and
uploaded to the server with the request. On the way back, an
updated viewstate is downloaded along with new hidden
fields, if any, and a likely shorter markup. In particular,
the response includes only the markup for the updatable
regions that have been modified during the postback. The
list includes the UpdatePanel control that triggered the
postback (and any nested panels), any other UpdatePanel
controls in the page that have the UpdateMode property set
to Always, and any UpdatePanel control that is
programmatically refreshed. The following code is an example
of how to programmatically refresh a panel based on run-time
conditions:

��

   UpdatePanel1.Update()

��

Consider the
sample page shown in
Figure 4 and the response returned. You can use a
variety of tools to monitor inbound and outbound HTTP
packets. For this column, I used Nikhil Kothari's Web
Development Helper tool, which you can download for free at
projects.nikhilk.net/Projects/WebDevHelper.aspx.

The response of
an AJAX postback is a text stream that can be seen as a
table of records with Size, Type, ID, and Content columns.
Each record is referred to as a delta node.
Figure 5 shows the table you obtain for my sample page.

A delta node
identifies a possible change that occurred during the AJAX
postback. Not all responses feature the same set of delta
nodes; it depends on what happens during the server
lifecycle of the request and the structure of the original
markup.

Figure 6 lists
all currently supported delta nodes. The JavaScript code
that is responsible for processing the delta table can be
found in the MicrosoftAjaxWebForms.js file.

Optimization Techniques for Partial
Rendering
��

After examining
Figure 6, you'll note that the response is made up of
two big blocks: markup and viewstate. Event validation data
(a security-related feature of ASP.NET 2.0), custom hidden
fields, scripts, and any other types of nodes usually take
just a few dozen bytes all together. The size of the
viewstate is an old problem for ASP.NET pages. Partial
rendering with ASP.NET AJAX, unfortunately, does nothing to
correct the problem. You still need to keep your viewstate
lean and mean to improve download time. So what can you do
to reduce the markup for a page? Essentially, you can have
smaller panels that bring back just the minimum amount of
markup you need for that particular click.

Look again at the
code in
Figure 4. The logic in the page is fairly simple, but it
illustrates the point. Basically, when the user clicks the
button, any content in the TextBox is used to update the
label.

The page contains
only one UpdatePanel control that wraps everything: textbox,
button, separator, and label. Logically speaking, this may
be acceptable. If you have to identify a region in the page
to refresh autonomously, this is a valid option. Defined in
this way, though, the region contains a significant portion
of code that isn't updated across postbacks. The TextBox and
Button controls, for instance, are not updated in this page
and the

separator is plain static text.

A better option
would be to reduce the UpdatePanel to comprehend the sole
Label control—the unique server control that gets updated in
any postback triggered by the button. In general, you should
carefully identify postback relationships between
controls—basically examining which updates which—and design
updatable panels to include the minimum number of controls
for a given user action. In the context of the sample shown
in
Figure 4, here's a better schema:

��



     OnClick="Button1_Click" />

��

The UpdatePanel
control now contains only the Label control. This means that
no (unchanged) markup will be returned for the textbox and
button controls. By default, an UpdatePanel control is
refreshed when either one of its child controls causes a
postback or another UpdatePanel in the page is refreshed.
This behavior can be altered with some public
properties—namely UpdateMode, ChildrenAsTriggers, and
Triggers.

In particular,
UpdateMode determines whether the UpdatePanel is set to
refresh unconditionally (the default) or only under certain
conditions. If you want a conditional refresh—a key setting
for optimizing updatable panels—start by setting UpdateMode
to Conditional:

��


     UpdateMode="Conditional">

��

Note, though, that conditional updates are relevant only if
you have multiple panels in a page.

There might be
situations where you find it difficult to split a page into
a set of normalized panels. A technique that might help you
in these cases is preventing children from triggering
postbacks. The ChildrenAsTriggers property is a Boolean
property (true by default) that determines whether child
controls of a panel act as triggers of AJAX postback events.
In the following snippet, the Button1 control doesn't cause
a refresh of the panel:

��


     ChildrenAsTriggers="false">
   
       
          OnClick="Button1_Click" />

��

When a child button is clicked in a panel configured like
this, an AJAX postback still occurs to execute any postback
code attached to the button—for instance, the Button1_Click
method—but no markup is returned to refresh the panel.

The third
property involved with conditional refreshing is Triggers.
This is a collection property and lists control events that
will command a refresh of the panel, like so:

��


   
       
   
   
      
           EventName="Click" />

��

In this case, the Click event on Button1 will refresh the
panel. The Button1 control may be placed anywhere in the
page. Note that I'm talking about server events. In other
words, if you want to refresh a panel when the user changes
the selection on a dropdown list, you must first add the
AutoPostBack attribute to the dropdown list.

You can still
configure a panel to refresh when a state-changed event
occurs on the server during an AJAX postback. Consider the
following code:

��



   
       
   
   
      
           ControlID="TextBox1"

           EventName="TextChanged" />

��

The label is refreshed whenever the content of TextBox1
changes:

��

Sub TextBox1_TextChanged(ByVal sender As Object, ByVal e As System.EventArgs) _

Handles TextBox1.TextChanged

   Label1.Text = TextBox1.Text

End Sub

��

The TextChanged event, as well as other state-changed
events, are server-side events that fire as a consequence of
a postback. The page still needs a button, or an
auto-postback control, to trigger the postback in order to
update the label.

The chart in
Figure 7 summarizes the potential benefits of
applying optimization techniques to updatable panels. With
partial rendering, the size of the request packet increases
a bit, but even in the least optimized case the response you
get back is smaller than in a classic ASP.NET scenario. And,
of course, users won't experience flickering.

Figure 7 Benefits
of Optimizing Updatable Panels (Click the image for a
larger view)

I should warn
you, though, that pages with a huge viewstate may not
benefit much from partial rendering. You might save 5KB of
markup, but that's faint relief if you have to carry 30KB of
viewstate.

But sometimes,
viewstate is only an innocent scapegoat for an issue that
originated elsewhere. Imagine you have a calendar control in
an ASP.NET page and you use updatable panels to keep the
refresh smooth:

��

The markup for
this fragment is no less than 7KB and can grow up to 10KB if
you add some styles. Now, if you disable the viewstate on
the calendar, you save approximately 1KB of data, which is
only about 10 percent of the total size. This is because the
Calendar is a heavy control in terms of markup. The
viewstate has little to do with it. An ASP.NET AJAX page
shouldn't use a Calendar control for date-picking functions.
Instead, the AJAX Control Toolkit provides an extender—the
CalendarExtender control—that, when added to a plain
TextBox, lets you pick dates, effectively saving 10KB of
markup.

This code pops up
a calendar whenever the textbox gets the input focus:

��



    TargetControlID="TextBox1"   

    OnClientDateSelectionChanged="updateLabel"

    Format="dd/MM/yyyy" />



     UpdateMode="Conditional">

��

The calendar is created entirely on the client and only
requires a couple of small icons to be downloaded. The
extender automatically writes the selected date to the
companion TextBox. However, with a bit of JavaScript code,
you can handle the client date selection event and do
whatever you need.

Weighing Partial Rendering
��

Partial rendering is the quickest
way of spicing up an ASP.NET Web site with AJAX
capabilities. It requires neither new skills nor a new
architecture. Instead, it lends itself to progressively
enhancing an existing Web site. But compared to a pure AJAX
approach—direct client calls to remote services—partial
rendering suffers from significant performance penalties.

As a rule of
thumb, I recommend partial rendering as the best approach
for building a rock-solid first AJAX version of a site. You
can then progressively move away from the classic ASP.NET
architecture and start building a set of back-end services
and a thick presentation layer. In the next installment of
Cutting Edge, I'll dive into the remote services approach to
AJAX. It's neat, architecturally speaking, but it may
require a complete redesign of your application and it
raises a few new issues.

Send your questions and comments for
Dino to cutting@microsoft.com.

��

NEW:
Explore the sample code online! - or - Code download available at: CuttingEdge2007_09.exe (333KB)

Dino Esposito is a mentor at
Solid Quality Learning and the author of Programming Microsoft
ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a
frequent speaker at industry events worldwide. Get in touch with
Dino at cutting@microsoft.com or join the blog at
weblogs.asp.net/despos.

��

Asynchronous Device Operations

�� — Jul, 29 2007 04:33:55 GMT


Asynchronous Device Operations
��
��

��

Jeffrey Richter

more
...

	Print
	E-mail
	Add to Favorites
	Rate
	RSS (Concurrent Affairs)
	Add RSS to Any
	Related Articles
	Live Spaces
	Digg This
	BlogThis!
	Slashdot
	del.icio.us
	Technorati
	Explore the code. Download the code.

Get
the sample code for this article.

NEW:
Explore the sample code online!

- or -

Code download available at: ConcurrentAffairs2007_06.exe (162KB)

��

Contents
��

CreateFile and
DeviceIoControl
Win32 Device
Communication
Synchronous Device I/O
in Managed Code
Asynchronous Device I/O
in Managed Code
DeviceIoControl��s
Overlapped Parameter
Opportunistic Locks
Conclusion
��

��

In my last column, I demonstrated how to implement two base
classes, AsyncResultNoResult and AsyncResult. Both
of these classes implement the IAsyncResult interface, which
is at the core of the common language runtime (CLR)
Asynchronous Programming Model (APM). In this column, I��m
going to use the AsyncResult class to implement the
APM in a way that allows you to perform hardware device
operations asynchronously. Reading this column will also
give you insight into how the Microsoft^�� .NET Framework itself performs asynchronous file and network
I/O operations. All of this code resides in my Power
Threading library, available from
wintellect.com.

CreateFile and DeviceIoControl
��

In Win32^��-based
coding, when you want to work with a file, you open it by
calling the Win32 CreateFile function and then perform
operations on the file by calling functions such as
WriteFile and ReadFile. Ultimately, when you��re finished
working with the file, you close it by calling the
CloseHandle function.

In the .NET Framework, constructing a FileStream object
internally calls CreateFile. Calling FileStream��s Read and
Write methods internally calls the Win32 WriteFile and
ReadFile functions respectively. And Calling FileStream��s
Close or Dispose methods internally calls the Win32
CloseHandle function. When working with files, most
applications just need to write data to and read data from
the file and so the FileStream class offers most of the
operations you need.

However, Windows^�� actually
offers many more operations that can be performed on a file.
For the more common operations, Windows provides specific
Win32 functions like WriteFile, ReadFile, FlushFileBuffers,
and GetFileSize. However, for infrequently used operations,
Win32 doesn��t supply specific functions; instead, it offers
one function, DeviceIoControl, that lets an application
communicate directly with the device driver (such as the
NTFS disk driver) responsible for manipulating the file.
Examples of infrequently used file operations include
opportunistic locking (microsoft.com/msj/0100/win32/win320100.aspx),
manipulating the change journal (microsoft.com/msj/0999/journal/journal.aspx),
compressing disk volumes/files, creating junction points,
formatting/repartitioning disks, and working with sparse
files.

Furthermore, DeviceIoControl can be used by any application
to communicate with any hardware device driver. Using
DeviceIoControl, an application can query the computer��s
battery status, query or change the LCD��s brightness,
manipulate a CD or DVD changer, query and eject USB devices,
and much more.

Win32 Device Communication
��

Let��s take a
quick look at the Win32 way of having an application
communicate with a device and then wrap this functionality
so that it can be used from managed code.

In Win32, the way you open a device is by calling the
CreateFile function. CreateFile��s first argument is a string
identifying the device to open. Normally, a path name is
specified for the string causing CreateFile to open a file.
However, you can pass special strings to CreateFile, causing
it to open devices.
Figure 1 shows some potential strings and describes the
kind of device the string causes CreateFile to open. Be
aware that some of these devices are restricted to the
system or members of the Administrators group and so
attempting to open some of these devices may fail unless
your application is running with the required access
permissions.

Now that you know how to open a device, let��s look at the
Win32 function DeviceIoControl:

��

BOOL DeviceIoControl(

   HANDLE hDevice,           // Handle returned from CreateFile

   DWORD  dwIoControlCode,   // Operation control code

   PVOID  pInBuffer,         // Address of input buffer

   DWORD  nInBufferSize,     // Size, in bytes, of input buffer

   PVOID  pOutBuffer,        // Address of output buffer

   DWORD  nOutBufferSize,    // Size, in bytes, of output buffer

   PDWORD pBytesReturned,    // Gets number of bytes written to 

                             // pOutBuffer

   POVERLAPPED pOverlapped); // For asynchronous operation

��

When calling this function, the handle parameter refers to a
file, directory, or device driver obtained by calling
CreateFile. The dwIoControlCode parameter indicates the
operation to perform on the device. Each operation is simply
identified via this 32-bit integer code. If an operation
requires arguments, then the arguments are placed in fields
of a data structure and the address of the structure is
passed for the pInBuffer parameter. The size of the
structure is passed in the nInBufferSize parameter.
Similarly, if the operation returns some data, then the
output data is placed in a buffer, which must be allocated
by the caller. The address of this buffer is passed in the
pOutBuffer parameter. The size of this buffer is passed in
the nOutBufferSize parameter. When DeviceIoControl returns,
it writes the number of bytes actually written to the output
buffer in the pBytesReturned parameter (which is passed by
reference). Finally, when performing synchronous operations,
NULL is passed for the pOverlapped parameter. But when
performing an asynchronous operation, the address of an
OVERLAPPED structure must be passed in. I��ll explain this is
great detail later in this column.

Now that you have a sense of how to communicate with devices
via Win32, let��s start writing a managed wrapper over these
Win32 functions so that you can write managed code in a .NET
language to communicate directly with hardware devices.

Synchronous Device I/O in Managed
Code
��

First, let��s
focus on how to perform synchronous device operations.
Later, I��ll add the code necessary to perform asynchronous
operations. In C#, I have defined a static DeviceIO class
that is a friendly wrapper around the Win32 CreateFile and
DeviceIoControl functions.
Figure 2 shows the public object model for this class.

In a managed application, you can call OpenDevice, passing
in a string similar to those shown in
Figure 1, plus the desired access and sharing.
OpenDevice will internally call CreateFile returning the
handle to the device. The handle is actually returned
wrapped in a SafeFileHandle object to ensure that it will
ultimately be closed and to get the other benefits offered
by way of all SafeHandle-derived classes. This
SafeFileHandle object can be passed as the first argument to
any of DeviceIO��s other static methods and this is
ultimately passed as the first argument when calling the
Win32 DeviceIoControl function.

The second argument to all the other methods is a
DeviceControlCode, a simple structure (value type) that
contains just one private instance field, an Int32, that
identifies a command code you want to send to a device. I
found that wrapping the Int32 command code into its own type
caused the code to read better, be more type-safe, and
benefit from IntelliSense^�� in
Visual Studio^��. Internally, the
Int32 value contained inside a DeviceControlCode instance is
passed as the second argument to the Win32 DeviceIoControl
function.

Now, when examining device control codes, you��ll detect that
there are a few common patterns, and I decided to offer
friendly methods to make working with these common patterns
convenient. All of these methods execute the operation
synchronously; that is, the calling thread will not return
until the operation is complete. Also, when calling any of
these methods, you must specify false for the useAsync
argument when calling OpenDevice.

The Control method encapsulates a pattern where the control
code you��re sending to a device causes the device to perform
some operation and the device has no resulting data to
return. There is a Control method that takes just a
SafeFileHandle and a control code and an overload that takes
an additional Object argument. The additional Object
argument allows you to pass some additional data to the
device that it will use for an operation. When you look up
control codes in the Win32 SDK documentation, the
documentation will indicate what additional information (if
necessary) you must pass for each control code. If you need
additional data, you must define a managed type that is the
equivalent of the Win32 data type, construct an instance of
it, initialize its fields, and pass a reference to the
instance for the inBuffer parameter.

The GetObject method encapsulates a pattern where
the control code you��re sending to a device causes the
device to return some data back to your application. To get
this return data, you must define a managed type that is the
equivalent of the Win32 data type, and specify this type as
the generic type, TResult, when calling GetObject.
Internally, GetObject will create an instance of this type
(which is why the generic type has the new constraint
applied to it), the device will initialize the fields inside
the call to DeviceIoControl, and GetObject will return the
initialized object back out to you. When calling GetObject,
there are two overloads allowing you to optionally pass
additional data to the device via the inBuffer parameter.

The GetArray method encapsulates the last pattern
where the control code you��re sending to a device causes the
device to return an array of elements to your application.
To get these elements, you must define a managed type that
is the equivalent of the data type of the Win32 element, and
specify this type as the generic type, TElement, when
calling GetArray. The managed data type must be defined as a
structure (value type) so that the array��s memory is laid
out linearly as required by DeviceIoControl; this is why
TElement is constrained to struct. Also, when calling
GetArray, you must specify the maximum number of elements
that you want returned from the device via the maxElements
parameter.

Internally, GetObject will create an array of these elements
and pass the address of the array to DeviceIoControl, which
will initialize the individual array elements. When
DeviceIoControl returns, it returns the number of bytes
actually placed in the array. GetArray uses this value to
shrink (if necessary) the array to the exact size so that
the number of elements in the array is equal to the number
of elements initialized. This is a big convenience as any
code that uses this array (returned from GetArray) can
simply query the array��s Length property to get the number
of items and use this value in its loop to process the
returned elements.

Figure 3 shows the managed equivalent of the Win32
DisplayBrightness structure and its helper DisplayPowerFlags
enumerated type.
Figure 4 shows an AdjustBrightness method that uses my
static DeviceIO class to constantly adjust your LCD��s
brightness up and down. I must say that this is not a very
useful application in real life, but it is a simple example
that shows the basic concepts.

Asynchronous Device I/O in Managed
Code
��

Some device
operations, like changing LCD brightness, are not really I/O
operations and so it makes sense to call DeviceIoControl
synchronously when performing these kinds of operations.
However, most calls to DeviceIoControl result in I/O (hence
the name of the function) and, therefore, it makes sense to
execute these operations asynchronously. In this section,
I��ll explain how to P/Invoke from managed code out to the
Win32 DeviceIoControl function in order to perform
asynchronous I/O operations to files, disks, and other
hardware devices.

My static DeviceIO class offers asynchronous versions of the
Control, GetObject, and GetArray methods by way of the CLR��s
APM.
Figure 5 shows the public methods (not shown earlier) of
this class that support the APM.

As you can see, all the BeginXxx methods follow the CLR��s
APM pattern; that is, they all return an IAsyncResult and
the last two parameters are an AsyncCallback and an Object.
The additional parameters match those of each method��s
synchronous counterpart. Similarly, all the EndXxx methods
accept a single parameter, an IAsyncResult, and each returns
the same data type as each method��s synchronous counterpart.

In order to call any of these asynchronous methods, two
things must happen. First, you must tell Windows that you
want to perform operations on the device asynchronously.
This is accomplished by passing the FILE_FLAG_OVERLAPPED
flag (which has a numeric value of 0x40000000) to the
CreateFile function. The managed equivalent of this flag is
FileOptions.Asynchronous (which, of course, also has a value
of 0x40000000). Second, you have to tell the device driver
to post operation completion entries into the CLR��s thread
pool. This is accomplished by calling ThreadPool��s static
BindHandle method, which takes one parameter, a reference to
a SafeHandle-derived object.

DeviceIO��s OpenDevice method performs both of these actions
when you pass true for its useAsync parameter. Here is how
my OpenDevice method is implemented:

��

public static SafeFileHandle OpenDevice(String deviceName, 

   FileAccess access, FileShare share, Boolean useAsync) {


   SafeFileHandle device = CreateFile(deviceName, access, share, 

      IntPtr.Zero, FileMode.Open, 

      useAsync ? FileOptions.Asynchronous : FileOptions.None, 

      IntPtr.Zero);


   if (device.IsInvalid) throw new Win32Exception();

   if (useAsync) ThreadPool.BindHandle(device);


   return device;

}

��

Now, let��s take a look at how BeginGetObject and
EndGetObject work internally. The other asynchronous methods
work similarly and you will be able to understand how they
work easily after walking through BeginGetObject and
EndGetObject. All of the BeginXxx methods are very simple;
they construct an object or array if the operation returns
some return value or array of elements, and then they
immediately call an internal helper method, AsyncControl,
which is implemented as shown in
Figure 6.

When performing an operation, you can pass the address of a
buffer containing additional input data to DeviceIoControl.
If DeviceIoControl returns data, the memory for that data
must be allocated before calling DeviceIoControl so that
DeviceIoControl can initialize the data; the data is then
examined when the operation completes. The problem is that
the operation could theoretically take hours to complete and
during this time garbage collections could occur that would
move the data around. As a result, the addresses passed to
DeviceIoControl would no longer refer to the desired buffers
and memory corruption would occur.

You can tell the GC not to move an object around by pinning
the object. My SafePinnedObject class is a helper class that
pins an object in memory, preventing the GC from moving it.
However, since the SafePinnedObject class is derived from
SafeHandle, it comes with all the benefits you normally get
with SafeHandle-derived types, including the assurance that
the object will be unpinned at some point in the future.
Figure 7 gives you an idea of how the SafePinnedObject
class is implemented (I��ve removed some validation code to
save space). For more information about SafeHandle-derived
types and about pinning objects in memory, see my book CLR
via C# (Microsoft Press^��, 2006).

After pinning both the input and output buffers,
AsyncControl constructs a DeviceAsyncResult object
passing into it the two SafePinnedObject objects, an
AsyncCallback, and an Object. DeviceAsyncResult is
another type internal to my implementation. This type is
derived from AsyncResult (as discussed in my last
column), which means this type implements the APM��s
IAsyncResult interface. When constructed, the
DeviceAsyncResult object simply saves all the arguments
passed to it in private fields and then returns.

DeviceIoControl��s Overlapped
Parameter
��

Now that all
this prep work is complete, AsyncControl is ready to call
DeviceIoControl. It does that by calling NativeControl
passing it the device handle, the control code, the input
buffer, the output buffer, and a reference to an Int32
bytesReturned variable (which will basically be ignored by
the method since the method will return before the operation
completes). The most important argument is the last one:
when performing an asynchronous operation, DeviceIoControl
must be passed the address of a NativeOverlapped structure
(the equivalent of a Win32 OVERLAPPED structure).
AsyncControl obtains this by calling a helper method,
GetNativeOverlapped, defined in my DeviceAsyncResult class.
This helper method is implemented as follows:

��

// Create and return a NativeOverlapped structure to be passed 

// to native code

public unsafe NativeOverlapped* GetNativeOverlapped() {

   // Create a managed Overlapped structure that refers to our 

   // IAsyncResult (this)

   Overlapped o = new Overlapped(0, 0, IntPtr.Zero, this);


   // Pack the managed Overlapped structure into a NativeOverlapped 

   // structure

   return o.Pack(CompletionCallback, 

      new Object[] { m_inBuffer.Target, m_outBuffer.Target });

}

��

This method constructs an instance of the
System.Threading.Overlapped class and initializes it. This
is a managed helper class that lets you set up and
manipulate an overlapped structure. However, you cannot pass
a reference to this object to native code. Instead you must
first pack the Overlapped object into a NativeOverlapped
object; a reference to the resulting NativeOverlapped object
can then be passed to native code. When you call Pack, you
pass it a delegate referring to a callback method that a
thread pool thread will execute when the operation
completes. In my code, this is the CompletionCallback
method.

Calling the Overlapped class��s Pack method does several
things. It allocates memory for a native OVERLAPPED
structure from the managed heap and pins it, which ensures
that the memory will not be moved should a GC occur. It then
initializes the fields of the NativeOverlapped structure��s
field from the fields set in the managed Overlapped object.
This includes creating a normal GCHandle for the
IAsyncResult object that the Overlapped object refers to
ensuring that the IAsyncResult object stays alive for the
duration of the operation.

Pack then pins the callback method (CompletionCallback)
delegate so that the fixed address can be passed to native
code, thus allowing native code to call back to managed code
when this operation completes. Pack also pins any additional
objects referenced by its second parameter, an Object array.
In my case, the inBuffer and outBuffer objects are already
pinned because I wrapped them using my SafePinnedObject
class. I need to pin them myself so that I can get their
address by calling GCHandle��s AddrOfPinnedObject method.

However, the Pack method pins them again, but in a special
way. Normally, when an AppDomain is unloaded, the CLR
automatically unpins any objects, allowing them to be
collected and thus preventing a memory leak. But Pack pins
the objects until the asynchronous operation completes. So
if an asynchronous operation is started and then the
AppDomain is unloaded, the Pack-pinned objects will not be
unpinned. After the operation completes, the objects will be
unpinned allowing them to be collected; this prevents memory
corruption.

Pack also records which AppDomain called Pack to ensure that
the thread pool thread that calls the CompletionCallback
method executes in the same AppDomain. Finally, Pack
captures the stack of code access security permissions so
that when the callback method executes, it executes under
the same security permissions that the code initiating the
asynchronous operation was under. You can call the
Overlapped class��s UnsafePack method if you prefer not to
propagate the stack of security permissions.

Pack returns the address of the pinned NativeOverlapped
structure; this address can be passed to any native function
that expects the address to a Win32 OVERLAPPED structure. In
my AsyncControl method, the NativeOverlapped structure��s
address is passed to NativeControl (see
Figure 8), which passes it to the Win32 DeviceIoControl
function.

When performing an asynchronous operation, DeviceIoControl
returns immediately and, ultimately, the DeviceAsyncResult
object that implements the IAsyncResult interface is
returned from DeviceIO��s BeginObject method.

When the operation completes, the device driver queues an
entry in the CLR��s thread pool. You��ll recall that the
driver knows to do this because DeviceIO��s OpenDevice method
internally calls ThreadPool��s BindHandle method when the
device is opened for asynchronous access. Eventually, a
thread pool thread will extract this queued entry, adopt any
saved permission sets, jump into the right AppDomain, and
call the CompletionCallback method (see
Figure 9). Don��t forget that the CompletionCallback
method is defined inside the DeviceAsyncResult class, which is derived from the AsyncResult class.

The first thing that the CompletionCallback method does is
free the nativeOverlapped object. This is extremely
important as it releases the GCHandle on the IAsyncResult
object, unpins all the objects passed to the Overlapped
class��s Pack method, and also unpins the NativeOverlapped
object itself allowing it to be GC��d. Forgetting to free the
NativeOverlapped object results in leaking several objects.

The rest of CompletionCallback��s code is pretty
straightforward. It is just a matter of recording in the
AsyncResult base class��s state whether the operation
failed or the result of the operation (adjusting the array
size if DeviceIO��s BeginGetArray was called to initiate the
operation). In CompletionCallback��s finally block, there is
some code to dispose of the SafePinnedObject objects,
ensuring that the input and output buffers are no longer
pinned and allowing them to be compacted by the GC and
ultimately have their memory reclaimed by the GC when the
application no longer cares about these buffers.

At some point, the application code will call DeviceIO��s
EndControl, EndGetObject, or EndGetArray method, passing in
the DeviceAsyncResult object returned from the
corresponding Begin method. Internally, all these End
methods just call AsyncResult��s EndInvoke method,
which either throws the exception set into it inside
CompletionCallback or returns the value set into it.

Opportunistic Locks
��

An
opportunistic lock gives you a simple example that
demonstrates an asynchronous device operation.
Figure 10 shows a static OpLockDemo class whose Main
method creates a file (using the FileOptions.Asynchronous
flag). The SafeFileHandle embedded inside the FileStream is
then passed to BeginFilter, which internally calls the
DeviceIO��s BeginControl method, passing the code that
establishes a request filter opportunistic lock on the file.
Now the file system device driver will watch to see if any
other application attempts to open the file and, if so, the
device driver will queue an entry to the CLR��s thread pool.
This will in turn call the anonymous method that sets the
s_earlyEnd field to 1, which indicates that another
application wants to access the file. The first application
will then close the file allowing the other application to
continue running with access to the file.

To test all of this, start the OpLockDemo application. Then
go to your desktop and attempt to delete the
FilterOpLock.dat file that the application creates. This
will cause the anonymous method to get called, ending the
sample application.

Conclusion
��

Windows and
its device drivers offer many features that the .NET
Framework class library does not wrap. Fortunately, though,
the .NET Framework does provide the appropriate mechanisms
allowing you to P/Invoke to gain access to these useful
features. The fact that you can perform these operations
asynchronously means you can build robust, reliable, and
scalable applications that take advantage of these features.

Send your questions and comments for
Jeffrey to mmsync@microsoft.com.

��

NEW:
Explore the sample code online! - or - Code download available at: ConcurrentAffairs2007_06.exe (162KB)

NTFS - Enhance Your Apps With File System Transactions

�� — Jul, 29 2007 04:25:42 GMT

NTFS

Enhance Your Apps With File System Transactions

��

Jason Olson

more
...

	Print
	E-mail
	Add to Favorites
	Rate
	RSS (Issues)
	Add RSS to Any
	Related Articles
	Live Spaces
	Digg This
	BlogThis!
	Slashdot
	del.icio.us
	Technorati
	Explore the code. Download the code.

This article discusses:

The Kernel Transaction
Manager
The Distributed
Transaction Coordinator
Using Transactional NTFS
from native code
Using Transactional NTFS
from managed code

This article uses the
following technologies:
Windows Vista, .NET Framework
��

Get
the sample code for this article.

NEW:Explore
the sample code online!

- or -

Code download available at: TxF2007_07.exe (159KB)

Contents
��

The Benefits of
Transactional NTFS
When to Avoid
Transactional NTFS
Getting Started
Secondary Resource
Managers
Management
What Happened to the
Transacted Command Line?
In Closing
��

��

t��s a revolutionary step for Windows^��-based
applications. And it can help you develop applications that
are more robust than they could ever have been before. It��s
Transactional NTFS, also known as TxF. In this article, I
explain Transactional NTFS, why it matters to developers,
and how you can use this new technology in your
applications.

��

What is
TxF? Conceptually, it is quite simple and can be expressed
in a logical equation: Transactional NTFS = Transactions +
NTFS. A new feature in Windows Vista™ and the next version
of Windows Server^��, code-named
"Longhorn," TxF introduces the concept of transacting file
operations.

When I
say transacted file operations, I am talking about
transactions with fully ACID semantics that can be exposed
directly to the developer. This means the ability to perform
actions that are fully atomic, consistent, isolated, and
durable is now enabled directly for file operations. These
ACID capabilities are something that database developers
have benefited from for years, and having this power for
file operations is a very welcomed addition.

You��re
probably wondering if all these transactional capabilities
have been built into NTFS, what is the performance overhead
of Transactional NTFS? Well, TxF has a strictly pay-to-play
model. If you aren��t using transacted file operations, there
is no overhead. And even if you are using transacted file
operations, the overhead is pretty minimal (in the 1 to 2
percent range). For an application that is not I/O-bound,
this performance impact probably would not even be
noticeable.

Here��s
another aspect of performance to be aware of: Transactional
NTFS is optimized for commit. When you write new pages to
disk within a transaction, they are written directly in
place while the previous pages are saved for retrieval by
other readers if necessary. Because of this optimization,
when the transaction is committed, the altered pages are
already in place on disk. As a result, the overhead of
committing the transaction is minimal.

The Benefits of Transactional NTFS
��

Transactional NTFS sounds
nice, in theory. But why should you, as a developer, use
TxF? There are a number of distinct reasons. Specifically,
TxF achieves three goals that help you develop powerful
applications:

��

Improved application stability
Improved platform stability
Increased innovation

��

Let��s take a closer look at how TxF achieves these goals.

Transactional NTFS enables better application stability by
reducing or eliminating the amount of error-handling code
that needs to be written and maintained for a given
application. This ultimately reduces application complexity
and makes the application easier to test. Say, for instance,
you are developing a document management system where a SQL
data source needs to be kept consistent with a file store on
disk. Ensuring this consistency can be tricky and
non-trivial in a non-transactional system. Without
transactional file operations, it would be nearly impossible
to account for every possible failure scenario, up to and
including the operating system crashing at any imaginable
point during the process.

One of
the ways this was handled in the past was by storing the new
version of the file with a temporary file name, writing the
new data to the SQL database, and then renaming the
temporary file to the real file name when committing the SQL
transaction. But consider what happens if the application
crashes or there is a power outage right after committing to
the SQL database but before the file is renamed. Not only
would you have an inconsistent data set, but you would also
have an artifact on the file system that you would have to
clean up at some point. As usual, the extremely difficult
part lies in the details of how many different ways the
process can fail.

Some
solutions will implement their own entire two-phase commit
protocol, which commits changes to the database and the file
system at the same time. While this gets you a step closer
to achieving the goal of application stability, implementing
a custom two-phase commit system in your own application can
be very difficult. And this requires code that you have to
write, test, and maintain yourself.

Wouldn��t
it be better if transaction management was built into the
platform rather than having to be implemented by each
developer who needs this type of behavior? With the
introduction of TxF in Windows Vista and Windows Server
"Longhorn," these capabilities are now built directly into
the platform. And because Transactional NTFS is embedded in
the system itself, it is capable of providing a level of
integration that would not otherwise be possible for your
applications.

One of
the coolest parts about Transactional NTFS is that it can
work with a large number of other transactional
technologies. Because TxF uses the new Kernel Transaction
Manager (KTM) features, and because the new KTM can work
directly with the Microsoft^�� Distributed Transaction Coordinator (DTC), any technology
that can work with DTC as a transaction coordinator can use
transacted file operations within a single transaction. This
means that you can now enlist transacted file operations
within the same transaction as SQL operations, Web service
calls via WS-AtomicTransaction, Windows Communication
Foundation services via the OleTransactionProtocol, or even
transacted MSMQ operations. You can even enlist transacted
file operations directly with other technologies that are
using XA-Transactions since DTC can also work with
XA-Transactions. Simply put, if DTC can work with something,
you can incorporate transacted file operations into it.

Using
the document management example, how can you ensure
consistency within a document management application more
easily with TxF? This is where the DTC comes in handy. To
absolutely ensure consistency between your SQL database and
your file store, you can start a transaction, perform your
SQL statements and file operations within that same
transaction, and then commit or roll back the complete
transaction based on the outcome. If your SQL call fails,
your file will never be written. If your file system call
fails, your SQL is rolled back. Everything remains
consistent, and all of this is handled automatically by the
platform since the operations are enlisted within a
transaction.

When
using this approach in my own application, I no longer have
to worry about handling and accounting for every possible
exception that could occur. I simply incorporate both
operations within the same transaction, guaranteeing that if
any exception is thrown, the platform will roll back the
operations in question. Less code, more robust, very cool!

That
leaves two more goals. As for platform stability, this is
achieved by some of the ways that Microsoft is using TxF in
its own technologies. There are three core features in
Windows Vista and Windows Server "Longhorn" that now make
use of Transactional NTFS: Windows Update, System Restore,
and Task Scheduler. All of these use TxF to write files to
the file system within the scope of a transaction in order
to handle rollback/commit in case of any exceptions, such as
a system reboot due to a loss of power. If you��ve ever
experienced a loss of power right in the middle of Windows
Update, you know this can leave your system dead and in need
of a fresh installation. But now that Windows Update uses
TxF, if this same situation occurs on your Windows Vista
system, the computer can recover by rolling back the
transaction when the system reboots. By adopting TxF
internally, Microsoft has helped make its own operating
system more stable.

Finally,
TxF drives innovation by providing a framework for using
transactions outside of SQL calls. Ultimately, Transactional
NTFS can fundamentally change the way developers write
applications, allowing them to build more robust code. By
incorporating transactions into your design, you can write
code without having to account for every single possible
failure that can occur. The operating system will take care
of those mundane details!

When to Avoid Transactional NTFS
��

Despite all the benefits
offered by TxF, there are some scenarios in which you should
think about not using it. These are the same scenarios that
are discussed when talking about avoiding traditional
database transactional systems.

You
shouldn��t use TxF if you have a system that will have
long-running transactions. The definition of "long-running"
here is relative, however. A long-running transaction is any
transaction that has been alive longer than many other
transactions within the same log. This could mean a few
seconds if there are thousands of transactions happening and
lasting for very brief periods of time. On the other hand, a
day or two might not be a long time if there are very few
transactions happening in the system.

You
should also be wary of using TxF if you are going to have
multiple writers against the same file. Transacted file
operations only allow for one writer at a time (even shared
writers), meaning that a second writer won��t be allowed to
open the file while it is being held open by another writer,
unless that second writer is within the same transaction.
Because of this, TxF can introduce contention into a system
that has multiple writers against a single file.

Finally,
you need to be careful about auto-started services and
mixing transaction resources. If the service is using
Transactional NTFS and SQL, the files it needs access to
could remain locked until recovery completes (since your
service has a direct dependency on DTC and SQL, those
services need to be started first, so you might not be able
to fully recover your files until DTC and SQL finish
recovering first).

Getting Started
��

Now let��s take a look at
how you can use TxF. If you are at all familiar with the
existing file I/O Win32^�� APIs,
the Transactional NTFS APIs should come naturally. The new
Transactional NTFS APIs are unchanged from their
non-transacted counterparts aside from the addition of a
couple of parameters at the end, which are specifically for
transactions.
Figure 1 gives some examples of Win32 file APIs and
their transacted counterparts. The complete list is much
longer; nearly every file API in Win32 has a corresponding
transacted version.

From an
application perspective, there are three primary vehicles
you can use when developing a TxF-aware application. You can
use the new KTM directly as the transaction coordinator.
This lets you develop applications that are solely using TxF
file operations within their transactions.

The
second option is to use DTC as the transaction coordinator.
When you use this method, you need to go through DTC to get
the handle for the kernel-level transaction object that you
can then pass into the Transacted APIs.

The
third way is actually an extension to the second approach
where you are using System.Transactions from managed code to
make use of DTC (or just the Lightweight Transaction Manager
if there is only a single durable enlistment). In a moment,
I��ll look at the challenges of this approach and what it
really means to managed developers.

Right
now, let��s take a closer look at these three approaches.
Figure 2 illustrates all the major moving parts in
the transaction platform in Windows Vista and Windows Server
"Longhorn."

Figure 2 The
Transaction Platform (Click the image for a larger view)

At the
API level, there are three main interfaces into TxF. The
first way is to go directly to KTM via the new ktmw32.h
header file. At the most basic level, this contains the
transaction coordinator Win32 calls for KTM:
CreateTransaction, CommitTransaction, and
RollbackTransaction. In this approach, you use these calls
directly and have KTM itself do the transaction coordination
if you are only doing transacted file operations within the
single transaction (see
Figure 3).

The
second interface is via DTC. With this approach, you use DTC
as the transaction coordinator. To use TxF through DTC, you
make use of COM and QueryInterface to grab the
IKernelTransaction interface directly from DTC��s
ITransaction interface. Once you have a reference to the new
IKernelTransaction COM interface, you can call its GetHandle
method in order to retrieve the kernel-level transaction
handle that can then be used to pass to the various
Transacted APIs in Win32 (see
Figure 4).

If you
are a managed developer, the solution isn��t quite as
elegant. Currently, to use TxF from managed code, you have
to use P/Invoke and COM interop. Don��t let that scare you
away, though. For a large number of uses, the P/Invoke and
COM interop code isn��t too bad. If you��re still too timid,
you can grab some sample code from the MSDN^�� Magazine Web site. The sample has a simple managed wrapper
around Transactional NTFS that you can freely use in your
own applications. (While you��re playing with this sample
code, you should also check out the
screencast. It demonstrates using TxF with just KTM,
with SQL, and with Windows Communication Foundation.) In the
short term, a more robust version of this wrapper will be
included in an upcoming release of the Windows VistaBridge
samples as part of the Windows SDK. In the longer term, the
product team is currently working on a plan to integrate
these APIs directly into the Microsoft .NET Framework.

To use
TxF from managed code, you can continue to use the
functionality of the System.Transactions namespace. To dig
down into TxF, you are essentially following the same path
that you would when using DTC in native code (see
Figure 4), but with the managed code equivalents. From
an object level, it looks something like
Figure 5.

Figure 5 TxF
from Managed Code

The
first thing you need to do is grab the IDtcTransaction COM
interface from the current Transaction (this can be done
using the TransactionInterop helper class in
System.Transactions). Once you have the IDtcTransaction, you
can cast it to an IKernelTransaction COM interface, which
under the hood will QueryInterface for the
IKernelTransaction COM interface. Then, once you have a
reference to the currently running kernel-level transaction
via IKernelTransaction, you call GetHandle on it to get a
handle that you can pass to the various new Transacted Win32
APIs. For example, you can pass this transaction handle to
CreateFileTransacted in order to obtain a SafeFileHandle for
a new file. This SafeFileHandle can then be passed to the
constructor of the StreamWriter class, which is then used to
write to the file (see
Figure 6).

You
should also know that TxF has read-committed consistency.
This has impact on both writing and reading a file
transactionally. When writing a file, only a single
transaction can be writing to the file (note that we are
talking about a single transaction, not a single writer).
Non-transacted writers are blocked by the transacted writer,
even if opened with share-write access. Additionally, a
non-transacted writer prevents any transacted reader or
writer from being opened on the file.

When
reading from a file transactionally, there is an isolated
view of the file for the life of the file handle. This
isolated view of the file is based on the most recent
contents of the file when the file handle was created. If a
transacted writer is in the middle of modifying the file,
those modifications won��t be picked up by a new transacted
reader until the changes are actually committed. In
addition, because of the read-committed consistency of
Transactional NTFS, this isolated view of the transacted
reader remains even if a transacted writer changes a file
and commits those changes anytime during the life of the
transacted reader. These transacted readers block
non-transacted writers but only while their file handle is
open. They do not block modification of the file for the
life of the transaction. If you need a consistent view of
the file throughout the life of the transaction, you need to
keep the transacted reader��s file handle open.

Finally,
developers should be aware that when using file handles for
transacted file operations, there is a distinct difference
between transacted and non-transacted file operations with
regard to file handles. If you get a file handle from one of
the new Transacted APIs, the file handle could become
invalid at any time, whether from a transaction committing
or a transaction rolling back.

Secondary Resource Managers
��

Secondary resource
managers allow applications to manage their own resources
running within a transaction, particularly their own
transaction logs, as well as other possible file operations
within a directory (and all subdirectories) specific to the
application.

I have a
good reason for discussing secondary resource managers in
this article. When TxF was developed, there was an emphasis
placed on availability over consistency. Since it is now
possible within a transaction to modify files that are
necessary to boot, the operating system has to guarantee
that these files are always available. In order to guarantee
these files are always available, the system drive��s
transaction manager is required to be the transaction
coordinator. Because of this, you cannot use a different
transaction coordinator, like DTC for instance, as the
"root" or superior transaction coordinator for these files.
As a result, a transaction that needs to use DTC with other
resource managers (such as SQL) cannot use TxF on the system
drive. This means that if you want to use Transactional NTFS
on the system drive within the same transaction as
operations against SQL Server™, for example, you must use a
different strategy.

The
simplest way to get around this limitation is to place all
of your data on a separate partition. If that is not
possible, or you cannot guarantee that the data will be on a
non-system partition, you can use a secondary resource
manager. When using a secondary resource manager to manage
transactional file operations for a given directory (and any
subdirectories), you are informing KTM that the application
will control resource management for any files residing
there. This allows the default resource manager on the
system volume to continue to be available and, hence, allows
a transaction coordinator other than KTM to become the
superior transaction coordinator.

At the
command line, you can create and manage secondary resource
managers using the FSUtil tool and the commands shown in
Figure 7. However, for applications that need to use a
secondary resource manager, it is better to control the
secondary resource manager programmatically. To do this, you
can use the DeviceIoControl Win32 API with specific
dwIoControlCodes, outlined in
Figure 8.

When
using secondary resource managers, the startup and recovery
of the resource manager must be done by your application.
The act of starting the resource manager creates the KTM
objects to manage transactions, and opens up the Common Log
File System (CLFS) log and metadata that is stored locally
with your directories; the act of recovering the resource
manager recovers all files to a consistent state within the
secondary resource manager. The resource manager will stop
when the last handle to the resource manager root goes away.
As a result, files within a secondary resource manager are
not automatically made consistent at system startup.
Instead, that is done when your application starts and
recovers the secondary resource manager. In addition, files
are not automatically made consistent when the system is
backed up with any software that uses the volume shadow copy
service (VSS) unless you provide a VSS writer. Of course, if
you restore an inconsistent secondary resource manager and
then start and recover it, the files will be made
consistent. Make sure that your backup software also backs
up the TxF metadata using the
FSCTL_TXFS_READ_BACKUP_INFORMATION and
FSCTL_TXFS_WRITE_BACKUP_INFORMATION dwIoControlCodes.

As I
mentioned, the use of secondary resource managers is only
required if the files participating in the transaction are
on the system drive and you want a technology other than KTM
to be the superior transaction coordinator or you want to
control the consistency and availability of the resource
manager. Secondary resource managers become very powerful in
this regard if you wish to have more direct control over the
behavior of the resource manager in your application. If the
files you are operating on are on a non-system drive, you do
not need to worry about the use of secondary resource
managers. We are currently working to lessen this
restriction in Windows Server "Longhorn" so this requirement
won��t be applied to the entire system drive, but only to the
parts of the system drive that Windows absolutely needs to
boot.

Management
��

For any application that
is going to live in a production environment, it is
important to think about how the application will be
managed. Therefore, you need to know how you can monitor TxF
itself.
Figure 9 shows a list of control codes that can be used
with DeviceIoControl to help you do just that.

Similar
to the information on resource managers provided previously,
you can get the same information as these control codes from
the FSUtil command-line tool. The commands are as follows:

��

fsutil transaction list
fsutil transaction fileinfo
fsutil transaction query

��

What Happened to the Transacted
Command Line?
��

If you were involved in
the early betas for Windows Vista, you may recall seeing a
transaction command available from the command line. This
allowed an administrator to start a transaction from the
command line and run an application, and then any file
operations within that application would implicitly be
transacted.

While
testing application compatibility in Windows Vista, the team
found that this implicit model was difficult for COM+ and
managed developers to use safely. As a result, the model was
changed to be opt-in and explicit to avoid this confusion.
The downside to this change is that if you wish to use TxF
in your app, your code will have to change to call the new
APIs.

In Closing
��

Transactional NTFS is a
truly innovative technology. It helps developers create more
reliable applications on a more powerful platform. With TxF,
developers can have atomic file operations that result in
consistent file data, easing the test burden by reducing the
number of error conditions to test for, and assisting in
multi-user environments by providing isolation via ACID
transaction semantics. Consider using TxF when developing
apps that target Windows Vista and Windows Server
"Longhorn." For more information, see the "Transaction
Resources" sidebar.

I would
like to thank Dana Groff, Christian Allred, and Jon Cargille
for their help with this article.

Transaction
Resources

Here��s a list I��ve compiled of resources relating to
Transactional NTFS. I��ll be updating this list over
time to reflect new resources, so be sure to watch
for updates on
my blog.

Web Sites:
��

��

Blogs:
��

��

Specific Blog Posts:
��

��

Media:
��

��

Close [x]

��

NEW:
Explore the sample code online! - or - Code download available at: TxF2007_07.exe (159KB)

�� Rustican.com - ���α׷��� ����

�Ž���(Mashup) ����

Yahoo Pipe

MS Popfly

IBM lotus mashup

Google Mashup Editor

Intel Mash maker

SQL 2005 ���� Ʃ�� �ڷ�

Work Items and Undo Support

Exploring Claims-Based Identity

AJAX Application Architecture, Part 1

Asynchronous Device Operations

NTFS - Enhance Your Apps With File System Transactions

Enhance Your Apps With File System Transactions

�� Rustican.com - ��α׷��

�Ž��(Mashup) ��

SQL 2005 �� Ʃ�� ڷ�