Working with legacy web services and WCF

It is quite common that WCF has problems working with old and none-.NET web services. Usually the old “web reference” (ASMX) tehcnology works better in this kind of situtations, but this one time I was determined to solve the challange using WCF.

After having Visual Studio generate me the client classes I did a unit test to see if I could call the web service successfully. It turned out that the call succeeded and the function returned a response. Unfortunately the response object only contained the result code and descrption, but the data property was null.

Usually in this kind of situations I first turn to Wireshark or some similiar network traffic packer analyzer to see what’s actually gets sent and returned. This this time I had to come up with an alternative way as the web service only allowed me to use a secure HTTPS address so all the traffic was encrypted. As the calls returned a valid object with part of the expected data I knew the authentication was working and there was nothing wrong with the message headers. This meant it was enough for me to see the message content and writing this simple message inspector worked as the solution.

    public class SoapMessageInspector : IClientMessageInspector, IEndpointBehavior
    {
        public string LastRequest { get; private set; }
        public string LastResponse { get; private set; }

        #region IClientMessageInspector Members

        public void AfterReceiveReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
        {
            LastResponse = reply.ToString();
        }

        public object BeforeSendRequest(ref System.ServiceModel.Channels.Message request, IClientChannel channel)
        {
            LastRequest = request.ToString();
            return null;
        }

        #endregion

        #region IEndpointBehavior Members

        public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
        {
            clientRuntime.MessageInspectors.Add(this);
        }

        public void AddBindingParameters(ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters) { }
        public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher) { }
        public void Validate(ServiceEndpoint endpoint) { }

        #endregion
    }

Usage

            inspector = new SoapMessageInspector();
            factory.Endpoint.Behaviors.Add(inspector);

The message inspector revealed to me that the call was sent ok and also the data returned by the server was fine. It was the WCF framework that failed to properly deserialize the response.

The property for the result data was called Any in the response class, so I took a look at the WSDL provided by the server.

<s:complexType name="response">
<s:sequence>
<s:element type="s0:statusType" name="status" maxOccurs="1" minOccurs="1"/>
<s:any maxOccurs="unbounded" minOccurs="0" processContents="skip" namespace="targetNamespace"/> </s:sequence>
</s:complexType>

The any WSDL element leaves the structure of the content undefined. WCF translates this to a Any property of type XmlElment. The reason why WCF could not process the response correctly was pobably caused by this and the minOccurs value.

After trying to edit some of the attributes of the generated classes without success, I decided to take over the parsing of the response using a custom response formater.

        public class MyResponseBehaviour : IOperationBehavior
        {
            public void ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation)
            {
                clientOperation.Formatter = new MyResponseFormatter(clientOperation.Formatter);
            }

            public void AddBindingParameters(OperationDescription operationDescription, System.ServiceModel.Channels.BindingParameterCollection bindingParameters) { }
            public void ApplyDispatchBehavior(OperationDescription operationDescription, DispatchOperation dispatchOperation) { }
            public void Validate(OperationDescription operationDescription) { }

        }

        public class MyResponseFormatter : IClientMessageFormatter
        {
            private const string XmlNameSpace = "http://www.eroom.com/eRoomXML/2003/700";

            private IClientMessageFormatter _InnerFormatter;
            public eRoomResponseFormatter(IClientMessageFormatter innerFormatter)
            {
                _InnerFormatter = innerFormatter;
            }

            #region IClientMessageFormatter Members

            public object DeserializeReply( System.ServiceModel.Channels.Message message, object[] parameters )
            {
                XPathDocument document = new XPathDocument(message.GetReaderAtBodyContents());
                XPathNavigator navigator = document.CreateNavigator();

                XmlNamespaceManager manager = new XmlNamespaceManager(navigator.NameTable);
                manager.AddNamespace("er", XmlNameSpace);

                if (navigator.MoveToFollowing("response", XmlNameSpace))
                {
                    ExecuteXMLCommandResponse commandResponse = new ExecuteXMLCommandResponse();
                    // and some XPath calls...
                    return commandResponse;
                }
                else
                {
                    throw new NotSupportedException("Failed to parse response");
                }
            }

            public System.ServiceModel.Channels.Message SerializeRequest( System.ServiceModel.Channels.MessageVersion messageVersion, object[] parameters )
            {
                return _InnerFormatter.SerializeRequest( messageVersion, parameters );
            }

            #endregion
    }

As the web service only provided one function returning a fairly simple response object writing the formater only required a couple of lines of code to parse
the response data using XPath. As soon as I had replaced the default formatter with my own, things started working perfectly.

            factory.Endpoint.Contract.Operations.Find("ExecuteXMLCommand").Behaviors.Add(new MyResponseBehaviour());

I found out later that the message inspector I had written earlier also provided me with a way to throw exceptions with meaningful messages as the server always included a error descrption in the SOAP error envelope that WCF did not reveal.

WCF and Domain Trust Failure

Lately users have been getting the following error when accessing a WCF services that uses Windows authentication.

"The trust relationship between the primary domain and the trusted domain failed."

Not all users are getting this error despite that they all belong to the same domain.

Trying to find a solution I came up with three options:

  • Remove any inactive trusted domains from Active Directory
  • In the local security policy of the server change the cache value of the “Interactive Logon: Number of previous logons to cache” to 0 (zero)
  • Change the unhandled exception policy back to the default behavior of previous .NET Framework versions.

As we needed a quick fix, the tech guys picked the last option and added the following lines to the Aspnet.config in the %WINDIR%\Microsoft.NET\Framework\v2.0.50727 directory:

<configuration>
<
runtime>

<legacyUnhandledExceptionPolicy enabled="true" />

</runtime>
</
configuration>

After restarting the server the users could once again access the service.

I believe the second option of disabling the logons cache would be a better solution, but that one it yet to be tested.

WCF: Windows Authentication and External Users

Using Windows authentication with WCF enables developers to support centralized access management for enterprise services. As default the framework uses the computer login credentials of the Windows user accessing the service.

This default behavior works for all the cases where the computer belongs to the same domain as the server, e.g. when using a company computer. But a when external users needs to access a service that uses Windows authentication, the client needs to send credentials the server can identify. For this you need to define the credentials prior to creating the channel.

factory.Credentials.Windows.ClientCredential =
new System.Net.NetworkCredential(name, password, domain);

 
Make sure not to mix the factory.Credentials.Windows.ClientCredential and factory.ClientCredential properties.

There is also a good resource about debugging Windows authentication errors at MSDN.

WCF Streams and Timeouts

I’ve been working on a WCF service for distributing file updates. Recently I did some testing to see how well the system performs through slow connections and discovered some issues I’m glad I run into before the release.

My first discovery was that WCF (using a streaming net.tcp binding) does not perform very well with high ping round-trip-times. In this case my first test setup was quite extreme, as I connected to the service using VPN through a WLAN router connected to the internet by 3G. The ping was something around 750ms.

My second discovery was that the default SendTimeout for a service is 1 minute. This is not nearly enough for a service returning files! I believe the default timeout will cause problems even with fast connections as it only depends on the size of the largest file. If the operation is still returning data after 1 minute, the server closes the stream without warning and returning just some timeout or IO exception.

Fortunately fixing the issue was easy, I just needed to increase the SendTimeout of the service:

binding.SendTimeout = new TimeSpan(1, 0, 0); // 1 hour

If you change the SendTimeout, you should also consider increasing the default 10 minute ReceiveTimeout of the client binding.

There is of course a small security risk in incresing the timeouts, but in my case it’s ok as my service will only be used in a internal network. I wonder if there is a way to define an operation specific SendTimeout?

Don’t use “Add Service Reference”!

The Visual Studio tools have always been crappy at generating code. The result is acceptable if you are a Microsoft representative selling new technology at some fancy seminar, but it is nothing you want to use in a release version of you application. The Visual Studio “Add Service Reference” (WCF) feature makes no exception.

The “Add Service Reference” feature might become handy when you need to connect to some none-.NET services, but in most other cases I sincerely recommend the manual approach. In this article I’m only going to tell you have, if you need more information check Miguel A. Castro’s article “WCF the Manual Way…the Right Way“.

Start by createing a basic class library project and reference the System.ServiceModel. This library is going to be referenced by both the client and the server. Don’t use the WCF templates as they will only make your life harder on the long run.

Add your service interface to the newly created project:

[ServiceContract(Name = “TestService”, Namespace = http://www.company.com/tests&#8221;)]
public
interface ITestService
{
    
    [OperationContract]
    TestData GetData(TestData data);

}

[DataContract]
public
class TestData
{
   
public TestData(string message)
    {
        Message = message;
    }

    [DataMember]
    public
string Message { get; set; }

   
public
override string ToString()
    {
        return “Some shared override”;
    }
}

Now add a new project to your solution and implement the server the same way you normaly would:

public class TestService: ITestService
{

    #region ITestService Members

    public TestData GetData(TestData data)
    {
        return new TestData(“Hello world! + data.Message);
    }

    #endregion
}

Instead of messing around with the XML configuration file, just configure your host by code:

ServiceHost host = new ServiceHost(typeof(TestService), new Uri[] { new Uri(uri) });
NetTcpBinding
binding = new NetTcpBinding(SecurityMode.Transport);

binding.TransferMode = TransferMode.Streamed;
binding.MaxBufferSize = 65536;

binding.MaxReceivedMessageSize = 104857600;
host.AddServiceEndpoint(typeof(ITestService), binding, “service1”);

host.Open();

As you can see there is not much new to implementing the service, but now it’s time for the client!

Create a new project for the client and reference the class library you first created. Instead of going for the “Add Service Reference” option create your own proxy:

public class TestServiceClient: ITestService
{

    private ITestService service;
    
    public TestServiceClient(Uri uri)
    {
        
// Any channel setup code goes here
   
     EndpointAddress address = new EndpointAddress(uri);
        NetTcpBinding binding = new NetTcpBinding(SecurityMode.Transport);
        binding.TransferMode = TransferMode.Streamed;
        binding.MaxBufferSize = 65536;
        binding.MaxReceivedMessageSize = 104857600;

        ChannelFactory<ITestService> factory = new ChannelFactory<ITestService>(binding, address);
        service = factory.CreateChannel();
    }

    #region ITestService Members

    public TestData GetData(TestData data)
    {
        return service.GetData(data);
    }

    #endregion
}

You can now place useful functions in your data classes and utilize them both on the client and server side. If you are running with .NET 3.5 you can even leave out the Data and Member Contract attributes of the data classes (same as choosing “Reuse types in referenced assemblies” when using thee Add Service Reference feature).

The biggest advantages of building your proxy manually compared to using the Add Service Reference feature is that you can now share classes and have the same client class implement several service interfaces instead of only one.

Closing Returned Streams in WCF

Streams should always be closed after usage to free the resources behind them. WCF web service functions returning values likes streams are no exception. You should never rely on your client to do this but for some reason many WCF streaming examples overlook this.

To correctly dispose return values WCF provides you with two options:

  • Setting the OperationBehaviorAttribute.AutoDisposeParameters to true
  • Using the OperationCompleted event.

I like to use the event as it gives me more control.

Here is what I think is a correctly implemented GetFile method:

public Stream GetFile(string path) {
   Sream fileStream = null;    

   try   
   {
      fileStream = File.OpenRead(path);
   }
   catch(Exception)
   {
      return null;
   }

   OperationContext clientContext = OperationContext.Current;
clientContext.OperationCompleted += new EventHandler(delegate(object sender, EventArgs args)
   {
      if (fileStream != null)
         fileStream.Dispose();
   });

       return fileStream;
}

The dispose method should not throw an error even if the client has already correctly closed the stream.

More info: http://msdn.microsoft.com/en-us/library/system.servicemodel.operationcontext.operationcompleted.aspx

WCF and Windows Security Revisited

Configuring WCF to require Windows authentication is a pretty trivial task. But this alone does not set any requirements for what the clients credentials need to be. If you want the clients to belong to a certain Active Directory group, you are required to do some coding to achive it.

One option is to define PrincipalPermission attributes in every class to define its security requirements. But this means hard coding the requirements. The only flexible solution I found was to write my own  ServiceAuthorizationManager and read the group name from the app.config.

public class CustomAuthorizationManager : ServiceAuthorizationManager
{
protected override bool CheckAccessCore(OperationContext operationContext)
{

    // For mex support
    if (operationContext.ServiceSecurityContext.IsAnonymous)
        return true;

    // When Windows authentication has been setup using an application setting
    if (Properties.Settings.Default["UserGroup"] != null)
    {

        WindowsIdentity identity = operationContext.ServiceSecurityContext.WindowsIdentity;

        if
(!identity.IsAuthenticated)
            throw new SecurityTokenValidationException("Windows authentication required");

        WindowsPrincipal principal = new WindowsPrincipal(identity);
        string group = Properties.Settings.Default["UserGroup"].ToString();

        return principal.IsInRole(group);

    } else {

        return base.CheckAccessCore(operationContext);}

    }

}

In the service app.config:

<serviceBehaviors><behavior name="Service1Behavior">
...
<serviceAuthorization serviceAuthorizationManagerType="CustomAuthorizationManager, MyAssembly"/>
</
behavior>
</
serviceBehaviors>

Working Web Services

Working with web services in .NET (starting from 2.0) should be as easy as 1-2-3. Tools are provided for auto-generating code based on a WSDL. With .NET 3 Microsoft tries to direct every developer to use WCF for handling web services. Trying to get a Axis based Java web service to work proved to require some tricks.

I first generated a web service client (proxy) using the Add Service Reference tool in Visual Studio. This got a me a WCF based class to interact with the web service. All this was easy, but when I tried to call one of the remote methods, I was literally left with nothing (null).

As the diagnostic features (message tracking and tracing) of WCF really didn’t reveal anything usefull, I had to implement my own custom behavior with a message inspector to see what was really happening behind the secenes. This revealed that messages where sent and received, but the data did not get processed by the client. In the end, there wasn’t much else to do than to fall back and to make use of the older .NET web service handling technology (System.Web.Services).

Once again generating a client based on the WSDL was easy. And when I now tried calling the remote function I was returned with the data I expected. So, once again legacy technology prevailed when the newer failed.

As authentication (basic HTTP) is required by most we services I needed my client to support this. Normally this should only require to supply some valid credentials:

MatrixWebServicesServiceproxy = new MatrixWebServicesService();
proxy.PreAuthenticate =
true;
proxy.Credentials =
new NetworkCredential("user", "password");

But this did not do the trick. Using a TCP sniffer I was able to see that the “Authorized” HTTP header did not get sent to the server. After some research I came to the following conclution: The .NET client (this also applies to WCF) is designed to expect a “Unauthorized” HTTP error (401) from the server to respond to, without that the credentials are not sent. Also setting the PreAuthenticate property to true will only force the client to send the credentials with every message after this has first been required by the server. The problem with the Java web service was that is sent a “Internal Server Error” HTTP error (501) and a valid SOAP error when the authentication failed. So the credentials where never sent.

The only solution I found for this was to modify the generated client code and force it to send the credentials with every message. This can be done by overriding the clients GetWebRequest method. As the client is based on a bunch of partial classes, you only need to add your own with the following code to get things running:

public partial class SomeJavaWebServices : System.Web.Services.Protocols.SoapHttpClientProtocol {

protected override WebRequest GetWebRequest(Uri uri)
{
    HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(uri);

    if (PreAuthenticate)
   
{

        NetworkCredential networkCredentials = Credentials.GetCredential(uri, “Basic”);
        if (networkCredentials != null)
        {

            byte[] credentialBuffer = new System.Text.UTF8Encoding().GetBytes(networkCredentials.UserName + “:” + networkCredentials.Password);

            request.Headers[“Authorization”] = “Basic “ + Convert.ToBase64String(credentialBuffer);
        }
        else
        {
            throw new ApplicationException(“No network credentials”);
        }
    }    return request;
}

 

}

WCF and Windows Security

Working on my first real WCF service project I encountered problems with getting the client to connect to the host when running it on another computer and a different user. This was probably challenging only because of my inexperience with the technology and the overwhelming amount of documentation (but very little discussion) on the subject.

When you want to use Windows security with a netTcpBinding, you must first configure the client and the host to do so:

<security mode="Transport">
<
transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>

This was pretty trivial.

The magic lays behind the identity configuration of the client endpoint. I’m still not sure why this is required, but to get Windows authentication to work, you need to define a correct User Principal Name (UPN) or Service Princial Name (SPN). 

So, if the host is running with user credentials, you should use its UPN:

<identity>
<
userPrincipalName value="user@some.com" />
</identity>

And if the host is running as s service, define a SPN:

<identity>
<
servicePrincipalName value="Host/MYCOMPUTER" />
</identity>

Host/<server> is the default SPN, but a domain administartor can also create a service specific one.

What I noticed when trying different configurations, was that if I leave the UPN/SPN value blank (or entered any value), the client will for some reason connect. This has to do something with the fact that as default Kerberos is used for the authentication, but if that fails NTLM takes over. So to make sure your settings are correct, try the following:

<client>
<endpoint behaviorConfiguration = "clientEndpointCredential">
...
</
client>
<
behaviors>
<
endpointBehaviors>
<
behavior name="clientEndpointCredential">
<
clientCredentials>
<
windows allowNtlm="false" />
</
clientCredentials>
</
behavior>
</
endpointBehaviors>
</
behaviors>

Comprehensive guide to WCF security: http://www.codeplex.com/WCFSecurityGuide