[SOLVED] HttpWebRequest is extremely slow!

Issue

I am using an open source library to connect to my webserver. I was concerned that the webserver was going extremely slow and then I tried doing a simple test in Ruby and I got these results

Ruby program: 2.11seconds for 10 HTTP
GETs

Ruby program: 18.13seconds for 100 HTTP
GETs

C# library: 20.81seconds for 10 HTTP
GETs

C# library: 36847.46seconds for 100 HTTP
GETs

I have profiled and found the problem to be this function:

private HttpWebResponse GetRawResponse(HttpWebRequest request) {
  HttpWebResponse raw = null;
  try {
    raw = (HttpWebResponse)request.GetResponse(); //This line!
  }
  catch (WebException ex) {
    if (ex.Response is HttpWebResponse) {
      raw = ex.Response as HttpWebResponse;
    }
  }
  return raw;
}

The marked line is takes over 1 second to complete by itself while the ruby program making 1 request takes .3 seconds. I am also doing all of these tests on 127.0.0.1, so network bandwidth is not an issue.

What could be causing this huge slow down?

UPDATE

Check out the changed benchmark results. I actually tested with 10 GETs and not 100, I updated the results.

Solution

What I have found to be the main culprit with slow web requests is the proxy property. If you set this property to null before you call the GetResponse method the query will skip the proxy autodetect step:

request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}

The proxy autodetect was taking up to 7 seconds to query before returning the response. It is a little annoying that this property is set on by default for the HttpWebRequest object.

Answered By – James Roland

Answer Checked By – Clifford M. (BugsFixing Volunteer)

Leave a Reply

Your email address will not be published. Required fields are marked *