This is a quick one today. I was recently working with a web application that runs a spider to index content on a bunch of intranet sites. The client was a large company with a very security conscious IT team, that had locked down all intranet access from the web server server in question. Luckily Dot Net supports proxies in a loving fashion, and I'll show you how.

As i just mentioned ASP.Net supports using a proxy for its calls out of the box and this is set very simply in the web.config’s <system.net> section. The great thing is it also allows you to exclude some URL’s from using it just like you would in your internet settings on your desktop – even more powerfully it allows these rules to be set using RegEx! How cool.

The section definition is listed on MSDN here.

I have reproduced it below for your pleasure.

<configuration>
    <system.net>
        <defaultProxy>
        <!-- add your proxy address etc -->
            <proxy
              proxyaddress="http://192.168.0.1:8080"
              bypassonlocal="true" />
        <!-- add you bypass urls or Regex's for them below -->
            <bypasslist>
                <add address="[a-z]+\.diaryofaninja\.com" />
            </bypasslist>
        </defaultProxy>
    </system.net>
</configuration>