Speeding up Internet Surfing (Squid + BIND)

I am not writing something very much ground-breaking, but since friends do ask me how to use their slow internet lines, here is the solution: Web Proxy with Cache and DNS Caching.

You might find exhaustive articles all over the net, this one is which I made from my experience. Getting them to work is so so simple. It isn’t a rocket science as you might be thinking.

You basically need Squid to act as Web Server Proxy and Bind9 as DNS Caching.  The work of Bind9 and squid is not limited just for caching, but you might want to know that Bind powers much of world’s DNS servers. Let’s start

You first need squid and squid-common packages for squid and bind9 and dnsutils for bind.

sudo apt-get install squid bind9

squid-common and dnsutils are dependencies of the the two, so you don’t need to specify

BIND

Just open the file named.conf.options in the folder /etc/bind and change the following section

// forwarders {
// 0.0.0.0;
// }

to

forwarders {
208.67.222.222;
208.67.220.220;
};

Here the two IP addresses are those of the DNS servers (OpenDNS in this case). You can even use the IP address of your router if the latter is configured for DNS.

Now restart bind daemon

sudo /etc/init.d/bind9 restart

Bind can also be used for more than just DNS caching. It can be used for Primary Master Server, Secondary Master Server or mix of all of them. The slightly geekish HOWTO is here. It is written for  Ubuntu, but nearly same for all other distros.

Squid

When you installed squid, it has been auto-configured as a caching proxy server. You just need to point your browser’s proxy settings to it. The best way is to set up Network Proxy so that every application can use it.

System > Preferences > Network Proxy

GNOME Proxy Settings
GNOME Proxy Settings
forwarders {
208.67.222.222;

Squid uses port no 3128 as default. You can change it later.

If you want Squid just to surf the internet, then can just change the proxy settings of Firefox.

Edit > Preferences > Advanced > Network

Choose the Settings Button, set up the proxy settings similar to that shown inthe above figure.

You can check the official docs at squid for configuring squid or use this simplified doc at Ubuntu Wiki. Again, it works on all the distros.

Using squid you can customized error pages, shut down network for specified duration of time, block websites and many more.

Suggested Reading:

Advertisements

19 thoughts on “Speeding up Internet Surfing (Squid + BIND)

    1. I also thought of using it, but put it off the list after reading this
      https://help.ubuntu.com/community/Dnsmasq

      It says, that

      Note that the package “dnsmasq” interferes with Network Manager which can use “dnsmasq-base” to provide DHCP services when sharing an internet connection. Therefore, if you use network manager (fine in simple set-ups only), then install dnsmasq-base, but not dnsmasq. If you have a more complicated set-up, uninstall network manager, use dnsmasq, or similar software (bind9, dhcpd, etc), and configure things by hand.

      Any workaround for it? I don’t want to uninstall NetworkManager. Not because I hate doing things manually, but because when things can be done using NetworkManager, then why not use it?

      1. Yes, it interferes and really badly at that. Remember how I had problems connecting to wifi? So much that I had to uninstall network manager.

        1. NetworkManager is an unusual piece of software!! When it works, it works perfectly. When it refuses, even lord can’t make it budge. 😦

          I do remember your situation of wifi. You spent hours connecting to wifi. Sometimes it was hilarious.

  1. “Just open the file named.conf.options and change the following section”—this file is located under /etc/bind.

    nice and small walkthrough..

    thanks for it!

  2. Just immigrate to USA if u need fast internet connection (hilarious and dumb solution I know but sorry I had to say it.)

    But yes if you want real super-duper ultra-fast connection, move to korea or japan. When cable internet connection was just invented and getting ready to be rolled out to the masses, I was highly impressed with Tokyo’s basic DSL connection which was churning out an impressive 300 kb/s at all times.

    1. In India, we also have fast internet connections, but it is not widely available. Plus, laptop is meant to be portable and you don’t know that the next place you visit is going to have a decent sane internet connection.

      The other face of this post is that when you can save bandwidth, then why shouldn’t you? If everyone cuts down this over heard, I think we would have more bandwidth in this world.

  3. check your squid logs. what is your cache hit rate? i bet it is nearly 0%. this is because your browser already caches whatever content it can

    squid only pays off as a central proxy in a larger network environment where many users go through the same proxy. in a home environment it is only an additional overhead on your system, but it is no smarter than the browser cache.

    1. Jack,
      There is another angle. I use more than one browser – Epiphany and Opera. Squid helps in this case.

      I can feel the improved speed, not a drastic one, but still there is improvement. The time delay has been minimised between I hit Enter and loading starts.

      I use sarg and analysed the logs. The logs were so exhaustive that I closed it. You know a better method to analyse them?

      1. You could try something like:

        grep TCP_MISS /var/log/squid/access.log | wc -l

        and

        grep TCP_HIT /var/log/squid/access.log | wc -l

        The 2 commands will give you the cache misses and hits, so you can see how many times squid actually saved you a fetch from the internet.

        I would be interested to see if you actually get a good result. I tried many different tweaks over many weeks and the result was always the same. This was a few years ago now, so maybe something has changed.

        1. $ grep TCP_MISS /var/log/squid/access.log | wc -l
          17989

          $ grep TCP_HIT /var/log/squid/access.log | wc -l
          674

          Not very encouraging! 😦

          Doesn’t look like something really big chances have been made since you last tried.

        2. I think the speed gain is probably due to DNS caching rather than Squid.

          If I set it to do more aggressive caching, then the browsing experience may be hampered as it might fetch the older page even when the page has actually been changed.
          This problem might be somewhat like I found in Drupal. I set aggressive caching in Admin Panel and always the older page was fetched.

  4. > forwarders {
    > 208.67.222.222;
    > 208.67.220.220;
    > };
    >
    > Here the two IP addresses are those of
    > the DNS servers (OpenDNS in this case).

    OpenDNS are just another web2.0 version of spyware – they mooch at what you are doing, make judgements on what you should or shouldn’t be able to access, and use the knowledge to advertise (ie use the teachings of psychologists to make us go along with their business interests even (especially) when the business interest is not in our interests).

    But one of the worst things for a DNS provider is that they provide non-compliant DNS! Their POS service will give you an IP for domain names that do not exist – this is so they can spam your face if you typo an URL in a browser, but it also means that if you typo an email address it could end up with OpenDNS instead of bouncing!

    If a user is setting up bind why not just use their ISPs DNS servers (the latency WILL be lower than opendns or Google) as forwarders, or just use bind as a fully-fledged DNS server.

    Personally I ended up on this article because I am on a shitty ISP (Three in the UK) who censor and tend to have far too much packet loss, especially when a big download or torrent is going, and I am looking for a way to make bind less patient. I want it to pretty much attack my ISPs DNS servers asking for DNS as I am sick of time-outs. No answer within half a second or so, ask again, etc..

    An illustration of the problem with OpenDNS:

    $ host doesntexist.fuckwit. 208.67.222.222

    doesntexist.fuckwit has address 67.215.65.132
    doesntexist.fuckwit has IPv6 address ::ffff:67.215.65.132
    ….

  5. Hoteluri Munte

    Hi there are using WordPress for your site platform? I’m new to the blog world but I’m trying
    to get started and set up my own. Do you require any html coding knowledge
    to make your own blog? Any help would be really appreciated!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s