The following tutorial uses Squid bumping to intercept SSL Second Life grid traffic in order to be able to better cache assets. In other words, this tutorial will show you how to use Squid to intercept and decrypt the traffic between your viewer and the SecondLife grid with the purpose of caching assets (textures and others).

You will need Squid v3.5+ with SSL compiled in and with the following compile-time features enabled:

                --enable-ssl \
                --enable-ssl-crtd

If your distribution is Debian, then you can probably follow the tutorial on compiling your own Squid.

First, run the command:

/usr/lib/squid3/ssl_crtd -c -s /var/lib/ssl_db 

in order to create the certificate store.

On Debian, you will have to change the owner of the /var/lib/ssl_db to the proxy user that Squid runs under:

chown -R proxy:proxy /var/lib/ssl_db

The next part is to create the required certificate for SSL bumping by using OpenSSL:

openssl req -new -newkey rsa:2048 -sha256 -days 365 -nodes -x509 -extensions v3_ca -keyout myCA.pem  -out /etc/squid/certs/myCA.pem

Some root certificates can be installed on Debian using:

aptitude install ca-certificates 

to be used in the Squid configuration.

Finally, edit /etc/squid/squid.conf and amend the file with the following:

# Listening Port
http_port 8123 ssl-bump cert=/etc/squid/certs/myCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

# SSL Bump Config
acl bump_domains dstdom_regex -i "/etc/squid/bump_domains.conf"
acl DiscoverSNIHost at_step SslBump1
ssl_bump peek DiscoverSNIHost
ssl_bump bump bump_domains

### SSL Tweaks
# On Debian, from the package ca-certificates - not needed for Squid v4+
sslproxy_cafile /etc/ssl/certs/ca-certificates.crt
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/ssl_db -M 8MB
sslcrtd_children 5

# Do not forward bumped domains through cache peers.
always_direct allow bump_domains
never_direct allow !bump_domains

# Rewrite lindenlab assets.
acl rewritedoms dstdomain .agni.lindenlab.com
store_id_program /etc/squid/secondlife_store_rewrite.py
store_id_children 64 startup=10 idle=5 concurrency=0
store_id_access allow rewritedoms
store_id_access deny all

### SecondLife-specific configuration parameters.
# SecondLife Refresh patterns.
refresh_pattern -i /cap/					129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i asset-cdn\.agni\.lindenlab\.com/.*		129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i asset-cdn\.glb\.agni\.lindenlab\.com/.*	129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload

# Access log
access_log /var/log/squid/access.log buffer-size=1MB
# Useful for debugging store re-writes: shows GET query terms.
strip_query_terms off

Now, all domains listed in /etc/squid/bump_domains.conf will be bumped so create the file /etc/squid/bump_domains.conf with the following contents:

(^|\.)lindenlab..+?$

The store re-write script will grab the UUID of the requested asset (either texture or mesh) and pass it to squid in a more palatable format. Using the configuration from the previous section, the script has to be placed at /etc/squid/secondlife_store_rewrite.py:

#!/usr/bin/python -u
# http://vex.fabulo.us/2015/12/return-of-the-squid-or-how-to-have-a-terabyte-texture-cache-using-squid-3/
import re
import sys
 
# http://sim34232.agni.lindenlab.com/cap/foo/?texture_id=klkjalkjsd
# http://asset-cdn.glb.agni.lindenlab.com/?texture_id=klklkjlkjs
# group 1 is the host - simNNNN or a CDN server
# group 2 is the port on the server
# group 3 cap
# group 4 is either 'texture' or 'mesh'
# group 5 the actual id
texturl_patt = re.compile('(.*)\.agni\.lindenlab\.com(.*)/(.*/?)\?(texture|mesh)_id=(.*)')
 
def rewrite(texturl):
    m = texturl_patt.search(texturl)
    if m is not None:
        r = 'http://{}.lindenlab.com/{}'.format(m.group(4),m.group(5))
        return r
    else:
        return None
 
def process(line):
    if line == 'quit':
        sys.exit(0)
 
    stuff = line.split()
    if stuff[0].isdigit():
        r = rewrite(stuff[1])
        if r is not None:
            print('{} OK store-id={}'.format(stuff[0], r))
        else:
            print('{} ERR'.format(stuff[0]))
    else:
        r=rewrite(stuff[0])
        if r is not None:
            print('OK store-id={}'.format(r))
        else:
            print('ERR')
 
while True:
    line = sys.stdin.readline().strip()
    if not line:
        break
    process(line)

and granted execute permissions: chmod +x /etc/squid/secondlife_store_rewrite.py.

You can now save all configurations and restart Squid.

To make the viewer use the Squid proxy, and assuming the configuration from the previous section, a few parameters have to be set.

• Preferences→Web→Web Proxy→Enable Web Proxy - should be ticked.
• Preferences→Web→Web Proxy→Address - fill in the address and port on which Squid is listening.

• Preferences→Network→Custom Port Connection - this should be set to the same http_port configured for Squid (following the example, this should be 8123)
• Preferences→Network→Http Proxy - should be set to Web because Squid does not support SOCKS connections.

If you save the settings now and attempt to connect, you will get and error telling you that the server certificate cannot be verified. This is because Squid is bumping the connection using the myCA.pem certificate. To resolve this issue, you can go to:

• Advanced→Debug Settings

and type in: NoVerifySSLCert and set the toggle to True. The warning is fair, because intercepting the SSL session is exactly what you want to accomplish with Squid.

You should now be able to connect to SecondLife through Squid.

This section is viewer-specific but the general guideline is to enable:

• HTTP inventory

• HTTP textures

such that all all requests for inventory or textures will flow through Squid.

You should now be able to monitor /var/log/squid/access.log to see all the requests that the viewer is making whilst minding for TCP_MISS (asset not cached) and TCP_HIT or TCP_MEM_HIT indicating that the asset was previously cached.

If the option strip_query_terms has been set to off, you should be able to see the GET request parameters which is useful for amending the secondlife_store_rewrite.py script.