User Tools
misc:2015brainstormpress
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
misc:2015brainstormpress [2015/02/20 15:18] tschulz |
misc:2015brainstormpress [2015/02/20 15:41] (current) tschulz [Detailed Caching Install Notes] |
||
|---|---|---|---|
| Line 12: | Line 12: | ||
| * Debian/Ubuntu updates (apt-get) | * Debian/Ubuntu updates (apt-get) | ||
| * Steam game updates (storeid will be need to handle CDNS) | * Steam game updates (storeid will be need to handle CDNS) | ||
| + | * adobe (flash/shockwave/reader) | ||
| + | * Oracle (java) | ||
| ===== What updates can't be cached ===== | ===== What updates can't be cached ===== | ||
| Line 21: | Line 23: | ||
| * Squid 3.4 or higher (3.5.1 compiled/sorry no package:)) | * Squid 3.4 or higher (3.5.1 compiled/sorry no package:)) | ||
| * recommend setting up cache server as transparent proxy (enable IPv4 packet forwarding, forward port 80 to squid) | * recommend setting up cache server as transparent proxy (enable IPv4 packet forwarding, forward port 80 to squid) | ||
| + | * ZFS raidz works really well for cache store | ||
| ===== Squid notable config changes ===== | ===== Squid notable config changes ===== | ||
| - | * <file>http_port 3129 intercept</file> | + | * Enable Transparent Proxying<file>http_port 3129 intercept</file> |
| - | * <file>maximum_object_size 5000 MB | + | * Set Max Object Size to something large<file>maximum_object_size 5000 MB |
| range_offset_limit -1 | range_offset_limit -1 | ||
| quick_abort_min -1 KB | quick_abort_min -1 KB | ||
| </file> | </file> | ||
| - | * <file>cache_dir ufs /var/spool/squid 50000 16 256</file> | + | * Set a large cache store<file>cache_dir ufs /var/spool/squid 50000 16 256</file> |
| - | * <file>acl updatesites dstdom_regex "/etc/squid/updatesites.txt" | + | * Only allow caching from update sites<file>acl updatesites dstdom_regex "/etc/squid/updatesites.txt" |
| cache allow updatesites | cache allow updatesites | ||
| cache deny all</file> | cache deny all</file> | ||
| - | * <file>store_id_access allow updatesites | + | * Setup store_id for CDN Matching<file>store_id_access allow updatesites |
| #store_id_access allow steamcdn | #store_id_access allow steamcdn | ||
| store_id_access deny all | store_id_access deny all | ||
| store_id_program /usr/lib/squid/storeid_file_rewrite /etc/squid/storeid_rewrite.conf | store_id_program /usr/lib/squid/storeid_file_rewrite /etc/squid/storeid_rewrite.conf | ||
| store_id_children 10 startup=3 idle=1 concurrency=0</file> | store_id_children 10 startup=3 idle=1 concurrency=0</file> | ||
| - | * <file>refresh_pattern ([^.]+.|)adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overr | + | * Override Cache timeouts and other cache HTTP headers<file>refresh_pattern ([^.]+.|)adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overr |
| refresh_pattern ([^.]+.|)java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overri | refresh_pattern ([^.]+.|)java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overri | ||
| refresh_pattern ([^.]+.|)sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overrid | refresh_pattern ([^.]+.|)sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire overrid | ||
| Line 60: | Line 63: | ||
| ===== Other Squid configs ===== | ===== Other Squid configs ===== | ||
| - | * <file storeid_rewrite.conf>^http:\/\/.+?\.ws\.microsoft\.com\/.+?_([0-9a-z]{40})\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf|appx) http://wupdate.squid.local/$1 | + | * <file config storeid_rewrite.conf>^http:\/\/.+?\.ws\.microsoft\.com\/.+?_([0-9a-z]{40})\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf|appx) http://wupdate.squid.local/$1 |
| ^http:\/\/.+?\.windowsupdate\.com\/.+?_([0-9a-z]{40})\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf|appx) http://wupdate.squid.local/$1 | ^http:\/\/.+?\.windowsupdate\.com\/.+?_([0-9a-z]{40})\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf|appx) http://wupdate.squid.local/$1 | ||
| ^http:\/\/[^\.]+\.cs\.steampowered\.com\/(.*) http://steamupdates.squid.internal/$1 | ^http:\/\/[^\.]+\.cs\.steampowered\.com\/(.*) http://steamupdates.squid.internal/$1 | ||
| ^http:\/\/[^\.]+\.phobos\.apple\.com\/(.*) http://appupdates.apple.squid.internal/$1 | ^http:\/\/[^\.]+\.phobos\.apple\.com\/(.*) http://appupdates.apple.squid.internal/$1 | ||
| </file> | </file> | ||
| + | * <file config updatesites.txt>([^.]+.|)adobe.com | ||
| + | ([^.]+.|)java.com | ||
| + | ([^.]+.|)sun.com | ||
| + | ([^.]+.|)oracle.com | ||
| + | ([^.]+.|)apple.com | ||
| + | ([^.]+.|)apple.com | ||
| + | ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com | ||
| + | ([^.]+.|)ubuntu.com | ||
| + | ([^.]+.|)steampowered.com | ||
| + | ([^.]+.|)google.com | ||
| + | ([^.]+.|)gstatic.com | ||
| + | </file> | ||
| + | * <file bash rc.local>iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to 10.1.0.21:3129 | ||
| + | exit 0 | ||
| + | </file> | ||
| + | * <file bash sysctl.conf>... | ||
| + | net.ipv4.ip_forward=1 | ||
| + | ...</file> | ||
| + | |||
| + | |||
| + | ===== Detailed Caching Install Notes ===== | ||
| + | * [[web_services:squid_update_cache|Squid Based Update Cache (Ubuntu 14.04)]] | ||
| + | * [[fileservices:zfs_ubuntu|ZFS on Ubuntu]] | ||
misc/2015brainstormpress.1424467139.txt.gz · Last modified: 2015/02/20 15:18 by tschulz
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
