25 KiB
Unified hosts file with social extensions
This repository consolidates several reputable hosts
files, and merges them
into a unified hosts file with duplicates removed. A variety of tailored hosts files are provided.
- Last updated: April 30 2018.
- Here's the raw hosts file with social extensions containing 55,768 entries.
List of all hosts file variants
The Non GitHub mirror is the link to use for some hosts file managers like Hostsman for Windows that don't work with Github download links.
Host file recipe | Readme | Raw hosts | Unique domains | Non Github mirror |
---|---|---|---|---|
Unified hosts = (adware + malware) | Readme | link | 54,621 | link |
Unified hosts + fakenews | Readme | link | 55,290 | link |
Unified hosts + gambling | Readme | link | 56,133 | link |
Unified hosts + porn | Readme | link | 64,473 | link |
Unified hosts + social | Readme | link | 55,768 | link |
Unified hosts + fakenews + gambling | Readme | link | 56,802 | link |
Unified hosts + fakenews + porn | Readme | link | 65,142 | link |
Unified hosts + fakenews + social | Readme | link | 56,437 | link |
Unified hosts + gambling + porn | Readme | link | 65,985 | link |
Unified hosts + gambling + social | Readme | link | 57,280 | link |
Unified hosts + porn + social | Readme | link | 65,620 | link |
Unified hosts + fakenews + gambling + porn | Readme | link | 66,654 | link |
Unified hosts + fakenews + gambling + social | Readme | link | 57,949 | link |
Unified hosts + fakenews + porn + social | Readme | link | 66,289 | link |
Unified hosts + gambling + porn + social | Readme | link | 67,132 | link |
Unified hosts + fakenews + gambling + porn + social | Readme | link | 67,801 | link |
Expectation: These unified hosts files should serve all devices, regardless of OS.
Sources of hosts data unified in this variant
Updated hosts
files from the following locations are always unified and
included:
Host file source | Description | Home page | Raw hosts | Update frequency | License | Issues |
---|---|---|---|---|---|---|
Steven Black's ad-hoc list | Additional sketch domains as I come across them. | link | raw | occasionally | MIT | issues |
Malware Domain List | Malware Domain List is a non-commercial community project. | link | raw | weekly | 'can be used for free by anyone' | issues |
add.Dead | Dead sites based on hostsfile.org content. | link | raw | occasionally | GPLv3+ | issues |
add.Spam | Spam sites based on hostsfile.org content. | link | raw | occasionally | GPLv3+ | issues |
Dan Pollock – someonewhocares | How to make the internet not suck (as much). | link | raw | frequently | non-commercial with attribution | issues |
MVPS hosts file | The purpose of this site is to provide the user with a high quality custom HOSTS file. | link | raw | monthly | CC BY-NC-SA 4.0 | issues |
yoyo.org | Blocking with ad server and tracking server hostnames. | link | raw | frequently | issues | |
Mitchell Krog's - Badd Boyz Hosts | Sketchy domains and Bad Referrers from my Nginx and Apache Bad Bot and Spam Referrer Blockers | link | raw | weekly | 'non-commercial with attribution' | issues |
CoinBlocker | Simple lists that can help prevent cryptomining in the browser or other applications | link | raw | frequently | GPLv3 | issues |
UncheckyAds | Windows installers ads sources sites based on https://unchecky.com/ content. | link | raw | occasionally | issues | |
add.2o7Net | 2o7Net tracking sites based on hostsfile.org content. | link | raw | occasionally | GPLv3+ | issues |
KADhosts | Fraud/adware/scam websites. | link | raw | frequently | GPLv3 | issues |
AdAway | AdAway is an open source ad blocker for Android using the hosts file. | link | raw | occasionally | CC BY 3.0 | issues |
add.Risk | Risk content sites based on hostsfile.org content. | link | raw | occasionally | GPLv3+ | issues |
Sinfonietta's social media blocking hosts file | A collection of category-specific host files. | link | raw | occasional | MIT | issues |
Extensions
The unified hosts file is extensible. You manage extensions by curating the
extensions/
folder tree. See the fakenews
, social
, gambling
, and porn
extension
folders.
Generate your own unified hosts file
Note if you are using Python 3, please install the dependencies with:
pip3 install --user -r requirements.txt
Note if you are using Python 2, please install the dependencies with:
pip2 install --user -r requirements_python2.txt
Note we recommend the --user
flag which installs the required dependencies at the user level. More information about it can be found on pip documentation.
To run unit tests, in the top level directory, just run:
python testUpdateHostsFile.py
The updateHostsFile.py
script, which is Python 2.7 and Python 3-compatible,
will generate a unified hosts file based on the sources in the local data/
subfolder. The script will prompt you whether it should fetch updated
versions (from locations defined by the update.json
text file in each
source's folder). Otherwise, it will use the hosts
file that's already there.
Usage
Using Python 3:
python3 updateHostsFile.py [--auto] [--replace] [--ip nnn.nnn.nnn.nnn] [--extensions ext1 ext2 ext3]
Using Python 2.7:
python updateHostsFile.py [--auto] [--replace] [--ip nnn.nnn.nnn.nnn] [--extensions ext1 ext2 ext3]
Command line options:
--help
, or -h
: display help.
--auto
, or -a
: run the script without prompting. When --auto
is invoked,
- Hosts data sources, including extensions, are updated.
- No extensions are included by default. Use the
--extensions
or-e
flag to include any you want. - Your active hosts file is not replaced unless you include the
--replace
flag.
--backup
, or -b
: Make a backup of existing hosts file(s) as you generate
over them.
--extensions <ext1> <ext2> <ext3>
, or -e <ext1> <ext2> <ext3>
: the names
of subfolders below the extensions
folder containing additional
category-specific hosts files to include in the amalgamation. Example:
--extensions porn
or -e social porn
.
--flush-dns-cache
, or -f
: skip the prompt for flushing the DNS cache.
Only active when --replace
is also active.
--ip nnn.nnn.nnn.nnn
, or -i nnn.nnn.nnn.nnn
: the IP address to use as the
target. Default is 0.0.0.0
.
--keepdomaincomments
, or -k
: false
(default) or true
, keep the comments
that appear on the same line as domains. The default is false
since some
router-based implementations can't handle comments in-line with hosts.
--noupdate
, or -n
: skip fetching updates from hosts data sources.
--output <subfolder>
, or -o <subfolder>
: place the generated source file
in a subfolder. If the subfolder does not exist, it will be created.
--replace
, or -r
: trigger replacing your active hosts
--skipstatichosts
, or -s
: false
(default) or true
, omit the standard
section at the top, containing lines like 127.0.0.1 localhost
. This is
useful for configuring proximate DNS services on the local network.
--compress
, or -c
: false
(default) or true
, Compress the hosts file
ignoring non-necessary lines (empty lines and comments) and putting multiple
domains in each line. Reducing the number of lines of the hosts file improves
the performances under Windows (with DNS Client service enabled).
--minimise
, or -m
: false
(default) or true
, like --compress
, but puts
each domain on a separate line. This is necessary because many implementations
of URL blockers that rely on hosts
files do not conform to the standard which
allows multiple hosts on a single line.
How do I control which sources are unified?
Add one or more additional sources, each in a subfolder of the data/
folder, and specify the url
key in its update.json
file.
Add one or more optional extensions, which originate from subfolders of the
extensions/
folder. Again the url in update.json
controls where this
extension finds its updates.
Create an optional blacklist
file. The contents of this file (containing a
listing of additional domains in hosts
file format) are appended to the
unified hosts file during the update process. A sample blacklist
is
included, and may be modified as you desire.
- NOTE: The
blacklist
is not tracked by git, so any changes you make won't be overridden when yougit pull
this repo fromorigin
in the future.
How do I include my own custom domain mappings?
If you have custom hosts records, place them in file myhosts
. The contents
of this file are prepended to the unified hosts file during the update
process.
The myhosts
file is not tracked by git, so any changes you make won't be
overridden when you git pull
this repo from origin
in the future.
How do I prevent domains from being included?
The domains you list in the whitelist
file are excluded from the final hosts
file.
The whitelist
uses partial matching. Therefore if you whitelist
google-analytics.com
, that domain and all its subdomains won't be merged
into the final hosts file.
The whitelist
is not tracked by git, so any changes you make won't be
overridden when you git pull
this repo from origin
in the future.
How can I contribute hosts records?
If you discover sketchy domains you feel should be included here, here are some ways to contribute them.
Option 1: contact one of our hosts sources
The best way to get new domains included is to submit an issue to any of the data providers whose home pages are listed here. This is best because once you submit new domains, they will be curated and updated by the dedicated folks who maintain these sources.
Option 2: add your domains to Steven Black's personal data file
Fork this hosts this repo and add your links to https://github.com/StevenBlack/hosts/blob/master/data/StevenBlack/hosts.
Then, submit a pull request.
WARNING: this is less desireable than Option 1 because the ongoing curation falls on us and what you've just done is created more work for us.
Option 3: create your own hosts list as a repo on Github
If you're able to curate your own collection of sketchy domains, then curate your own hosts list. Then signal the existance of your remo as a new issue and we may include your new repo into the collection of sources we pull whenever we create new versions.
What is a hosts file?
A hosts file, named hosts
(with no file extension), is a plain-text file
used by all operating systems to map hostnames to IP addresses.
In most operating systems, the hosts
file is preferential to DNS
.
Therefore if a domain name is resolved by the hosts
file, the request never
leaves your computer.
Having a smart hosts
file goes a long way towards blocking malware, adware,
and other irritants.
For example, to nullify requests to some doubleclick.net servers, adding these lines to your hosts file will do it:
# block doubleClick's servers
0.0.0.0 ad.ae.doubleclick.net
0.0.0.0 ad.ar.doubleclick.net
0.0.0.0 ad.at.doubleclick.net
0.0.0.0 ad.au.doubleclick.net
0.0.0.0 ad.be.doubleclick.net
# etc...
We recommend using 0.0.0.0
instead of 127.0.0.1
Traditionally most host files use 127.0.0.1
, the loopback address, to establish an IP connection to the local machine.
We prefer to use 0.0.0.0
, which is defined as a non-routable meta-address used to designate an invalid, unknown,
or non applicable target.
Using 0.0.0.0
is empirically faster, possibly because there's no wait for a timeout resolution. It also does not
interfere with a web server that may be running on the local PC.
Why not use just 0
instead of 0.0.0.0
?
We tried that. Using 0
doesn't work universally.
Location of your hosts file
To modify your current hosts
file, look for it in the following places and modify it with a text
editor.
Mac OS X, iOS, Android, Linux: /etc/hosts
folder.
Windows: %SystemRoot%\system32\drivers\etc\hosts
folder.
Updating hosts file on Windows
On Linux and Mac OS X, you can simply run the Python script, but on Windows, more work is required due to compatibility issues in implementing some of the functionality for Windows. It is preferable to run the batch file as follows:
updateHostsWindows.bat
This file MUST be run in command prompt with administrator privileges in the repository directory. In addition to updating the hosts file, it can also replace the existing hosts file, and reload the DNS cache. It goes without saying that in order for this to work, you must be connected to the internet.
To open a command prompt as administrator in the repository's directory, do the following:
Windows XP: Start -> Run -> cmd
Windows Vista, 7: Start Button -> type cmd
-> right-click Command Prompt ->
"Run as Administrator"
Windows 8: Start -> Swipe Up -> All Apps -> Windows System -> right-click Command Prompt -> "Run as Administrator"
Windows 10: Start Button -> type cmd
-> right-click Command Prompt ->
"Run as Administrator"
You can also refer to the "Third-Party Hosts Managers" section for further recommended solutions from third parties.
Reloading hosts file
Your operating system will cache DNS lookups. You can either reboot or run the following commands to manually flush your DNS cache once the new hosts file is in place.
The Google Chrome browser may require manually cleaning up its DNS Cache on chrome://net-internals/#dns page to thereafter see the changes in your hosts file. See: https://superuser.com/questions/723703 |
---|
Windows
Open a command prompt with administrator privileges and run this command:
ipconfig /flushdns
If you want to use a huge hosts file by merging hphosts (NOT INCLUDED HERE) you need to DISABLE and STOP Dnscache service before you replace hosts file in Windows Systems. You have been warned. |
---|
Before flushing the DNS cache, open a command prompt with administrator privileges and run this command:
sc config "Dnscache" start= disabled
sc stop "Dnscache"
Linux
Open a Terminal and run with root privileges:
Debian/Ubuntu sudo /etc/rc.d/init.d/nscd restart
Linux with systemd: sudo systemctl restart network.service
Fedora Linux: sudo systemctl restart NetworkManager.service
Arch Linux/Manjaro with Network Manager: sudo systemctl restart NetworkManager.service
Arch Linux/Manjaro with Wicd: sudo systemctl restart wicd.service
Others: Consult this wikipedia article.
Mac OS X
Open a Terminal and run:
sudo dscacheutil -flushcache;sudo killall -HUP mDNSResponder
Release management
This repository uses Release-It!, an excellent CLI release tool for Github repos and npm packages, to automate creating releases. This is why the package.json and .release-it.json files are bundled.
Goals of this unified hosts file
The goals of this repo are to:
-
automatically combine high-quality lists of hosts,
-
provide easy extensions,
-
de-dupe the resultant combined list,
-
and keep the resultant file reasonably sized.
A high-quality source is defined here as one that is actively curated. A hosts source should be frequently updated by its maintainers with both additions and removals. The larger the hosts file, the higher the level of curation is expected.
For example, the (huge) hosts file from hosts-file.net is not included here because it is very large (780,000+ entries) and doesn't currently display a corresponding high level of curation activity.
It is expected that this unified hosts file will serve both desktop and mobile devices under a variety of operating systems.
Third-Party Hosts Managers
- Unified Hosts AutoUpdate (for Windows): The Unified Hosts AutUpdate package is purpose-built for this unified hosts project as well as in active development by community members. It's sophisticated enough to allow any novice the ability to install and uninstall the blacklist of their choosing to their local hosts file and keep it automatically up to date, while also being minimal enough to be able to be easily placed in a shared network location and deployed across an organization via group policies. And since it is in active development by community members, your bug reports, feature requests, and other feedback are most welcome.
Interesting Applications
-
Block ads and malware via local BIND9 DNS server (for Debian, Raspbian & Ubuntu): Set up a local DNS server with a
/etc/bind/named.conf.blocked
file, sourced from here. -
Block ads, malware, and deploy parental controls via local DualServer DNS/DHCP server (for BSD, Windows & Linux): Set up a blacklist for everyone on your network using the power of the unified hosts reformatted for DualServer. And if you're on Windows, this project also maintains an update script to make updating DualServer's blacklist even easier.
-
Blocking ads and malwares with unbound – Unbound is a validating, recursive, and caching DNS resolver.
-
DNSMasq conversion script This github gist has a short shell script (bash, will work on any 'nix) and uses 'wget' & 'awk' present in most distros, to fetch a specified hosts file and convert it the format required by dnsmasq. Supports ipv4 and ipv6. Designed to be used as either a shell script, or can be dropped into /etc/cron.weekly (or wherever suits). Script is short and easily edited, also has a short document attached with notes on dnsmasq setup.