Marown/hosts
1
0
Fork 0
mirror of https://github.com/StevenBlack/hosts.git synced 2024-07-07 13:02:27 +02:00
Code Issues Packages Projects Releases Wiki Activity
2,981 Commits 1 Branch 657 Tags 1.6 GiB
1a293025de
Commit Graph

304 Commits

Author SHA1 Message Date
funilrys
079ad6b674
Fix test issue. 
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347497718#L748
 2018-02-28 23:13:13 +01:00
funilrys
c82f691952
Review of get_file_by_url() 
Please note that this patch also introduce
which is in charge of converting a domain in a line into
IDNA and/or UTF-8 format.

Also note the introduction of BeautifulSoup() which helps
us to decode data from the downloaded URL.

Fixes (issue(s)/protocol(s) I was able to reproduce):
 * https://github.com/StevenBlack/hosts/issues/514#issuecomment-368932152

Possible fix of (issue(s)/protocol(s) I wasn't able to reproduce):
 * https://github.com/StevenBlack/hosts/issues/514#issue-300048106
 * https://github.com/StevenBlack/hosts/issues/494#issue-296166492
 * https://github.com/StevenBlack/hosts/issues/420#issue-267453114
 * https://github.com/StevenBlack/hosts/issues/372#issue-246927047
 * https://github.com/StevenBlack/hosts/issues/382#issuecomment-322010562
 2018-02-28 23:06:58 +01:00
FadeMind
33ba41bbea
updateHostsFile.py: cross-compatible sudo 2018-02-27 19:26:40 +01:00
Steven Black
007254e10f
Merge pull request #503 from ankitpati/minimise-hosts-file 
Add Option to Minimise hosts File
 2018-02-16 22:07:47 -05:00
Ankit Pati
01db326459
Add Option to Minimise hosts File 
This option (--minimise, -m) differs from compress (--compress, -c) in
that it keeps each host on a different line, while still removing
unnecessary newlines and comments.

This is necessary because many implementations of URL blockers that rely
on hosts files do not conform to the standard which allows multiple
hosts on a single line.
 2018-02-17 00:22:25 +05:30
Ankit Pati
db29acda65
FIX: Remove Hardcoded Offset in Compression 
An offset of 7 was hardcoded in the function compress_file, presumably
to skip over the default target IP address of 0.0.0.0 in a hosts file.
However, this causes problems when the default is overridden using the
--ip or -i flag, causing visibly garbled output in the generated hosts
file.

Fix is to calculate the length of target IP at runtime.
 2018-02-16 23:45:10 +05:30
FadeMind
5b5e854c7f
updateHostsFile.py: add ipv6 additionals 
@StevenBlack.
DEFAULT hosts file in Linux

127.0.0.1       localhost
127.0.1.1       mydevicename
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
 2018-02-09 19:18:54 +01:00
Stefano
d4dadf6ca3 Fixed the number of domains in each line. 
Fixed the number of domains in each line and added the support to
inline comments (they will be ignored as the comment lines).
Code refactoring.
 2018-01-02 22:35:27 +01:00
Stefano
510109d6d4 Updated the documentation with the new compress option. 
Removed a redundant skipstatichosts option.
 2018-01-02 10:51:26 +01:00
Stefano
ff9e05ffec Added an option to compress the created hosts file. 
In particular, the compression option removes non-necessary lines (empty lines and comments) and puts multiple domains in each line.
This option should solve the issue #411 regarding the DNS client service of Windows.
 2017-12-30 20:55:12 +01:00
Stefano
ce8d3abf7e Added an option to compress the created host file. 
In particular, the compression option removes non-necessary lines (empty lines and comments) and puts multiple domains in each line.
This option should solve the issue #411 regarding the DNS client service of Windows.
 2017-12-30 17:12:04 +01:00
Steven Black
bb5d174866 Fix linter complaint. 2017-12-18 23:57:02 -05:00
Steven Black
04a9a712b3 Issue #180: fix - support JSON array transformation for hosts sources. 2017-12-17 22:06:05 -05:00
gfyoung
24ab22e139
Refactor out source data updating 2017-08-20 11:52:28 -07:00
Steven Black
68dcf5c125 Remove references to .zip files in the generator programs. 2017-08-20 12:00:19 -04:00
gfyoung
f83a56d317 Refactor out global settings usage in update logic 2017-08-09 08:13:22 -07:00
gfyoung
942069db81 Refactor out global settings usage in exclusions 2017-08-07 08:43:47 -07:00
gfyoung
4b96f3f34a Refactor out global settings usage in user prompt 2017-07-13 08:20:03 -07:00
Steven Black
d945d7bcc0 Merge pull request #341 from indrajitr/hostmatch-fix 
Honor whitelist exclusion only for domain or subdomain match
 2017-07-09 13:13:57 -04:00
Dylan Armstrong
02a28cd05b Fix the number of unique entries. 2017-07-08 20:20:03 -05:00
gfyoung
415b89a022 Reduce dependency on global settings variable 
Global variables make code less modular
and therefore more difficult to test.
 2017-07-03 18:42:15 -07:00
gfyoung
dce24af399 Add unittests for updateHostsFile.py 2017-06-28 20:13:01 -07:00
Indrajit Raychaudhuri
fd046bc460 Honor whitelist exclusion only for domain or subdomain match 
* Whitelisting `s.youtube.com` should not whitelist `ads.youtube.com`; but
  can whitelist `a.s.youtube.com`
* Whitelisting `twitter.com` should not whitelist `ttwitter.com` or
  `twitter.com.au`; but can whitelist `a.twitter.com`
 2017-06-22 01:43:56 -05:00
Indrajit Raychaudhuri
7e1b06ecf9 Tighten regex rule to discard IP address as hostname 2017-06-20 02:06:31 -05:00
Steven Black
96ac61c119 Merge pull request #325 from gfyoung/terminal-color-support 
Check if terminal supports color
 2017-05-29 23:00:35 -04:00
gfyoung
2d6a8676a4 Check if terminal supports color 
Closes gh-151.
 2017-05-28 12:40:32 -04:00
gfyoung
3b581ce782 Refactor DNS cache flushing for Linux 2017-05-28 12:40:25 -04:00
gfyoung
51a0ec21bd Document all functions in Python scripts 2017-05-23 21:58:22 -04:00
Steven Black
526a457a38 Merge pull request #322 from gfyoung/kde-neon-dns-cache 
Check /usr/lib and /lib for systemd
 2017-05-23 10:03:22 -04:00
gfyoung
2aecbf9707 Concatenate str only in os.path.join 
Closes gh-316.
 2017-05-22 23:31:35 -04:00
gfyoung
8806131c74 Check /usr/lib and /lib for systemd 
Closes gh-321.
 2017-05-22 21:19:31 -04:00
gfyoung
8f3e71e327 Wrap os.path.join to handle LOCALE issues 
Closes gh-81.
Closes gh-318.
 2017-05-21 21:51:13 -04:00
Steven Black
d343798ed1 Ref: Issue #312 undo commit 2ff5dc2 
cc @gfyoung

Commit 2ff5dc2 is generating errors on macOS.

See Issue #318.
 2017-05-21 21:34:54 -04:00
gfyoung
c8e8ddee6c Wrap os.path.join to handle LOCALE issues 
Closes gh-81.
 2017-05-19 16:27:12 -04:00
Steven Black
f87290eff8 Merge pull request #310 from gfyoung/ubuntu-mint-dns-cache 
Add DNS cache flushing calls for Linux
 2017-05-19 07:30:54 -04:00
gfyoung
b3838ef2d2 Add DNS cache flushing calls for Linux 
1) Add command for Linux Mint
2) Add additional command for NCSD cache

Closes gh-128.
 2017-05-18 11:30:23 -04:00
gfyoung
8cb610a757 Don't crash if data source unreachable 
Closes gh-26.
 2017-05-18 00:17:11 -04:00
gfyoung
26cac85ca3 Patch all style errors in updateHostsFile.py 2017-05-15 15:00:16 -04:00
gfyoung
21dd1d9d56 Standardize underscore names in updateHostsFile.py 2017-05-15 13:32:39 -04:00
gfyoung
0015363385 Condense Python 2-3 compatibility code 2017-05-14 22:30:36 -04:00
gfyoung
190592577d Condense exclusion customization logic 2017-05-14 22:10:19 -04:00
gfyoung
ceb3765d24 Flush DNS cache on Windows 2017-05-14 22:07:43 -04:00
gfyoung
118a8e78a5 Remove unnecessary space between print and parenthesis 2017-05-14 14:54:24 -04:00
Steven Black
83af37862c Merge pull request #292 from gfyoung/remove-unused-imports 
Remove unnecessary imports in updateHostsFile.py
 2017-05-14 11:03:22 -04:00
gfyoung
5830bb8db0 Have query_yes_no return boolean 
Bugs have been found where we assumed
that query_yes_no returned True/False.
Not only is it easier to check booleans
instead of strings, but it also looks
a lot cleaner in the code.
 2017-05-14 00:15:58 -04:00
gfyoung
de2e45e8b2 Remove unnecessary imports in updateHostsFile.py 2017-05-13 23:43:00 -04:00
gfyoung
bdce1217bb Patch response checking in promptForFlushDnsCache 
Previously, it was checking if query_yes_no returned
False, which does not happen, as the function only
returns "yes" or "no". As a result, the check was
always returning True, even when the user did not
want to flush the DNS cache.
 2017-05-08 01:10:40 -04:00
DPTJKKVH
f9e8910ba0 Include 127.0.0.53 <hostname> to fix DNS issues from Ubuntu 17.04 onwards 
Ubuntu 17.04 replaces dnsmasq with systemd-resolve which uses a different local IP address for domain name resolution.
By including this simple line of code we basically do the same what Issue #110 requested to fix Issue #109 but this time for systemd-resolve.
 2017-04-23 11:20:29 +02:00
Tomasz Przybył
71f2b8ee3f [FIX] traceroute and netstat display anomaly 
Based on hosts file generated by Uncheky app
@StevenBlack
 2017-04-13 19:21:10 +02:00
Steven Black
590b62111e Trim periods in addition to whitespace. Closes #271. 2017-03-30 12:08:50 -04:00
Steven Black
3ccf4cca0f Issue #243: fix - don’t include domain comments, by default. 2017-02-17 22:29:58 -05:00
Steven Black
85babd9720 Recursively collect files in subfolvers in Python versions below 3.5. This closes Issue #218. 2016-12-18 23:17:01 -05:00
Steven Black
db58b6db95 Allow nesting of base hosts data, and within extension folders. 2016-12-18 19:57:55 -05:00
Steven Black
d39ad733dc ‘hostsFile’ is a better variable name here than ‘dataFile’. 2016-12-18 18:34:41 -05:00
Steven Black
ad2100aaf2 ‘hostsfilename’ is a better variable name than ‘datafilenames’. 2016-12-18 18:32:44 -05:00
Steven Black
ac2a06f74b Updating all sources: add recursive=True to the call to glob() 2016-12-18 17:55:55 -05:00
Steven Black
c022c2b230 Update data regardless fo folder depth using glob.glob(). 2016-12-18 11:02:21 -05:00
Steven Black
d1c3921eb7 Restore updateHostsFile.py to 7119b725e7 because the prior commit’s diff is cluded by space/tab changes. 2016-12-18 10:55:35 -05:00
Steven Black
d3b8cc7339 Refactoring in updateHostsFile.py 2016-12-18 00:34:12 -05:00
Steven Black
7119b725e7 Added metadata to Sinfonietta’s hosts source files. 2016-12-11 13:44:51 -05:00
Steven Black
e22e3dc594 Create a sourcesdata hash for each flavor of hosts file generated. 2016-12-11 12:22:51 -05:00
Steven Black
3f4bfa2d70 Code commenting. 2016-12-11 12:22:02 -05:00
Steven Black
d089583a45 This code is never called. 2016-12-11 12:21:38 -05:00
Steven Black
962010ddcc Issue #208: fix - save the source data to populate the reame files. 2016-12-10 23:52:42 -05:00
Tom Hoover
1e4de72b65 add an optional blacklist file 
The contents of this file (containing a listing of additional
domains in 'hosts' file format) are appended to the unified hosts
file during the update process.
 2016-12-10 14:17:56 -06:00
Steven Black
a6fcff9f22 Change update.info files to update.json, update the python generators, and docs. 2016-12-09 20:59:43 -05:00
Steven Black
f2f6a18440 Issue #188: Optionally create a hosts.zip file in addition to the hosts file. 2016-10-16 23:07:06 -04:00
cryzed
85ada890ba Handle the auto flag in promptForFlushDnsCache 2016-06-13 02:13:36 +02:00
cryzed
fd6769005e Implement feature proposed in #138 2016-06-12 14:04:38 +02:00
cclauss
b20fa35eba Update updateHostsFile.py 2016-05-02 10:59:54 +02:00
cclauss
854f2410a1 Update updateHostsFile.py 2016-04-26 17:52:00 +02:00
cclauss
1b46ffd85b Update updateHostsFile.py 2016-04-26 16:11:51 +02:00
cclauss
9ec9f22dda Update updateHostsFile.py 2016-04-26 16:03:36 +02:00
cclauss
46877f22c4 Update updateHostsFile.py 2016-04-26 15:49:31 +02:00
cclauss
5e08804cec Use 'with open() as' to automaticly close() files 2016-04-26 15:46:14 +02:00
Indrajit Raychaudhuri
8c089ff0ef Add option to skip static localhost entries in the final hosts file. 
This is particularly handy when you want to add the generated hosts file
as additional static 'hosts' file in DNS servers without affecting the default
`/etc/hosts`.

For example, `--addn-hosts=<file>` in dnsmasq [1] can be used exactly for this
purpose.

[1] http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html
 2016-04-19 14:44:39 -05:00
Steven Black
3b30ff8000 Sort the display of --help options. 2016-04-03 21:30:46 -04:00
Steven Black
b194e25020 Now the backup feature is optional, default is no backup. 2016-04-03 21:29:47 -04:00
Steven Black
6f60a86fad Typo in settings key. Fixes #126. 2016-04-03 21:19:41 -04:00
Steven Black
60c43d299b Issue #118: fix – resolve problems with whitelisting domains. 2016-04-01 23:49:22 -04:00
itachi46
6f1484a01c Fixed file not found error due to relative path 
When updateHostFile.py is not run from the project directory we get a file not found error trying to find readmeData.json
 2016-03-31 23:35:42 +11:00
Steven Black
6dc8c77b64 Tweak for updatehostsfile.py which wasn't updating extension sources. 2016-03-28 23:04:34 -04:00
Steven Black
ef37166285 Style: normalize the code to double quotes for strings. 2016-03-28 13:27:17 -04:00
Steven Black
b996001c95 Scope management. 2016-03-25 23:45:18 -04:00
Steven Black
92e8e4d8d9 Whitespace. 2016-03-25 23:44:54 -04:00
Steven Black
2d44d18af6 Jettison legacy codeblock. 2016-03-25 23:44:22 -04:00
Steven Black
30e5d843d4 Issue #77: fix bug with emitting comments. 2016-03-24 23:34:28 -04:00
Steven Black
94ea41fa16 Issue #77: fix — account for empty lines. 2016-03-24 18:32:46 -04:00
Steven Black
883faf0f64 Issue #77: fix - trim trailing whitespace from input lines. 2016-03-24 18:27:24 -04:00
Steven Black
97ef0fdded Issue #117 — fix: restart network.service if applicable. 2016-03-24 18:07:17 -04:00
Steven Black
a0a884c2de Better header section in the unified hosts file. 2016-03-24 10:29:51 -04:00
Tomasz Przybył
9ff912af5f fix typo in IP address @ 392 line 2016-03-24 10:54:23 +01:00
Steven Black
5f7c9cbd1c Global ectomy. 2016-03-24 01:10:13 -04:00
Steven Black
c2aed66c83 Moved the Python3 test earlier. 2016-03-23 23:36:47 -04:00
Steven Black
d1da3e6638 Function updateReadme is now orphaned. 2016-03-23 21:26:20 -04:00
Steven Black
97fba05f22 Nor update the readmeData.json file for each build. 2016-03-23 00:36:01 -04:00
Steven Black
8212663c8d Merge pull request #105 from itachi46/improve-dns-flush 
Added dnsmasq and better messaging
 2016-03-20 12:47:08 -04:00
Steven Black
f18700c676 Merge pull request #110 from matkoniecz/hostname 
include 127.0.1.1 <hostname> line
 2016-03-20 12:17:41 -04:00
Steven Black
4e0e298217 Merge pull request #112 from matkoniecz/refuse_update 
make -n working not only in combination with -a
 2016-03-20 12:16:38 -04:00
xXKingRalphXx
86df89d33e Shortened python version check in updateHostsFile.py 2016-03-19 22:17:36 -04:00
itachi46
0649aa1c8c Changed variable 
changed variable name of dnsFlushedOccured to dnsCacheFound to better indicate what the value represents
 2016-03-19 12:35:22 +11:00
Mateusz Konieczny
6395934dfa include 127.0.1.1 <hostname> line 
partial, maybe full fix for #109
 2016-03-18 21:32:45 +01:00
Mateusz Konieczny
4ce0fbdf3f make -n working not only in combination with -a 
fixes #107
 2016-03-18 21:27:09 +01:00
itachi46
408069da79 Added dnsmasq and better messaging 
* Added restarting dnsmasq as a way to flush DNS cache
* Added success messages to services being restarted
* Added an error message if none of the conditions were met instead of
silently failing

I did not really touch OSX side of the things as I am not sure what the
command does and I do not have access to an OSX box.
 2016-03-18 18:24:04 +11:00
Steven Black
190a53c4cf Note the number of unique domains in each hosts file's header. 2016-03-16 00:33:49 -04:00
Steven Black
c565266416 Making it work in Python3. 2016-03-13 11:59:25 -04:00
Steven Black
31ea0fc25b Fixes #102. 2016-03-13 11:24:17 -04:00
Steven Black
e946d6fa8f Improved the readme.nmd files in subfolders. 2016-03-13 03:02:26 -04:00
Steven Black
9923ed2194 Create different readme.md for different extension combinations. 2016-03-13 01:47:56 -05:00
Steven Black
a081330fea New option to pre-empt data source updates. 2016-03-12 23:44:49 -05:00
Steven Black
d67a7bc3be Create the output folder if it doesn't pre-exist. 2016-03-12 23:44:23 -05:00
Steven Black
ec5178b7a5 Adding optional output path argument. 2016-03-12 23:17:37 -05:00
Steven Black
3de2d618c8 Issue #92: fix - update.info is now treated as a list. 
Can now have several URLs, one per line, in update.info files.
 2016-03-05 20:28:32 -05:00
Steven Black
76fe64ba90 Amalgamated -> Unified. 2016-02-27 20:16:41 -05:00
Steven Black
bee5292c80 Improve amalgaated hosts file heading. 2016-02-26 22:55:07 -05:00
Steven Black
513bdd1ce8 Only process source folders, and not files that may be present. 2016-02-23 09:50:20 -05:00
Steven Black
01915623a8 Typo. 2016-02-23 09:36:33 -05:00
Steven Black
a3ac83b67d More succinct formatting of the generation time. 2016-02-23 09:30:34 -05:00
Steven Black
ffc17dd7fa Issue #57: fix - ignore blank lines in the whitelist. 2016-02-22 21:48:02 -05:00
Steven Black
b96412650a New command-line option, --replace, to activate host file replacement when in --auto mode. 2016-02-22 21:26:52 -05:00
Steven Black
3dde2e39d2 Changes to support extensions. 2016-02-22 19:20:20 -05:00
Steven Black
40eb85a797 First code alterations for extensions. 
This is still WIP.
 2016-02-22 10:22:26 -05:00
Steven Black
223b9409da Issue #91: fix - make the date and time windows filesystem-compatible. 2016-02-21 20:55:38 -05:00
Steven Black
e8033bff72 Issue #91: fix - hoping this works better under windows. 2016-02-21 11:37:03 -05:00
Steven Black
012c6a49b0 Add 'fe80::1%lo0 localhost' which is in the OS X default hosts file. 
Partly addresses Issue #86 and its arising discussion.
 2016-02-20 19:33:29 -05:00
Steven Black
76ea0b54c8 Fix rendering of the 'local' domain found in someonewhocares.org. Resolves #89. 2016-02-19 10:12:26 -05:00
Steven Black
ab129243b8 Add localhost.localdomain and broadcasthost to the base hosts. This fixes Issue #87. 2016-02-19 08:10:03 -05:00
Steven Black
691a3f5e2e Use 'GMT' instead of +0000 notation. 2016-02-16 11:26:17 -05:00
Steven Black
f57714c1d7 Better way to express the updated date. 2016-02-16 11:24:45 -05:00
Steven Black
c28bef1376 Update the readme to include a freshness date. 2016-02-16 11:21:45 -05:00
Steven Black
af4c7865b5 Issue #75: pass-in the target IP address as an option. 
Use option --ip nnn.nnn.nnn.nnn or -i nnn.nnn.nnn.nnn to specify the IP address you want in the amalgamated hosts file.
 2016-02-15 17:06:38 -05:00
Steven Black
35a16da7a1 TARGET_HOST is now targetIP. 2016-02-15 16:16:42 -05:00
Steven Black
e3a11f0d59 Removed needless line feed in header. 2016-02-15 16:11:30 -05:00
Tomasz Przybył
3a2cee341f Automatically restart wicd service rule 
@StevenBlack

Regards
 2016-02-15 19:21:23 +01:00
Steven Black
a4b254d1cf Issue #77: fix - replace tabs with space. 2016-02-15 12:36:38 -05:00
Steven Black
d69aa1add3 Issue #77: fix - Remove spurrious spaces. 2016-02-15 12:29:02 -05:00
Steven Black
9b28b369e8 Add a link to the latest raw version of the hosts file. 2016-02-15 10:43:29 -05:00
gigaslayer131890
df45003106 Modified the removeOldHostsFile() function to keep backups of the old hosts file. 2016-02-14 23:15:22 +00:00
Steven Black
8930205670 Suppress marginal messages in --auto mode. 2016-02-13 21:06:37 -05:00
Steven Black
32cde9692e Less verbose when echoing source updates. 2016-02-13 20:53:22 -05:00
Steven Black
1eb1509e41 Import argparse, and add an --auto flag to run the update and merge with no prompts. 2016-02-10 22:35:09 -05:00
Steven Black
ea43296b84 Remove initialized but thereafter unused variables. 2016-02-07 20:25:33 -05:00
Steven Black
d167327b2a Added local pylint directives. 2016-02-07 20:09:42 -05:00
Steven Black
5dfa76790d Going with pylint's recommendation for singleton comparisons. 2016-02-07 19:41:44 -05:00
Steven Black
ff682e2a0d Standardize on double quotes in print() calls everywhere. 2016-02-07 19:37:27 -05:00
Steven Black
b3e770c4b4 Split long code lines. 2016-02-07 19:34:27 -05:00
Steven Black
708d5a9dad Removed more superfluous parens. 2016-02-07 19:32:00 -05:00
Steven Black
eb00a6ca08 Removed unnecessary semicolons. 2016-02-07 19:28:31 -05:00
Steven Black
873cb4c0d4 Split long code line. 2016-02-07 19:25:32 -05:00
Steven Black
066ae7893c Removed superfluous parens. 2016-02-07 19:23:40 -05:00