Commit Graph

207 Commits

Author SHA1 Message Date
funilrys
f5c8ac58b2
Fix tests issue.
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347500695#L399
2018-02-28 23:23:30 +01:00
funilrys
079d5ddd7f
Fix tests issue
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347500695#L397

Please also note that I introduced that patch because
we do not directly use lxml but it is required by
BeautifulSup() to parse the HTML.
2018-02-28 23:22:32 +01:00
funilrys
1fea720034
Fix tests issue
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347500695#L398
2018-02-28 23:20:01 +01:00
funilrys
d3ef85df17
Review typo + fix test issue.
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347497718#L749
2018-02-28 23:15:01 +01:00
funilrys
079ad6b674
Fix test issue.
This patch fix https://travis-ci.org/funilrys/hosts/jobs/347497718#L748
2018-02-28 23:13:13 +01:00
funilrys
c82f691952
Review of get_file_by_url()
Please note that this patch also introduce
which is in charge of converting a domain in a line into
IDNA and/or UTF-8 format.

Also note the introduction of BeautifulSoup() which helps
us to decode data from the downloaded URL.

Fixes (issue(s)/protocol(s) I was able to reproduce):
 * https://github.com/StevenBlack/hosts/issues/514#issuecomment-368932152

Possible fix of (issue(s)/protocol(s) I wasn't able to reproduce):
 * https://github.com/StevenBlack/hosts/issues/514#issue-300048106
 * https://github.com/StevenBlack/hosts/issues/494#issue-296166492
 * https://github.com/StevenBlack/hosts/issues/420#issue-267453114
 * https://github.com/StevenBlack/hosts/issues/372#issue-246927047
 * https://github.com/StevenBlack/hosts/issues/382#issuecomment-322010562
2018-02-28 23:06:58 +01:00
Steven Black
007254e10f
Merge pull request #503 from ankitpati/minimise-hosts-file
Add Option to Minimise hosts File
2018-02-16 22:07:47 -05:00
Ankit Pati
01db326459
Add Option to Minimise hosts File
This option (--minimise, -m) differs from compress (--compress, -c) in
that it keeps each host on a different line, while still removing
unnecessary newlines and comments.

This is necessary because many implementations of URL blockers that rely
on hosts files do not conform to the standard which allows multiple
hosts on a single line.
2018-02-17 00:22:25 +05:30
Ankit Pati
db29acda65
FIX: Remove Hardcoded Offset in Compression
An offset of 7 was hardcoded in the function compress_file, presumably
to skip over the default target IP address of 0.0.0.0 in a hosts file.
However, this causes problems when the default is overridden using the
--ip or -i flag, causing visibly garbled output in the generated hosts
file.

Fix is to calculate the length of target IP at runtime.
2018-02-16 23:45:10 +05:30
FadeMind
5b5e854c7f
updateHostsFile.py: add ipv6 additionals
@StevenBlack.
DEFAULT hosts file in Linux

127.0.0.1       localhost
127.0.1.1       mydevicename
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
2018-02-09 19:18:54 +01:00
Stefano
d4dadf6ca3 Fixed the number of domains in each line.
Fixed the number of domains in each line and added the support to
inline comments (they will be ignored as the comment lines).
Code refactoring.
2018-01-02 22:35:27 +01:00
Stefano
510109d6d4 Updated the documentation with the new compress option.
Removed a redundant skipstatichosts option.
2018-01-02 10:51:26 +01:00
Stefano
ff9e05ffec Added an option to compress the created hosts file.
In particular, the compression option removes non-necessary lines (empty lines and comments) and puts multiple domains in each line.
This option should solve the issue #411 regarding the DNS client service of Windows.
2017-12-30 20:55:12 +01:00
Stefano
ce8d3abf7e Added an option to compress the created host file.
In particular, the compression option removes non-necessary lines (empty lines and comments) and puts multiple domains in each line.
This option should solve the issue #411 regarding the DNS client service of Windows.
2017-12-30 17:12:04 +01:00
Steven Black
bb5d174866 Fix linter complaint. 2017-12-18 23:57:02 -05:00
Steven Black
04a9a712b3 Issue #180: fix - support JSON array transformation for hosts sources. 2017-12-17 22:06:05 -05:00
gfyoung
24ab22e139
Refactor out source data updating 2017-08-20 11:52:28 -07:00
Steven Black
68dcf5c125 Remove references to .zip files in the generator programs. 2017-08-20 12:00:19 -04:00
gfyoung
f83a56d317 Refactor out global settings usage in update logic 2017-08-09 08:13:22 -07:00
gfyoung
942069db81 Refactor out global settings usage in exclusions 2017-08-07 08:43:47 -07:00
gfyoung
4b96f3f34a Refactor out global settings usage in user prompt 2017-07-13 08:20:03 -07:00
Steven Black
d945d7bcc0 Merge pull request #341 from indrajitr/hostmatch-fix
Honor whitelist exclusion only for domain or subdomain match
2017-07-09 13:13:57 -04:00
Dylan Armstrong
02a28cd05b Fix the number of unique entries. 2017-07-08 20:20:03 -05:00
gfyoung
415b89a022 Reduce dependency on global settings variable
Global variables make code less modular
and therefore more difficult to test.
2017-07-03 18:42:15 -07:00
gfyoung
dce24af399 Add unittests for updateHostsFile.py 2017-06-28 20:13:01 -07:00
Indrajit Raychaudhuri
fd046bc460 Honor whitelist exclusion only for domain or subdomain match
* Whitelisting `s.youtube.com` should not whitelist `ads.youtube.com`; but
  can whitelist `a.s.youtube.com`
* Whitelisting `twitter.com` should not whitelist `ttwitter.com` or
  `twitter.com.au`; but can whitelist `a.twitter.com`
2017-06-22 01:43:56 -05:00
Indrajit Raychaudhuri
7e1b06ecf9 Tighten regex rule to discard IP address as hostname 2017-06-20 02:06:31 -05:00
Steven Black
96ac61c119 Merge pull request #325 from gfyoung/terminal-color-support
Check if terminal supports color
2017-05-29 23:00:35 -04:00
gfyoung
2d6a8676a4 Check if terminal supports color
Closes gh-151.
2017-05-28 12:40:32 -04:00
gfyoung
3b581ce782 Refactor DNS cache flushing for Linux 2017-05-28 12:40:25 -04:00
gfyoung
51a0ec21bd Document all functions in Python scripts 2017-05-23 21:58:22 -04:00
Steven Black
526a457a38 Merge pull request #322 from gfyoung/kde-neon-dns-cache
Check /usr/lib and /lib for systemd
2017-05-23 10:03:22 -04:00
gfyoung
2aecbf9707 Concatenate str only in os.path.join
Closes gh-316.
2017-05-22 23:31:35 -04:00
gfyoung
8806131c74 Check /usr/lib and /lib for systemd
Closes gh-321.
2017-05-22 21:19:31 -04:00
gfyoung
8f3e71e327 Wrap os.path.join to handle LOCALE issues
Closes gh-81.
Closes gh-318.
2017-05-21 21:51:13 -04:00
Steven Black
d343798ed1 Ref: Issue #312 undo commit 2ff5dc2
cc @gfyoung

Commit 2ff5dc2 is generating errors on macOS.

See Issue #318.
2017-05-21 21:34:54 -04:00
gfyoung
c8e8ddee6c Wrap os.path.join to handle LOCALE issues
Closes gh-81.
2017-05-19 16:27:12 -04:00
Steven Black
f87290eff8 Merge pull request #310 from gfyoung/ubuntu-mint-dns-cache
Add DNS cache flushing calls for Linux
2017-05-19 07:30:54 -04:00
gfyoung
b3838ef2d2 Add DNS cache flushing calls for Linux
1) Add command for Linux Mint
2) Add additional command for NCSD cache

Closes gh-128.
2017-05-18 11:30:23 -04:00
gfyoung
8cb610a757 Don't crash if data source unreachable
Closes gh-26.
2017-05-18 00:17:11 -04:00
gfyoung
26cac85ca3 Patch all style errors in updateHostsFile.py 2017-05-15 15:00:16 -04:00
gfyoung
21dd1d9d56 Standardize underscore names in updateHostsFile.py 2017-05-15 13:32:39 -04:00
gfyoung
0015363385 Condense Python 2-3 compatibility code 2017-05-14 22:30:36 -04:00
gfyoung
190592577d Condense exclusion customization logic 2017-05-14 22:10:19 -04:00
gfyoung
ceb3765d24 Flush DNS cache on Windows 2017-05-14 22:07:43 -04:00
gfyoung
118a8e78a5 Remove unnecessary space between print and parenthesis 2017-05-14 14:54:24 -04:00
Steven Black
83af37862c Merge pull request #292 from gfyoung/remove-unused-imports
Remove unnecessary imports in updateHostsFile.py
2017-05-14 11:03:22 -04:00
gfyoung
5830bb8db0 Have query_yes_no return boolean
Bugs have been found where we assumed
that query_yes_no returned True/False.
Not only is it easier to check booleans
instead of strings, but it also looks
a lot cleaner in the code.
2017-05-14 00:15:58 -04:00
gfyoung
de2e45e8b2 Remove unnecessary imports in updateHostsFile.py 2017-05-13 23:43:00 -04:00
gfyoung
bdce1217bb Patch response checking in promptForFlushDnsCache
Previously, it was checking if query_yes_no returned
False, which does not happen, as the function only
returns "yes" or "no". As a result, the check was
always returning True, even when the user did not
want to flush the DNS cache.
2017-05-08 01:10:40 -04:00