J
|
b81988333c
|
Refactors and adds parametrs to argv to allow scrapping HTML/XML for both provinces or coords (polygon).
|
2019-09-18 22:09:00 +02:00 |
|
J
|
7438e0f5a7
|
Adds sleep as an argument
|
2019-09-18 19:14:03 +02:00 |
|
J
|
50d4ad6e93
|
Manually closes ElasticSearch socket. Fixes / Updates tests.
|
2019-09-18 19:11:04 +02:00 |
|
J
|
9f7d5fda51
|
Fixes XML scrapping for processing optional arguments. Removes bounding boxes to be eventually changed to polygons. Adds parameters to process by province.
|
2019-09-18 18:24:53 +02:00 |
|
J
|
0478146b27
|
Fixes parsing of bad-designed XML part of the code from Catastro (sometimes returns a list, sometimes only 1 entry but not within a list)
|
2019-09-18 01:37:04 +02:00 |
|
J
|
06bb139e63
|
Updates requirements.txt with xmltodict
|
2019-09-18 01:07:26 +02:00 |
|
J
|
b583814fb8
|
Adds XML Webservices Scrapping by all addresses in Spain, instead of by coordinates
|
2019-09-18 01:04:32 +02:00 |
|
J
|
8ffdf7faed
|
Adds smaller coordinates regions to avoid lots of sea
|
2019-09-17 13:48:50 +02:00 |
|
J
|
4071b66a65
|
Undoes the sleep(2) since Cadastro closes connection
|
2019-09-17 11:00:39 +02:00 |
|
J
|
daba3becaf
|
Changes sleep to 2sec
|
2019-09-17 10:51:05 +02:00 |
|
J
|
832f0e6239
|
Added option to execute the script with a specific json coordinate file.
|
2019-09-17 10:48:16 +02:00 |
|
J
|
89b3cb5994
|
CHhnges coordinates system, now uses Kibana Geo Point json format. Different regions provided to avoid a big suboptimized square with lots of sea points
|
2019-09-16 21:22:59 +02:00 |
|
J
|
c29de7faf2
|
Adds initialize_elasticsearch script to configurate the ES index
|
2019-09-16 17:45:24 +02:00 |
|
J
|
08b84b97a6
|
Refactors logger to log different levels in different places. Creates a main to execute from terminal.
|
2019-09-16 15:16:52 +02:00 |
|
J
|
7aa3e17912
|
Adds a JsonEncoder class to recursively manage serialization for ES. Changes scrapping function to return lists (because of multiparcelas).
|
2019-09-16 12:41:52 +02:00 |
|
J
|
103c87778b
|
Adds logger, utils, tests, new scrapping by time, fixes multiparcela errors.
|
2019-09-16 10:00:07 +02:00 |
|
J
|
b7a778d55c
|
Adds first prototype version with fully working tests.
|
2019-09-15 14:40:42 +02:00 |
|