mirror of
https://github.com/josejuanmartinez/libreCatastro.git
synced 2024-06-02 23:05:35 +02:00
1.2 KiB
1.2 KiB
#libreCATASTRO An opensource, MIT-licensed application that scraps the official Spanish Cadaster registry and stores information in Elastic Searcher.
Features
Scrapping
- From XML webservices. Check http://www.catastro.meh.es/ws/Webservices_Libres.pdf
- From HTML
- Scraps by zone/site and by property in them
- Scraps rural and urban properties
- Retrieves a picture of every property
Storing
- Stores in ElasticSearch
- Allows visualization in Kibana
Visualization
Includes a configured Kibana that shows.
- A heatmap in the map of Spain (World) where the properties are
- All data in tables
- The picture of the property
DoS Warning
Spanish Cadaster has set restrictions, banning temporarily IPs that more than 10 queries in 5 seconds. A sleep command has been set to 5sec where needed, and can be configured at your own risk.
Installation
Having Docker and Docker-compose installed, run first:
docker-compose up -d
Then configure ElasticSearch index:
python3 initialize_elasticsearch.py
That simple!
Execution
python main.py[--coords] [--pictures] [--filenames filename1 filename2 ...] [--provinces province1 province2 ...] [--sleep sleep_time] [--html]