mirror of
https://github.com/codeforspain/ds-organizacion-administrativa.git
synced 2024-06-20 20:55:39 +02:00
Add script municipios
This commit is contained in:
parent
2ed973be56
commit
33e5fc8d73
15
.gitignore
vendored
Normal file
15
.gitignore
vendored
Normal file
|
@ -0,0 +1,15 @@
|
||||||
|
# User specific & automatically generated files #
|
||||||
|
#################################################
|
||||||
|
.idea/
|
||||||
|
|
||||||
|
# OS generated files #
|
||||||
|
######################
|
||||||
|
.DS_Store
|
||||||
|
.DS_Store?
|
||||||
|
._*
|
||||||
|
.Spotlight-V100
|
||||||
|
.Trashes
|
||||||
|
Icon?
|
||||||
|
ehthumbs.db
|
||||||
|
Thumbs.db
|
||||||
|
*~
|
84
README.md
84
README.md
|
@ -1,2 +1,84 @@
|
||||||
# ds-organizacion-administrativa
|
# ds-organizacion-administrativa
|
||||||
Listado de comunidades, provincias, municipios con su correspondiente código INE
|
Listado de comunidades, provincias, municipios con sus correspondientes códigos INE
|
||||||
|
|
||||||
|
## Datos
|
||||||
|
|
||||||
|
|
||||||
|
### Municipios
|
||||||
|
|
||||||
|
|
||||||
|
URL: [Relación de municipios y códigos por provincias](http://www.ine.es/daco/daco42/codmun/codmun16/16codmunmapa.htm)
|
||||||
|
Tipo: Excel (xlsx)
|
||||||
|
Descripción: http://www.ine.es/daco/daco42/codmun/codmun00i.htm
|
||||||
|
Script: [/scripts/municipios/](/scripts/municipios/)
|
||||||
|
Datos procesaods: [/data/municipios.json](data/municipios.json) | [/data/municipios.csv](data/municipios.csv)
|
||||||
|
|
||||||
|
La lista de municipios sufre modificaciones todos los años para reflejar tanto los nuevos municipios (segregados), los que han desaparecido (incorporaciones o fusiones) y los cambios en su denominaciones oficiales.
|
||||||
|
|
||||||
|
|
||||||
|
### Formato de los datos
|
||||||
|
|
||||||
|
|
||||||
|
Incluye los campos
|
||||||
|
|
||||||
|
CPRO: Codigo INE de la provincia
|
||||||
|
CMUN: Codigo INE del municipio en relacion a la provincia
|
||||||
|
DC: Digito de control
|
||||||
|
NOMBRE: Denominacion oficial
|
||||||
|
|
||||||
|
Ejemplo en CSV:
|
||||||
|
|
||||||
|
| CPRO | CMUN | DC | NOMBRE |
|
||||||
|
|------|------|----|------------------|
|
||||||
|
| 01 | 001 | 4 | Alegría-Dulantzi |
|
||||||
|
| 01 | 002 | 9 | Amurrio |
|
||||||
|
| 01 | 049 | 3 | Añana |
|
||||||
|
| 01 | 003 | 5 | Aramaio |
|
||||||
|
|
||||||
|
|
||||||
|
En JSON:
|
||||||
|
|
||||||
|
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"CPRO": "01", // CÓDIGO DE LA PROVINCIA
|
||||||
|
"CMUN": "001", // CÓDIGO DEL MUNICIPIO
|
||||||
|
"DC": "
|
||||||
|
"NOMBRE": "Alegría-Dulantzi"" // DENOMINACIÓN DEL MUNICIPIO
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"CPRO": "01",
|
||||||
|
"CMUN": "002",
|
||||||
|
"NOMBRE": "Amurrio"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"CPRO": "01",
|
||||||
|
"CMUN": "049",
|
||||||
|
"NOMBRE": "Añana"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"CPRO": "01",
|
||||||
|
"CMUN": "003",
|
||||||
|
"NOMBRE": "Aramaio"
|
||||||
|
},
|
||||||
|
|
||||||
|
|
||||||
|
### Historico
|
||||||
|
|
||||||
|
En la carpeta [/data/historical](/data/historical) se almacena un histórico con los datos correspondientes a otros años.
|
||||||
|
|
||||||
|
### Comunidades Autonomas (PENDIENTE)
|
||||||
|
|
||||||
|
URL: http://www.ine.es/daco/daco42/codmun/cod_ccaa.htm
|
||||||
|
Tipo: HTML
|
||||||
|
|
||||||
|
No ha sufrido modificaciones en los últimos años.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Provincias (PENDIENTE)
|
||||||
|
|
||||||
|
URL: http://www.ine.es/daco/daco42/codmun/cod_provincia.htm
|
||||||
|
Tipo: HTML
|
||||||
|
|
||||||
|
Las modificaciones que se producen son en su denominación oficial.
|
||||||
|
|
BIN
archive/01codmun.xls
Normal file
BIN
archive/01codmun.xls
Normal file
Binary file not shown.
BIN
archive/02codmun.xls
Normal file
BIN
archive/02codmun.xls
Normal file
Binary file not shown.
BIN
archive/03codmun.xls
Normal file
BIN
archive/03codmun.xls
Normal file
Binary file not shown.
BIN
archive/04codmun.xls
Normal file
BIN
archive/04codmun.xls
Normal file
Binary file not shown.
BIN
archive/05codmun.xls
Normal file
BIN
archive/05codmun.xls
Normal file
Binary file not shown.
BIN
archive/06codmun.xls
Normal file
BIN
archive/06codmun.xls
Normal file
Binary file not shown.
BIN
archive/07codmun.xls
Normal file
BIN
archive/07codmun.xls
Normal file
Binary file not shown.
BIN
archive/08codmun.xls
Normal file
BIN
archive/08codmun.xls
Normal file
Binary file not shown.
BIN
archive/09codmun.xls
Normal file
BIN
archive/09codmun.xls
Normal file
Binary file not shown.
BIN
archive/10codmun.xls
Normal file
BIN
archive/10codmun.xls
Normal file
Binary file not shown.
BIN
archive/11codmun.xls
Normal file
BIN
archive/11codmun.xls
Normal file
Binary file not shown.
BIN
archive/12codmun.xls
Normal file
BIN
archive/12codmun.xls
Normal file
Binary file not shown.
BIN
archive/13codmun.xls
Normal file
BIN
archive/13codmun.xls
Normal file
Binary file not shown.
BIN
archive/14codmun.xls
Normal file
BIN
archive/14codmun.xls
Normal file
Binary file not shown.
BIN
archive/15codmun.xls
Normal file
BIN
archive/15codmun.xls
Normal file
Binary file not shown.
BIN
archive/16codmun.xlsx
Normal file
BIN
archive/16codmun.xlsx
Normal file
Binary file not shown.
1
archive/municipios-2016.csv
Symbolic link
1
archive/municipios-2016.csv
Symbolic link
|
@ -0,0 +1 @@
|
||||||
|
/home/inigo/code/ds-organizacion-administrativa/data/municipios-2016.csv
|
|
1
archive/municipios-2016.json
Symbolic link
1
archive/municipios-2016.json
Symbolic link
|
@ -0,0 +1 @@
|
||||||
|
/home/inigo/code/ds-organizacion-administrativa/data/municipios-2016.json
|
8110
data/historical/municipios-2004.csv
Normal file
8110
data/historical/municipios-2004.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2004.json
Normal file
1
data/historical/municipios-2004.json
Normal file
File diff suppressed because one or more lines are too long
8110
data/historical/municipios-2005.csv
Normal file
8110
data/historical/municipios-2005.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2005.json
Normal file
1
data/historical/municipios-2005.json
Normal file
File diff suppressed because one or more lines are too long
8111
data/historical/municipios-2006.csv
Normal file
8111
data/historical/municipios-2006.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2006.json
Normal file
1
data/historical/municipios-2006.json
Normal file
File diff suppressed because one or more lines are too long
8112
data/historical/municipios-2007.csv
Normal file
8112
data/historical/municipios-2007.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2007.json
Normal file
1
data/historical/municipios-2007.json
Normal file
File diff suppressed because one or more lines are too long
8113
data/historical/municipios-2008.csv
Normal file
8113
data/historical/municipios-2008.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2008.json
Normal file
1
data/historical/municipios-2008.json
Normal file
File diff suppressed because one or more lines are too long
8113
data/historical/municipios-2009.csv
Normal file
8113
data/historical/municipios-2009.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2009.json
Normal file
1
data/historical/municipios-2009.json
Normal file
File diff suppressed because one or more lines are too long
8115
data/historical/municipios-2010.csv
Normal file
8115
data/historical/municipios-2010.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2010.json
Normal file
1
data/historical/municipios-2010.json
Normal file
File diff suppressed because one or more lines are too long
8117
data/historical/municipios-2011.csv
Normal file
8117
data/historical/municipios-2011.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2011.json
Normal file
1
data/historical/municipios-2011.json
Normal file
File diff suppressed because one or more lines are too long
8117
data/historical/municipios-2012.csv
Normal file
8117
data/historical/municipios-2012.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2012.json
Normal file
1
data/historical/municipios-2012.json
Normal file
File diff suppressed because one or more lines are too long
8118
data/historical/municipios-2013.csv
Normal file
8118
data/historical/municipios-2013.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2013.json
Normal file
1
data/historical/municipios-2013.json
Normal file
File diff suppressed because one or more lines are too long
8118
data/historical/municipios-2014.csv
Normal file
8118
data/historical/municipios-2014.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2014.json
Normal file
1
data/historical/municipios-2014.json
Normal file
File diff suppressed because one or more lines are too long
8120
data/historical/municipios-2015.csv
Normal file
8120
data/historical/municipios-2015.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2015.json
Normal file
1
data/historical/municipios-2015.json
Normal file
File diff suppressed because one or more lines are too long
8126
data/historical/municipios-2016.csv
Normal file
8126
data/historical/municipios-2016.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2016.json
Normal file
1
data/historical/municipios-2016.json
Normal file
File diff suppressed because one or more lines are too long
1
data/municipios.csv
Symbolic link
1
data/municipios.csv
Symbolic link
|
@ -0,0 +1 @@
|
||||||
|
historical/municipios-2016.csv
|
|
1
data/municipios.json
Symbolic link
1
data/municipios.json
Symbolic link
|
@ -0,0 +1 @@
|
||||||
|
historical/municipios-2016.json
|
57
datapackage.json
Normal file
57
datapackage.json
Normal file
|
@ -0,0 +1,57 @@
|
||||||
|
{
|
||||||
|
"name": "ds-organizacion-administrativa",
|
||||||
|
"title": "Organiazación Administrativa de España",
|
||||||
|
"descriptions": "Listado de comunidades, provincias, municipios con su correspondiente código INE",
|
||||||
|
"licenses" : [
|
||||||
|
{
|
||||||
|
"type": "odc-pddl",
|
||||||
|
"url": "http://opendatacommons.org/licenses/pddl/"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"author" : {
|
||||||
|
"name": "Code for Spain",
|
||||||
|
"web": "http://www.codeforspain.org"
|
||||||
|
},
|
||||||
|
"keywords": [ "Provincias", "Municipios", "Comunidades Autonomas"],
|
||||||
|
"version": "0.0.1",
|
||||||
|
"last_updated": "2016-04-20",
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"name": "Instituto Nacional de Estadistica",
|
||||||
|
"web": "http://www.ine.es/jaxi/menu.do?type=pcaxis&path=/t20/e245/codmun&file=inebase"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"resources": [
|
||||||
|
{
|
||||||
|
"path": "data/municipios.csv",
|
||||||
|
"schema": {
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"name": "CPRO",
|
||||||
|
"type": "number",
|
||||||
|
"description": "Código INE de la provincia"
|
||||||
|
"pattern": "[0-9]{2}"
|
||||||
|
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "CMUN",
|
||||||
|
"type": "number",
|
||||||
|
"description": "Código INE del municipio en relación a la provincia. CPRO contacatenado con CMUN resulta en el codigo INE del municipio",
|
||||||
|
"pattern": "[0-9]{3}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "DC",
|
||||||
|
"type": "number",
|
||||||
|
"description": "Dígito de control que, asignado mediante una regla de cálculo, permite la detección de errores de grabación y codificación",
|
||||||
|
"pattern": "[0-9]{1}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "NOMBRE",
|
||||||
|
"type": "number",
|
||||||
|
"description": "Denominación Oficial del Municipios"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
34
scripts/municipios/Readme.md
Normal file
34
scripts/municipios/Readme.md
Normal file
|
@ -0,0 +1,34 @@
|
||||||
|
# Municipios
|
||||||
|
|
||||||
|
Procesa los archivos fuente de municipios que publica el INE todos los años, y que están alojados en ../archive.
|
||||||
|
|
||||||
|
Si no los encuentra, los descarga.
|
||||||
|
|
||||||
|
|
||||||
|
## Modo de Uso
|
||||||
|
|
||||||
|
$ ./process [-hvd]
|
||||||
|
|
||||||
|
|
||||||
|
### opciones
|
||||||
|
|
||||||
|
-h muestra esta ayuda y termina
|
||||||
|
-d fuerza la descarga de los ficheros fuente aunque ya existan
|
||||||
|
-v modo verbose
|
||||||
|
|
||||||
|
|
||||||
|
## Requisitos
|
||||||
|
|
||||||
|
Hay que tener instalado [csvkit](https://csvkit.readthedocs.org/en/540/index.html). Para este script se ha usado la version 1.0.0.
|
||||||
|
|
||||||
|
Se instala mediante:
|
||||||
|
|
||||||
|
$ sudo pip install csvkit
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Así mismo, `pip` tiene que estar instalado. En Ubuntu esto se hace mediante:
|
||||||
|
|
||||||
|
$ sudo apt-get install python-pip python-dev build-essential
|
||||||
|
$ sudo pip install --upgrade pip
|
||||||
|
$ sudo pip install --upgrade virtualenv
|
125
scripts/municipios/process.sh
Executable file
125
scripts/municipios/process.sh
Executable file
|
@ -0,0 +1,125 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
BASE_PATH="`dirname \"$0\"`" # relative
|
||||||
|
BASE_PATH="`( cd \"$BASE_PATH\"/../.. && pwd )`" # absolute
|
||||||
|
|
||||||
|
# Ruta de los archivos fuente
|
||||||
|
ARCHIVE=$BASE_PATH/archive
|
||||||
|
|
||||||
|
# Ruta donde se almacenan los datos procesados
|
||||||
|
DATA=$BASE_PATH/data
|
||||||
|
|
||||||
|
# Ruta donde se almacenan los datos procesados historicos
|
||||||
|
HISTORICAL_DATA=$BASE_PATH/data/historical
|
||||||
|
|
||||||
|
export PYTHONWARNINGS="ignore"
|
||||||
|
_VERBOSE=0
|
||||||
|
FORCE_DOWNLOAD=0
|
||||||
|
|
||||||
|
# verbose mode
|
||||||
|
function log () {
|
||||||
|
if [[ $_VERBOSE -eq 1 ]]; then
|
||||||
|
echo -e "$@"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# Usage
|
||||||
|
function show_help() {
|
||||||
|
cat << EOF
|
||||||
|
Uso: ${0##*/} [-hvd]
|
||||||
|
Procesa los archivos fuente de municipios que publica el INE todos los años, y que están alojados en ../archive.
|
||||||
|
Si no los encuentra, los descarga.
|
||||||
|
|
||||||
|
-h muestra esta ayuda y termina
|
||||||
|
-d fuerza la descarga de los ficheros fuente aunque ya existan
|
||||||
|
-v modo verbose
|
||||||
|
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# A POSIX variable
|
||||||
|
OPTIND=1 # Reset
|
||||||
|
|
||||||
|
# Initialize our own variables:
|
||||||
|
|
||||||
|
while getopts "dh?vf:" opt; do
|
||||||
|
case "$opt" in
|
||||||
|
h|\?)
|
||||||
|
show_help
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
v)
|
||||||
|
_VERBOSE=1
|
||||||
|
export PYTHONWARNINGS=
|
||||||
|
;;
|
||||||
|
d)
|
||||||
|
FORCE_DOWNLOAD=1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Creamos los directorios si no existen
|
||||||
|
mkdir -p $ARCHIVE
|
||||||
|
mkdir -p $DATA
|
||||||
|
mkdir -p $HISTORICAL_DATA
|
||||||
|
|
||||||
|
|
||||||
|
CURRYEAR=`date +%y`
|
||||||
|
log Año actual: $CURRYEAR
|
||||||
|
log "----------------------------".
|
||||||
|
|
||||||
|
EXT=xls
|
||||||
|
|
||||||
|
for i in $( seq 4 $CURRYEAR); do
|
||||||
|
|
||||||
|
if [ "$i" -gt "15" ]; then
|
||||||
|
EXT=xlsx
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$i" -eq "4" ]; then
|
||||||
|
LINES_TO_STRIP=1
|
||||||
|
else
|
||||||
|
LINES_TO_STRIP=2
|
||||||
|
fi
|
||||||
|
|
||||||
|
YEAR=`printf %02d $i`
|
||||||
|
SOURCEFILE="$YEAR"codmun.$EXT
|
||||||
|
CSVFILE=municipios-20$YEAR.csv
|
||||||
|
JSONFILE=municipios-20$YEAR.json
|
||||||
|
|
||||||
|
log Procesando año 20$YEAR
|
||||||
|
|
||||||
|
cd $ARCHIVE
|
||||||
|
if [ ! -f $SOURCEFILE ] || [ $FORCE_DOWNLOAD -eq 1 ]; then
|
||||||
|
log No existe copia local del archivo $SOURCEFILE. DESCARGAMOS \\n
|
||||||
|
log
|
||||||
|
curl -O http://www.ine.es/daco/daco42/codmun/codmun"$YEAR"/$SOURCEFILE
|
||||||
|
log \\n
|
||||||
|
fi
|
||||||
|
|
||||||
|
log Generamos $CSVFILE
|
||||||
|
|
||||||
|
in2csv $SOURCEFILE |tail -n +$LINES_TO_STRIP >$HISTORICAL_DATA/$CSVFILE
|
||||||
|
|
||||||
|
log Generamos $JSONFILE
|
||||||
|
csvjson $HISTORICAL_DATA/$CSVFILE > $HISTORICAL_DATA/$JSONFILE
|
||||||
|
log "\\n----------------------------"\\n
|
||||||
|
|
||||||
|
|
||||||
|
done
|
||||||
|
|
||||||
|
log Creamos symlinks \(último procesado\)\\n
|
||||||
|
|
||||||
|
echo ln -rfs $HISTORICAL_DATA/$CSVFILE $DATA/municipios.csv
|
||||||
|
echo ln -rfs $HISTORICAL_DATA/$JSONFILE $DATA/municipios.json
|
||||||
|
|
||||||
|
ln -rfs $HISTORICAL_DATA/$CSVFILE $DATA/municipios.csv
|
||||||
|
ln -rfs $HISTORICAL_DATA/$JSONFILE $DATA/municipios.json
|
||||||
|
|
||||||
|
log Nota: Se puede ignorar el warning \"Discarded range with reserved name\" en caso de que se produzca
|
||||||
|
|
Loading…
Reference in New Issue
Block a user