mirror of
https://github.com/codeforspain/ds-organizacion-administrativa.git
synced 2024-06-06 22:15:20 +02:00
Add script municipios
This commit is contained in:
parent
2ed973be56
commit
33e5fc8d73
15
.gitignore
vendored
Normal file
15
.gitignore
vendored
Normal file
|
@ -0,0 +1,15 @@
|
|||
# User specific & automatically generated files #
|
||||
#################################################
|
||||
.idea/
|
||||
|
||||
# OS generated files #
|
||||
######################
|
||||
.DS_Store
|
||||
.DS_Store?
|
||||
._*
|
||||
.Spotlight-V100
|
||||
.Trashes
|
||||
Icon?
|
||||
ehthumbs.db
|
||||
Thumbs.db
|
||||
*~
|
84
README.md
84
README.md
|
@ -1,2 +1,84 @@
|
|||
# ds-organizacion-administrativa
|
||||
Listado de comunidades, provincias, municipios con su correspondiente código INE
|
||||
Listado de comunidades, provincias, municipios con sus correspondientes códigos INE
|
||||
|
||||
## Datos
|
||||
|
||||
|
||||
### Municipios
|
||||
|
||||
|
||||
URL: [Relación de municipios y códigos por provincias](http://www.ine.es/daco/daco42/codmun/codmun16/16codmunmapa.htm)
|
||||
Tipo: Excel (xlsx)
|
||||
Descripción: http://www.ine.es/daco/daco42/codmun/codmun00i.htm
|
||||
Script: [/scripts/municipios/](/scripts/municipios/)
|
||||
Datos procesaods: [/data/municipios.json](data/municipios.json) | [/data/municipios.csv](data/municipios.csv)
|
||||
|
||||
La lista de municipios sufre modificaciones todos los años para reflejar tanto los nuevos municipios (segregados), los que han desaparecido (incorporaciones o fusiones) y los cambios en su denominaciones oficiales.
|
||||
|
||||
|
||||
### Formato de los datos
|
||||
|
||||
|
||||
Incluye los campos
|
||||
|
||||
CPRO: Codigo INE de la provincia
|
||||
CMUN: Codigo INE del municipio en relacion a la provincia
|
||||
DC: Digito de control
|
||||
NOMBRE: Denominacion oficial
|
||||
|
||||
Ejemplo en CSV:
|
||||
|
||||
| CPRO | CMUN | DC | NOMBRE |
|
||||
|------|------|----|------------------|
|
||||
| 01 | 001 | 4 | Alegría-Dulantzi |
|
||||
| 01 | 002 | 9 | Amurrio |
|
||||
| 01 | 049 | 3 | Añana |
|
||||
| 01 | 003 | 5 | Aramaio |
|
||||
|
||||
|
||||
En JSON:
|
||||
|
||||
|
||||
[
|
||||
{
|
||||
"CPRO": "01", // CÓDIGO DE LA PROVINCIA
|
||||
"CMUN": "001", // CÓDIGO DEL MUNICIPIO
|
||||
"DC": "
|
||||
"NOMBRE": "Alegría-Dulantzi"" // DENOMINACIÓN DEL MUNICIPIO
|
||||
},
|
||||
{
|
||||
"CPRO": "01",
|
||||
"CMUN": "002",
|
||||
"NOMBRE": "Amurrio"
|
||||
},
|
||||
{
|
||||
"CPRO": "01",
|
||||
"CMUN": "049",
|
||||
"NOMBRE": "Añana"
|
||||
},
|
||||
{
|
||||
"CPRO": "01",
|
||||
"CMUN": "003",
|
||||
"NOMBRE": "Aramaio"
|
||||
},
|
||||
|
||||
|
||||
### Historico
|
||||
|
||||
En la carpeta [/data/historical](/data/historical) se almacena un histórico con los datos correspondientes a otros años.
|
||||
|
||||
### Comunidades Autonomas (PENDIENTE)
|
||||
|
||||
URL: http://www.ine.es/daco/daco42/codmun/cod_ccaa.htm
|
||||
Tipo: HTML
|
||||
|
||||
No ha sufrido modificaciones en los últimos años.
|
||||
|
||||
|
||||
|
||||
### Provincias (PENDIENTE)
|
||||
|
||||
URL: http://www.ine.es/daco/daco42/codmun/cod_provincia.htm
|
||||
Tipo: HTML
|
||||
|
||||
Las modificaciones que se producen son en su denominación oficial.
|
||||
|
|
BIN
archive/01codmun.xls
Normal file
BIN
archive/01codmun.xls
Normal file
Binary file not shown.
BIN
archive/02codmun.xls
Normal file
BIN
archive/02codmun.xls
Normal file
Binary file not shown.
BIN
archive/03codmun.xls
Normal file
BIN
archive/03codmun.xls
Normal file
Binary file not shown.
BIN
archive/04codmun.xls
Normal file
BIN
archive/04codmun.xls
Normal file
Binary file not shown.
BIN
archive/05codmun.xls
Normal file
BIN
archive/05codmun.xls
Normal file
Binary file not shown.
BIN
archive/06codmun.xls
Normal file
BIN
archive/06codmun.xls
Normal file
Binary file not shown.
BIN
archive/07codmun.xls
Normal file
BIN
archive/07codmun.xls
Normal file
Binary file not shown.
BIN
archive/08codmun.xls
Normal file
BIN
archive/08codmun.xls
Normal file
Binary file not shown.
BIN
archive/09codmun.xls
Normal file
BIN
archive/09codmun.xls
Normal file
Binary file not shown.
BIN
archive/10codmun.xls
Normal file
BIN
archive/10codmun.xls
Normal file
Binary file not shown.
BIN
archive/11codmun.xls
Normal file
BIN
archive/11codmun.xls
Normal file
Binary file not shown.
BIN
archive/12codmun.xls
Normal file
BIN
archive/12codmun.xls
Normal file
Binary file not shown.
BIN
archive/13codmun.xls
Normal file
BIN
archive/13codmun.xls
Normal file
Binary file not shown.
BIN
archive/14codmun.xls
Normal file
BIN
archive/14codmun.xls
Normal file
Binary file not shown.
BIN
archive/15codmun.xls
Normal file
BIN
archive/15codmun.xls
Normal file
Binary file not shown.
BIN
archive/16codmun.xlsx
Normal file
BIN
archive/16codmun.xlsx
Normal file
Binary file not shown.
1
archive/municipios-2016.csv
Symbolic link
1
archive/municipios-2016.csv
Symbolic link
|
@ -0,0 +1 @@
|
|||
/home/inigo/code/ds-organizacion-administrativa/data/municipios-2016.csv
|
|
1
archive/municipios-2016.json
Symbolic link
1
archive/municipios-2016.json
Symbolic link
|
@ -0,0 +1 @@
|
|||
/home/inigo/code/ds-organizacion-administrativa/data/municipios-2016.json
|
8110
data/historical/municipios-2004.csv
Normal file
8110
data/historical/municipios-2004.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2004.json
Normal file
1
data/historical/municipios-2004.json
Normal file
File diff suppressed because one or more lines are too long
8110
data/historical/municipios-2005.csv
Normal file
8110
data/historical/municipios-2005.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2005.json
Normal file
1
data/historical/municipios-2005.json
Normal file
File diff suppressed because one or more lines are too long
8111
data/historical/municipios-2006.csv
Normal file
8111
data/historical/municipios-2006.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2006.json
Normal file
1
data/historical/municipios-2006.json
Normal file
File diff suppressed because one or more lines are too long
8112
data/historical/municipios-2007.csv
Normal file
8112
data/historical/municipios-2007.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2007.json
Normal file
1
data/historical/municipios-2007.json
Normal file
File diff suppressed because one or more lines are too long
8113
data/historical/municipios-2008.csv
Normal file
8113
data/historical/municipios-2008.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2008.json
Normal file
1
data/historical/municipios-2008.json
Normal file
File diff suppressed because one or more lines are too long
8113
data/historical/municipios-2009.csv
Normal file
8113
data/historical/municipios-2009.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2009.json
Normal file
1
data/historical/municipios-2009.json
Normal file
File diff suppressed because one or more lines are too long
8115
data/historical/municipios-2010.csv
Normal file
8115
data/historical/municipios-2010.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2010.json
Normal file
1
data/historical/municipios-2010.json
Normal file
File diff suppressed because one or more lines are too long
8117
data/historical/municipios-2011.csv
Normal file
8117
data/historical/municipios-2011.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2011.json
Normal file
1
data/historical/municipios-2011.json
Normal file
File diff suppressed because one or more lines are too long
8117
data/historical/municipios-2012.csv
Normal file
8117
data/historical/municipios-2012.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2012.json
Normal file
1
data/historical/municipios-2012.json
Normal file
File diff suppressed because one or more lines are too long
8118
data/historical/municipios-2013.csv
Normal file
8118
data/historical/municipios-2013.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2013.json
Normal file
1
data/historical/municipios-2013.json
Normal file
File diff suppressed because one or more lines are too long
8118
data/historical/municipios-2014.csv
Normal file
8118
data/historical/municipios-2014.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2014.json
Normal file
1
data/historical/municipios-2014.json
Normal file
File diff suppressed because one or more lines are too long
8120
data/historical/municipios-2015.csv
Normal file
8120
data/historical/municipios-2015.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2015.json
Normal file
1
data/historical/municipios-2015.json
Normal file
File diff suppressed because one or more lines are too long
8126
data/historical/municipios-2016.csv
Normal file
8126
data/historical/municipios-2016.csv
Normal file
File diff suppressed because it is too large
Load Diff
1
data/historical/municipios-2016.json
Normal file
1
data/historical/municipios-2016.json
Normal file
File diff suppressed because one or more lines are too long
1
data/municipios.csv
Symbolic link
1
data/municipios.csv
Symbolic link
|
@ -0,0 +1 @@
|
|||
historical/municipios-2016.csv
|
|
1
data/municipios.json
Symbolic link
1
data/municipios.json
Symbolic link
|
@ -0,0 +1 @@
|
|||
historical/municipios-2016.json
|
57
datapackage.json
Normal file
57
datapackage.json
Normal file
|
@ -0,0 +1,57 @@
|
|||
{
|
||||
"name": "ds-organizacion-administrativa",
|
||||
"title": "Organiazación Administrativa de España",
|
||||
"descriptions": "Listado de comunidades, provincias, municipios con su correspondiente código INE",
|
||||
"licenses" : [
|
||||
{
|
||||
"type": "odc-pddl",
|
||||
"url": "http://opendatacommons.org/licenses/pddl/"
|
||||
}
|
||||
],
|
||||
"author" : {
|
||||
"name": "Code for Spain",
|
||||
"web": "http://www.codeforspain.org"
|
||||
},
|
||||
"keywords": [ "Provincias", "Municipios", "Comunidades Autonomas"],
|
||||
"version": "0.0.1",
|
||||
"last_updated": "2016-04-20",
|
||||
"sources": [
|
||||
{
|
||||
"name": "Instituto Nacional de Estadistica",
|
||||
"web": "http://www.ine.es/jaxi/menu.do?type=pcaxis&path=/t20/e245/codmun&file=inebase"
|
||||
}
|
||||
],
|
||||
"resources": [
|
||||
{
|
||||
"path": "data/municipios.csv",
|
||||
"schema": {
|
||||
"fields": [
|
||||
{
|
||||
"name": "CPRO",
|
||||
"type": "number",
|
||||
"description": "Código INE de la provincia"
|
||||
"pattern": "[0-9]{2}"
|
||||
|
||||
},
|
||||
{
|
||||
"name": "CMUN",
|
||||
"type": "number",
|
||||
"description": "Código INE del municipio en relación a la provincia. CPRO contacatenado con CMUN resulta en el codigo INE del municipio",
|
||||
"pattern": "[0-9]{3}"
|
||||
},
|
||||
{
|
||||
"name": "DC",
|
||||
"type": "number",
|
||||
"description": "Dígito de control que, asignado mediante una regla de cálculo, permite la detección de errores de grabación y codificación",
|
||||
"pattern": "[0-9]{1}"
|
||||
},
|
||||
{
|
||||
"name": "NOMBRE",
|
||||
"type": "number",
|
||||
"description": "Denominación Oficial del Municipios"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
34
scripts/municipios/Readme.md
Normal file
34
scripts/municipios/Readme.md
Normal file
|
@ -0,0 +1,34 @@
|
|||
# Municipios
|
||||
|
||||
Procesa los archivos fuente de municipios que publica el INE todos los años, y que están alojados en ../archive.
|
||||
|
||||
Si no los encuentra, los descarga.
|
||||
|
||||
|
||||
## Modo de Uso
|
||||
|
||||
$ ./process [-hvd]
|
||||
|
||||
|
||||
### opciones
|
||||
|
||||
-h muestra esta ayuda y termina
|
||||
-d fuerza la descarga de los ficheros fuente aunque ya existan
|
||||
-v modo verbose
|
||||
|
||||
|
||||
## Requisitos
|
||||
|
||||
Hay que tener instalado [csvkit](https://csvkit.readthedocs.org/en/540/index.html). Para este script se ha usado la version 1.0.0.
|
||||
|
||||
Se instala mediante:
|
||||
|
||||
$ sudo pip install csvkit
|
||||
|
||||
|
||||
|
||||
Así mismo, `pip` tiene que estar instalado. En Ubuntu esto se hace mediante:
|
||||
|
||||
$ sudo apt-get install python-pip python-dev build-essential
|
||||
$ sudo pip install --upgrade pip
|
||||
$ sudo pip install --upgrade virtualenv
|
125
scripts/municipios/process.sh
Executable file
125
scripts/municipios/process.sh
Executable file
|
@ -0,0 +1,125 @@
|
|||
#!/bin/bash
|
||||
|
||||
BASE_PATH="`dirname \"$0\"`" # relative
|
||||
BASE_PATH="`( cd \"$BASE_PATH\"/../.. && pwd )`" # absolute
|
||||
|
||||
# Ruta de los archivos fuente
|
||||
ARCHIVE=$BASE_PATH/archive
|
||||
|
||||
# Ruta donde se almacenan los datos procesados
|
||||
DATA=$BASE_PATH/data
|
||||
|
||||
# Ruta donde se almacenan los datos procesados historicos
|
||||
HISTORICAL_DATA=$BASE_PATH/data/historical
|
||||
|
||||
export PYTHONWARNINGS="ignore"
|
||||
_VERBOSE=0
|
||||
FORCE_DOWNLOAD=0
|
||||
|
||||
# verbose mode
|
||||
function log () {
|
||||
if [[ $_VERBOSE -eq 1 ]]; then
|
||||
echo -e "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
# Usage
|
||||
function show_help() {
|
||||
cat << EOF
|
||||
Uso: ${0##*/} [-hvd]
|
||||
Procesa los archivos fuente de municipios que publica el INE todos los años, y que están alojados en ../archive.
|
||||
Si no los encuentra, los descarga.
|
||||
|
||||
-h muestra esta ayuda y termina
|
||||
-d fuerza la descarga de los ficheros fuente aunque ya existan
|
||||
-v modo verbose
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# A POSIX variable
|
||||
OPTIND=1 # Reset
|
||||
|
||||
# Initialize our own variables:
|
||||
|
||||
while getopts "dh?vf:" opt; do
|
||||
case "$opt" in
|
||||
h|\?)
|
||||
show_help
|
||||
exit 0
|
||||
;;
|
||||
v)
|
||||
_VERBOSE=1
|
||||
export PYTHONWARNINGS=
|
||||
;;
|
||||
d)
|
||||
FORCE_DOWNLOAD=1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Creamos los directorios si no existen
|
||||
mkdir -p $ARCHIVE
|
||||
mkdir -p $DATA
|
||||
mkdir -p $HISTORICAL_DATA
|
||||
|
||||
|
||||
CURRYEAR=`date +%y`
|
||||
log Año actual: $CURRYEAR
|
||||
log "----------------------------".
|
||||
|
||||
EXT=xls
|
||||
|
||||
for i in $( seq 4 $CURRYEAR); do
|
||||
|
||||
if [ "$i" -gt "15" ]; then
|
||||
EXT=xlsx
|
||||
fi
|
||||
|
||||
if [ "$i" -eq "4" ]; then
|
||||
LINES_TO_STRIP=1
|
||||
else
|
||||
LINES_TO_STRIP=2
|
||||
fi
|
||||
|
||||
YEAR=`printf %02d $i`
|
||||
SOURCEFILE="$YEAR"codmun.$EXT
|
||||
CSVFILE=municipios-20$YEAR.csv
|
||||
JSONFILE=municipios-20$YEAR.json
|
||||
|
||||
log Procesando año 20$YEAR
|
||||
|
||||
cd $ARCHIVE
|
||||
if [ ! -f $SOURCEFILE ] || [ $FORCE_DOWNLOAD -eq 1 ]; then
|
||||
log No existe copia local del archivo $SOURCEFILE. DESCARGAMOS \\n
|
||||
log
|
||||
curl -O http://www.ine.es/daco/daco42/codmun/codmun"$YEAR"/$SOURCEFILE
|
||||
log \\n
|
||||
fi
|
||||
|
||||
log Generamos $CSVFILE
|
||||
|
||||
in2csv $SOURCEFILE |tail -n +$LINES_TO_STRIP >$HISTORICAL_DATA/$CSVFILE
|
||||
|
||||
log Generamos $JSONFILE
|
||||
csvjson $HISTORICAL_DATA/$CSVFILE > $HISTORICAL_DATA/$JSONFILE
|
||||
log "\\n----------------------------"\\n
|
||||
|
||||
|
||||
done
|
||||
|
||||
log Creamos symlinks \(último procesado\)\\n
|
||||
|
||||
echo ln -rfs $HISTORICAL_DATA/$CSVFILE $DATA/municipios.csv
|
||||
echo ln -rfs $HISTORICAL_DATA/$JSONFILE $DATA/municipios.json
|
||||
|
||||
ln -rfs $HISTORICAL_DATA/$CSVFILE $DATA/municipios.csv
|
||||
ln -rfs $HISTORICAL_DATA/$JSONFILE $DATA/municipios.json
|
||||
|
||||
log Nota: Se puede ignorar el warning \"Discarded range with reserved name\" en caso de que se produzca
|
||||
|
Loading…
Reference in New Issue
Block a user