Mario Zechner
c3abf97363
More import fixes.
2023-06-21 18:28:35 +02:00
Mario Zechner
d16c7005d6
More import fixes.
2023-06-21 18:27:32 +02:00
Mario Zechner
807c6e90e2
Fix import.
2023-06-21 18:25:21 +02:00
Mario Zechner
bb2511eefc
Use local category file instead of querying site.
2023-06-21 18:20:36 +02:00
Mario Zechner
b05702aff5
Hofer and MPREIS categories.
2023-06-21 15:20:28 +02:00
Mario Zechner
303d25ccb5
Categories for Billa & Spar, infra to add catgories for other stores.
...
Billa maps directly to the canonical categories. Spar uses a mapping file stores/spar-categories.json.
Each store has a generateCategoryMapping() function which is called once in analysis.js:updateData() and analysis.js:replay(). The function is responsible for
* Fetching the latest categories
* Merging them with already mapped categories
* Report new categories that haven't been mapped yet
* Report categories that have been mapped but are no longer part of the latest set of categories
* Save the merged mappings to disk
This schema might not work for all stores, in which case updateData() and replay() will use a knn approach to figure out the category for an item. See #81
2023-06-21 01:29:00 +02:00
Mario Zechner
6569b17da2
Remove binary encoding, web worker, clean-up.
2023-06-18 23:23:02 +02:00
Mario Zechner
898fa44ff1
Add description to items where available, use it in search index.
2023-06-17 20:54:45 +02:00
Mario Zechner
c7537c341e
Binary format optimization 4.4mb -> 3.9mb, don't store urls were not needed, use product-id instead of code-internal for spar items,
2023-06-17 01:11:21 +02:00
Matthias Hochsteger
e66121e2f2
Use price-per-unit field for weighted products in spar (seems most robust)
2023-06-06 20:38:48 +02:00
Mario Zechner
f60436b59b
More Spar unit fixing.
2023-06-05 22:06:38 +02:00
Mario Zechner
91f30e553f
Don't lowercase Spar item names.
2023-06-05 21:59:35 +02:00
Mario Zechner
d30ef57d9b
Fix a bunch of SPAR unit/quantity issues. See #71 .
2023-06-05 21:42:57 +02:00
Mario Zechner
4f4c42126a
Fix restore for Dossier data.
2023-06-05 21:14:10 +02:00
Mario Zechner
f83af199e6
Merge pull request #73 from mhochsteger/fix_spar_quantities_with_tausendertrennpunkt
...
Fix quantities with "Tausendertrennpunkt" in Spar
2023-06-05 14:45:36 +02:00
Matthias Hochsteger
11e40125f6
Fix quantities with "Tausendertrennpunkt" in Spar
...
Didn't dare to ignore dots globally, as I discovered stuff like "500.000 GR" in
MPREIS data.
Fixes #71
2023-06-05 14:42:42 +02:00
Matthias Hochsteger
505b3c75b3
fallback argument in convertUnit
...
Fixes #70
2023-06-05 14:26:42 +02:00
Matthias Hochsteger
ce13e6cc1d
Units handling for Unimarkt, some clenaup
2023-06-03 15:09:29 +02:00
Mario Zechner
23f512087e
Refactored and fix #55
...
- `readJson()` now just checks for the file extension to decide whether to uncompress instead of taking a flag.
- moved migration logic from index.js to analysis.js:migrateToGzip
- fixed `restore()` in analysis.js
- also calling `migrateToGzip()` in replay.js
- Fix billa canonicalization for Dossier data
- Fix spar canonicalization for Dossier data and data from 2022.
2023-06-02 18:34:14 +02:00
Mario Zechner
8bf0d65d89
Merge branch 'main' into compress-json
2023-06-02 16:56:22 +02:00
Mario Zechner
c6bbd0e03b
Increased maxWidth to 150 in prettier config, formatted all the things. See #52 .
2023-06-02 16:45:54 +02:00
Christian Tschugg
02bd7e5ff8
Compress raw data files on disk, fixes badlogic/heissepreise#51
2023-06-02 16:24:58 +02:00
Matthias Hochsteger
4c55a085b2
Use short-description-2 in spar for quantity/unit
...
Also fix round error in cases without quantity short-description
2023-06-01 07:42:28 +02:00
Matthias Hochsteger
dbe22ef5e6
Merge remote-tracking branch 'origin/main' into unify_units
2023-05-28 22:31:12 +02:00
Mario Zechner
35c3bbbcca
Merge pull request #29 from iantsch/ui-generic-store
...
Refactor UI to generic store syntax
2023-05-28 20:37:45 +02:00
Matthias Hochsteger
524ca40307
Convert all weighted articles to 100g/ml
2023-05-28 19:48:06 +02:00
Matthias Hochsteger
c08ad5dd0a
isWeighted for billa, hofer, lidl, spar
2023-05-28 19:11:18 +02:00
Matthias Hochsteger
e4a438725d
Unify error messages in shops
2023-05-28 18:51:17 +02:00
Matthias Hochsteger
22f380c390
Generate quantity and unified unit for spar data
2023-05-27 22:12:20 +02:00
Mario Zechner
ce8e0fb102
Style changes, Ingest Dossier Data from xls converted to csv see https://www.dossier.at/dossiers/supermaerkte/quellen/anatomie-eines-supermarkts-die-methodik/
2023-05-27 12:33:18 +02:00
Christian Tschugg
a886580469
Refactor UI to generic store syntax
2023-05-26 18:12:29 +02:00
Christian Tschugg
f2ffe5957d
Refactor to generic store syntax
2023-05-26 12:45:30 +02:00
