Consumer Price Index Methodology

How prices are collected, normalized, matched, and published on Libnanews.

Real-time data quality

Operational indicators computed automatically from the pipeline.

Active sources

5

Last run status: ok

Last run at

09 May 2026, 03:37

Connector failures (24h): 1

Open anomalies

24

Needs review queue: 2351

Confirmed rate (7d)

43.18%

Avg score: 0.876

Matching quality by source (7d)

Metro Market Lebanon: 207/447 confirmed (46.31%) | Avg score: 0.876
Carrefour Lebanon: 122/340 confirmed (35.88%) | Avg score: 0.875
Spinneys Lebanon: 27/82 confirmed (32.93%) | Avg score: 0.859
Le Charcutier: 32/35 confirmed (91.43%) | Avg score: 0.950
SuperDokan Lebanon: 17/34 confirmed (50.00%) | Avg score: 0.847

1) Sources and frequency

Data comes from automated connectors per supermarket source.
The target pipeline is: scrape -> normalize -> match -> publish.
Collection runs daily, with manual trigger available in admin.

2) Product and price normalization

Each price keeps its original value and currency (USD, LBP, others).
A USD pivot is computed to compare chains on a common basis.
Formats are standardized: sizes (ml/l/g/kg), packs xN, and unit price.

3) Product matching

Raw listings are linked to canonical products with a similarity score.
Matching combines multilingual tokens, brand, size, unit, and category consistency.
Ambiguous cases are either provisional or sent to validation queue.

4) Publishing and confidence

Publishing states: published, provisional, or quarantined based on confidence.
Displayed confidence mixes scrape quality, matching quality, and unit consistency.
User validations can raise confidence for edge cases.

5) CPI and basket inflation

Libnanews CPI is calculated from a weighted Lebanon reference basket.
Monthly and yearly indicators are provided in USD and LBP.
If history is insufficient, the inflation section stays in data collection mode.

6) Known limitations

Some sources change HTML structure and may temporarily reduce coverage.
Classification errors may still happen on close variants (size/flavor/format).
In-store prices may differ from online prices depending on availability and local promotions.