Consumer Price Index Methodology

How prices are collected, normalized, matched, and published on Libnanews.

Real-time data quality

Operational indicators computed automatically from the pipeline.

Active sources

5

Last run status: error

Last run at

09 Oct 2026, 03:35

Connector failures (24h): 0

Open anomalies

30

Needs review queue: 5724

Confirmed rate (7d)

45.61%

Avg score: 0.875

Matching quality by source (7d)

Metro Market Lebanon: 243/490 confirmed (49.59%) | Avg score: 0.880
Carrefour Lebanon: 203/466 confirmed (43.56%) | Avg score: 0.870
Spinneys Lebanon: 21/68 confirmed (30.88%) | Avg score: 0.870

1) Sources and frequency

Data comes from automated connectors per supermarket source.
The target pipeline is: scrape -> normalize -> match -> publish.
Collection runs daily, with manual trigger available in admin.

2) Product and price normalization

Each price keeps its original value and currency (USD, LBP, others).
A USD pivot is computed to compare chains on a common basis.
Formats are standardized: sizes (ml/l/g/kg), packs xN, and unit price.

3) Product matching

Raw listings are linked to canonical products with a similarity score.
Matching combines multilingual tokens, brand, size, unit, and category consistency.
Ambiguous cases are either provisional or sent to validation queue.

4) Publishing and confidence

Publishing states: published, provisional, or quarantined based on confidence.
Displayed confidence mixes scrape quality, matching quality, and unit consistency.
User validations can raise confidence for edge cases.

5) CPI and basket inflation

Libnanews CPI is calculated from a weighted Lebanon reference basket.
Monthly and yearly indicators are provided in USD and LBP.
If history is insufficient, the inflation section stays in data collection mode.

6) Known limitations

Some sources change HTML structure and may temporarily reduce coverage.
Classification errors may still happen on close variants (size/flavor/format).
In-store prices may differ from online prices depending on availability and local promotions.