Consumer Price Index Methodology

How prices are collected, normalized, matched, and published on Libnanews.

Real-time data quality

Operational indicators computed automatically from the pipeline.

Active sources

5

Last run status: ok

Last run at

10 Aug 2026, 03:33

Connector failures (24h): 0

Open anomalies

20

Needs review queue: 4723

Confirmed rate (7d)

47.61%

Avg score: 0.882

Matching quality by source (7d)

Metro Market Lebanon: 178/383 confirmed (46.48%) | Avg score: 0.885
Carrefour Lebanon: 158/328 confirmed (48.17%) | Avg score: 0.881
Spinneys Lebanon: 31/72 confirmed (43.06%) | Avg score: 0.871
SuperDokan Lebanon: 21/32 confirmed (65.62%) | Avg score: 0.884

1) Sources and frequency

Data comes from automated connectors per supermarket source.
The target pipeline is: scrape -> normalize -> match -> publish.
Collection runs daily, with manual trigger available in admin.

2) Product and price normalization

Each price keeps its original value and currency (USD, LBP, others).
A USD pivot is computed to compare chains on a common basis.
Formats are standardized: sizes (ml/l/g/kg), packs xN, and unit price.

3) Product matching

Raw listings are linked to canonical products with a similarity score.
Matching combines multilingual tokens, brand, size, unit, and category consistency.
Ambiguous cases are either provisional or sent to validation queue.

4) Publishing and confidence

Publishing states: published, provisional, or quarantined based on confidence.
Displayed confidence mixes scrape quality, matching quality, and unit consistency.
User validations can raise confidence for edge cases.

5) CPI and basket inflation

Libnanews CPI is calculated from a weighted Lebanon reference basket.
Monthly and yearly indicators are provided in USD and LBP.
If history is insufficient, the inflation section stays in data collection mode.

6) Known limitations

Some sources change HTML structure and may temporarily reduce coverage.
Classification errors may still happen on close variants (size/flavor/format).
In-store prices may differ from online prices depending on availability and local promotions.