Machikarte publishes approximately 128 million speech rows from 1,794 Japanese basic municipalities. We continuously verify "where the data comes from", "when it was retrieved", and "whether the content is authentic", and disclose every correction as a full historical record.

Acquisition-time reliability

1. Source transparency

Each municipality's source URL and minutes system type (sophia / ssp / voices / dbsearch / kensakusystem / universal / family / voicetechno — 8 system families) is recorded in municipality_registry.jsonand publicly disclosed.

2. Original document preservation

Every source PDF / HTML is stored in Google Drive and Google Cloud Storage with a SHA-256 hash. Each speech row can be traced back to its original file via source_doc_id.

3. Unique municipality identifier

council_id is generated as a SHA-256 derivative to prevent collisions. To avoid mistaking same-named municipalities (e.g. Shimizu-cho in Hokkaido vs Shizuoka), source URLs are managed independently per municipality and a dictionary of 18 known traps is continuously maintained.

Automated integrity verification

1. Load-time verification

Immediately before each BigQuery write, the data is scored against whether it contains the canonical municipality name and prefecture name. Writes are aborted when the threshold is not met.

Threshold: sample size × 0.3. Below this the write is aborted. Implementation: verify_council_name_match() inscripts/framework/council_name_verify.py.

2. Monthly full-fleet audit

At 04:00 JST on the first day of every month all 1,794 municipalities are mechanically audited (scripts/audit_bq_council_name_match.py). Any flagged inconsistency is investigated and corrected within 24 hours.

3. Additional evidence

• Council size vs municipality population sanity check
• Speech-date year-range sanity check
• speech_order continuity check (continuous_quality_views.sql)
• Duplicate detection (dedupe_speeches.py)

Full disclosure of corrections

Correction logs reflect self-reported errors and corrective effort by Machikarte; they do not indicate the quality or merit of any local council.

Total 49,426 rows (0.039% of all data) corrected and removed. The full history is published daily via the isvd_machikarte.corrections table.

Third-party verifiability

• Each speech's source_url points directly to its public source
• Each speech's source_doc_id resolves to its archived original PDF (via the source_documents table)
• Verification scripts are published on GitHub and can be re-run by anyone
• The BigQuery table correlate-workspace.isvd_machikarte.speeches is read-accessible to approved AI agents and researchers

Quantitative quality indicators (as of 2026-06-02)

Continuous improvement

• Monthly audit results posted to Discord
• Same-name trap dictionary (known_same_name_traps.json) continuously updated (currently 18 entries)
• Audit algorithm refinement + manual spot checks to improve precision
• Inconsistency reports from users / researchers answered within 24 hours

Known limitations

The automated audit's scoring method has known room for improvement, and certain municipalities are difficult to evaluate purely mechanically due to the nature of their minutes. These are addressed through manual spot checks and algorithm refinement. Improvement status is updated continuously on this page.

Found an error? Please contact us via the corrections form. We respond within 24 hours.

Last updated: 2026-06-02