#acl +All:read Default = LocalGovPL (Korpus Debat Samorządowych) = LocalGovPL is a large-scale, speaker-annotated corpus of Polish local government meeting transcripts processed using an automatic two-stage LLM pipeline. The corpus consists of 31,899 sessions from 749 councils recorded between 2018 and 2025 (approximately 363M words). It is released in TEI P5 format with explicit links between utterances and registered participants. The corpus covers various levels of local administration – municipalities (PL ''gminy''), counties (PL ''powiaty''), cities (PL ''miasta''), and regional assemblies (PL ''sejmiki województw'') – including both plenary sessions and committee meetings. The primary goal of the resource is to facilitate research on the language of local governance, including studies of argumentation, interactional patterns, policy framing, and social dynamics within institutional dialogue. Beyond linguistic research, the corpus supports applications in speech-to-text alignment, automatic summarization, speaker role identification, and computational social science. == Data Sources == The raw transcripts were collected from two main publicly available sources: 1. Websites maintained by local administrative bodies – a set of specialized HTML extraction parsers was implemented to retrieve and normalise transcripts. 1. [[https://esesja.tv/|eSesja.tv]] – the meeting streaming platform used by local governments, from which transcription files in WebVTT format were downloaded. The dataset covers meetings from November 2018 to June 2025 and includes several thousand hours of deliberation. Due to the decentralised publication practices of local institutions, the source transcripts exhibit substantial variability in format, structure, and language conventions. The preprocessing stage included normalisation of document encoding, removal of irrelevant metadata (e.g., agenda headers or timestamps), and segmentation into individual utterance candidates. == Processing Pipeline == The automatic structuring pipeline consists of two main stages, both powered by large language models (LLMs). === Stage 1: Speaker Extraction === Potential speaker names are identified using a combination of rule-based name recognition and contextual inference performed by LLMs. The models are prompted to detect person names and administrative roles, e.g., Chairperson (PL ''Przewodniczący''), Mayor (PL ''Burmistrz''), Councilor (PL ''Radny''), ensuring both high recall and accurate disambiguation in cases of title repetition or partial name mentions. === Stage 2: Utterance Attribution === The LLMs are then used to assign each utterance segment to one of the previously extracted speakers. This stage requires interpreting discourse cues such as addressing forms, transitions, and speaker introductions. The output is a fully structured transcript in which each utterance is associated with a speaker identifier (speaker name, role, and meeting session). === Processing Configuration === For the public release, both stages were executed end-to-end with !DeepSeek-chat-v3-0324. Long transcripts were processed with a chunking strategy (threshold >1,500 lines, approximately 60,000 characters) and merged by global line numbers. === Throughput and Cost === || '''Metric''' || '''Value''' || || Transcripts processed || 31,899 || || Total input tokens || ~1,100,000,000 || || Total output tokens || ~55,000,000 || || Total processing time (days) || 16.82 || || Total cost (USD) || 373.18 || || Avg input tokens per transcript || 34,038.3 || || Avg output tokens per transcript || 1,742.3 || || Avg generation time (s) || 41.964 || || Avg cost per transcript (USD) || 0.01078 || == Corpus Statistics == The LocalGovPL corpus represents a substantial collection of local government meeting transcripts, spanning over seven years of administrative proceedings across 749 councils. || '''Category''' || '''Count''' || '''Average per Session''' || || '''Basic Statistics''' || || || || Total transcripts || 31,899 || – || || Date range || 2018-11 to 2025-06 || – || || Number of councils || 749 || – || || Transcripts per council || – || 42.59 || || '''Duration Statistics''' || || || || Average session duration || – || 2.23 hours || || '''Content Statistics''' || || || || Total words || 362,664,794 || 11,369 || || Total characters || 2,468,439,776 || 77,383 || || '''Speaker Statistics''' || || || || Average speakers per session || – || 12.77 || || Average utterances per session || – || 80.2 || == Corpus Format == Corpus files are made available in '''XML TEI P5''' format, following the same design choices as the [[https://clip.ipipan.waw.pl/PPC|Polish Parliamentary Corpus (PPC)]], ensuring interoperability with existing tools and facilitating cross-corpus comparisons. Each meeting transcription is represented by a pair of XML files: === Session Header (header.xml) === The `header.xml` file contains the TEI header with document-level metadata and the participant registry, including: * '''`title`''' – meeting title used as the document name (e.g., ''Sesja Rady 30 stycznia 2019'' / Council Session on January 30, 2019) * '''`publisher`''' – the organising body responsible for the session (e.g., ''Rada Miejska Nowego Miasta Lubawskiego'' / Municipal Council of Nowe Miasto Lubawskie) * '''`system`''' – source system label for provenance tracking (e.g., ''Sesja Rady Lokalnej'' / Local Council Session) * '''`house`''' – assembly or chamber type (e.g., ''Rada Powiatu'' / County Council) * '''`sitting ID`''' – numeric identifier of the sitting * '''`type`''' – content type of the source (e.g., ''Transkrypcja sesji'' / Session transcript) * '''`total rows`''' – number of input transcript rows prior to structuring * '''`speaker count`''' – number of distinct speakers recognised in the session * '''`date`''' – session date in ISO format (e.g., 2019-01-30) Each '''`person`''' in the participant list is uniquely identified and carries a normalised name and role: * `person[@xml:id]` provides a stable identifier (e.g., `chairman_of_municipal_council`) * `persName` holds the display name (e.g., ''Przewodniczący Rady Miejskiej'' / Chairman of the Municipal Council) * `@role` encodes the role (e.g., ''Burmistrz Gminy'' / Mayor of the Municipality) === Utterance Structure (text_structure.xml) === The `text_structure.xml` file contains the speech content segmented into `
`isions and ``tterances. Each utterance carries: * '''`xml:id`''' – a unique utterance identifier (e.g., `u-1.1`) * '''`who`''' – a pointer to the speaking participant using a TEI cross-reference to `header.xml` * '''`start`''' / '''`end`''' – timestamps delimiting the utterance span in the source recording Documents may be wrapped in a `` element that includes `header.xml` via XML Inclusions (`xi:include`). The logical linkage between utterances (`/@who`) and declared speakers (`/person[@xml:id]`) is maintained regardless of wrapping. == Evaluation == === Test Dataset === A subset of '''30 transcripts''' from '''23 councils''' (spanning '''June 2022 to January 2025''') was manually annotated to create a reference benchmark for evaluating both speaker identification and attribution. Each session lasts approximately '''2.36 hours''' and contains nearly '''13,682 words''', with an average of '''17.27 speakers''' contributing about '''102.87 utterances''' per session. === Speaker Identification (Stage 1) === Macro-averaged precision, recall, and F1 over the 30-session benchmark (with relaxed identity equivalence): || '''Model configuration''' || '''Macro P''' || '''Macro R''' || '''Macro F1''' || || Gemini-2.5-pro || 0.9058 || 0.8814 || 0.8786 || || Gemini-2.5-flash || 0.9071 || 0.8800 || 0.8783 || || !DeepSeek-chat-v3-0324 || 0.8287 || 0.8375 || 0.8169 || || !DeepSeek-r1-0528 || 0.6281 || 0.5887 || 0.5904 || || Llama-3.3-70b-instruct || 0.3537 || 0.3673 || 0.3491 || === Speaker Attribution (Stage 2) === Speaker-aware word error rate (sWER; lower is better) averaged across 30 sessions under three evaluation protocols: || '''Model configuration''' || '''Abstract''' || '''GT participants''' || '''Relaxed names''' || || Gemini-2.5-pro || 0.0393 || 0.0460 || 0.0592 || || Gemini-2.5-flash || 0.0907 || 0.1287 || 0.1257 || || !DeepSeek-chat-v3-0324 || 0.2061 || 0.2094 || 0.2381 || || !DeepSeek-r1-0528 || 0.4582 || 0.2498 || 0.4684 || || Llama-3.3-70b-instruct || 0.6969 || 0.7945 || 0.7378 || The three evaluation protocols are: * '''Abstract speaker attribution''' – speakers are treated as abstract entities (e.g., speaker-1, speaker-2); the Hungarian algorithm finds the optimal one-to-one mapping. This isolates the utterance attribution task from name recognition. * '''Ground-truth participants''' – the system receives the gold-standard list of participants and only needs to determine which known speaker is talking at each point. * '''End-to-end with relaxed name matching''' – both stages run without external assistance. A predicted speaker matches the reference if surnames match, titles/roles match, or the Levenshtein similarity between names is ≥ 0.8. == Download == /* * [[https://kdp.ipipan.waw.pl/static/localgov/locgovpl-tei.tar.gz|Full corpus (TEI XML)]] */ * [[https://kdp.ipipan.waw.pl/static/localgov/locgovpl-sample.tar.gz|Sample subset (30 annotated sessions)]] == Searching the Corpus == * [[https://locgovpl.ipipan.waw.pl/|LocalGovPL Search Engine]] (in Polish) == Licence == All data used in this corpus originate from official public records published by governmental institutions. The collection and redistribution of these materials is conducted in compliance with the '''Polish Act of 11 August 2021 on Open Data and the Re-use of Public Sector Information''' (Dz.U. 2021 poz. 1641), which mandates the openness of public sector information for reuse. The corpus does not include any personal data beyond names of public officials acting in their professional capacity. The resource is intended for '''research and educational purposes''', and all derivative uses must comply with applicable open-data regulations. '''Risk of Misattribution.''' As an automatically processed resource, the corpus may contain attribution errors (sWER ≈ 4–6%). Users should exercise caution when attributing specific controversial or sensitive statements to individual public officials based solely on this automated dataset. == See Also == * [[https://clip.ipipan.waw.pl/PPC|Polish Parliamentary Corpus (PPC)]] – the corpus of Polish parliamentary (Sejm and Senate) proceedings encoded in TEI P5 format, which served as the design model for LocalGovPL. * [[https://www.clarin.eu/parlamint|ParlaMint]] – a project providing speaker- and role-annotated parliamentary proceedings across many countries. * [[https://councildataproject.org/|Council Data Project (CDP)]] – an open infrastructure for collecting and curating municipal governance data. * [[https://esesja.tv/|eSesja.tv]] – the meeting streaming platform used by Polish local governments, one of the primary data sources for this corpus. === Funding === The corpus was financed by the European Regional Development Fund as a part of the 2014–2020 Smart Growth Operational Programme, CLARIN — Common Language Resources and Technology Infrastructure, project no. POIR.04.02.00–00C002/19, the Polish Ministry of Education and Science grant 2022/WK/09, continued as part of the investment: CLARIN ERIC – European Research Infrastructure Consortium: Common Language Resources and Technology Infrastructure (period: 2024-2026) funded by the Polish Ministry of Science and Higher Education (Programme: ”Support for the participation of Polish scientific teams in international research infrastructure projects”), agreement number 2024/WK/01 and by CLARIN-PL, the European Regional Development Fund, FENG programme, agreement number FENG.02.04-IP.040004/24. == Licence == {{http://i.creativecommons.org/l/by/4.0/88x31.png}} [[http://creativecommons.org/licenses/by/4.0/deed.en_US|Creative Commons Attribution 4.0 Unported License]] == Please cite == Czerski D., Ogrodniczuk M. (2026). ''LocalGovPL: A Corpus of Speaker-Attributed Polish Local Government Transcripts''. Proceedings of the 15th Language Resources and Evaluation Conference (LREC 2026). Palma de Mallorca, 2026. European Language Resources Association (ELRA). <>