LVMED: Dataset of Latvian text normalisation samples for the medical domain

CLARIN Centre of Latvian language resources and tools

Authors: Lasmanis, Viesturs Jūlijs and Grūzītis, Normunds

Item identifier: http://hdl.handle.net/20.500.12574/85

Project URL: https://gitlab.com/ailab/lvmed

Demo URL: https://huggingface.co/AiLab-IMCS-UL/lvmed

Date issued: 2023-05

Type: corpus, text

Size: 79834 sentences

Language(s): Latvian

Description: The CSV dataset contains sentence pairs for a text-to-text transformation task: given a sentence that contains 0..n abbreviations, rewrite (normalize) the sentence in full words (word forms). Training dataset: 64,665 sentence pairs Validation dataset: 7,185 sentence pairs. Testing dataset: 7,984 sentence pairs. All sentences are extracted from a public web corpus (https://korpuss.lv/id/Tīmeklis2020) and contain at least one medical term.

Publisher: AiLab IMCS UL

Subject(s): medical domain text normalization

Collection(s): Language resources and tools of AiLab IMCS UL

Show full item record

Files in this item

This item is

Publicly Available

and licensed under:
Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)

Name: dataset.zip
Size: 6.73 MB
Format: application/zip
Description: Sentence pairs provided in a two-column CSV format.
MD5: 4f02442c2684d7a64cea9bdc8ddcde3b

Download file Preview

File Preview

dataset
- train.csv23 MB
- test.csv2 MB
- statistics
  - term_counter.csv8 kB
  - abbrev_counter.csv6 kB
  - pair_counter.csv14 kB
- valid.csv2 MB

Partners, Coordination, Funding

Repository

More

Activities of CLARIN Latvia are supported by project of the European Regional Development Fund Nr. 1.1.1.5/18/I/016 University of Latvia and institutes in the European Research Area - Excellency, activity, mobility, capacity.

This platform runs under the software developed for the LINDAT/CLARIN repository for linguistics , available on GitHub