<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "https://jats.nlm.nih.gov/publishing/1.3/JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xml:lang="ru">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Computing, Telecommunication and Control</journal-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Информатика, телекоммуникации и управление</trans-title>
        </trans-title-group>
      </journal-title-group>
      <issn pub-type="epub">2687-0517</issn>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">3</article-id>
      <article-id pub-id-type="doi">10.18721/JCSTCS.18203</article-id>
      <title-group>
        <article-title>Dataset creation for comprehensive performance evaluation of automatic speech recognition systems</article-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Создание набора данных для комплексной оценки производительности систем автоматического распознавания речи</trans-title>
        </trans-title-group>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <name>
            <surname>Andrusenko</surname>
            <given-names>Andrei</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
          <email>andrusenkoau@gmail.com</email>
        </contrib>
        <contrib contrib-type="author">
          <contrib-id contrib-id-type="orcid">0000-0003-1116-7765</contrib-id>
          <contrib-id contrib-id-type="scopus">56049610600</contrib-id>
          <name>
            <surname>Drobintsev</surname>
            <given-names>Pavel</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
          <email>drobintsev_pd@spbstu.ru</email>
        </contrib>
      </contrib-group>
      <aff id="aff1">Peter the Great St. Petersburg Polytechnic University</aff>
      <pub-date publication-format="electronic" date-type="pub" iso-8601-date="2025-06-09">
        <day>09</day>
        <month>06</month>
        <year>2025</year>
      </pub-date>
      <volume>18</volume>
      <issue>2</issue>
      <fpage>33</fpage>
      <lpage>44</lpage>
      <abstract xml:lang="en">
        <p>The performance evaluation of Automatic Speech Recognition (ASR) systems heavily depends on the availability of diverse and representative test datasets encompassing a wide range of complexities in various domains. This work introduces a novel methodology for collecting and preparing datasets for comprehensive ASR system evaluation. The proposed dataset incorporates a modern vocabulary enriched with numerous unique terms and proper nouns, facilitating an in-depth evaluation of overall ASR performance and the effectiveness of context-biasing techniques in computer science. Additionally, the dataset retains critical text features such as Punctuation and Capitalization (P&amp;C), enabling a rigorous evaluation of P&amp;C prediction algorithms. We present a detailed account of the dataset creation process, along with its statistical and qualitative analysis. Furthermore, we benchmark state-of-the-art ASR models, context-biasing approaches, and P&amp;C prediction techniques using the proposed dataset, providing valuable insights into their relative performance.</p>
      </abstract>
      <kwd-group xml:lang="en">
        <kwd>automatic speech recognition</kwd>
        <kwd>test dataset</kwd>
        <kwd>large language models</kwd>
        <kwd>punctuation and capitalization</kwd>
        <kwd>context-biasing</kwd>
      </kwd-group>
    </article-meta>
  </front>
</article>
