<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "https://jats.nlm.nih.gov/publishing/1.3/JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xml:lang="ru">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Computing, Telecommunication and Control</journal-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Информатика, телекоммуникации и управление</trans-title>
        </trans-title-group>
      </journal-title-group>
      <issn pub-type="epub">2687-0517</issn>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">5</article-id>
      <article-id pub-id-type="doi">10.18721/JCSTCS.17105</article-id>
      <title-group>
        <article-title>The study of the vision transformer architecture by explainability methods</article-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Исследование архитектуры визуального трансформера методами объяснимости</trans-title>
        </trans-title-group>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <name>
            <surname>Utkin</surname>
            <given-names>Ivan</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Shkuropatsky</surname>
            <given-names>Vitaly</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
          <email>vitalius-47@mail.ru</email>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Pronikov</surname>
            <given-names>Alexander</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Rakov</surname>
            <given-names>Evgeniy</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
        </contrib>
      </contrib-group>
      <aff id="aff1">Mozhaisky Military Space Academy</aff>
      <pub-date publication-format="electronic" date-type="pub" iso-8601-date="2024-03-29">
        <day>29</day>
        <month>03</month>
        <year>2024</year>
      </pub-date>
      <volume>17</volume>
      <issue>1</issue>
      <fpage>54</fpage>
      <lpage>64</lpage>
      <abstract xml:lang="en">
        <p>The article discusses issues of explainability of the operating principles of a machine learning model. As the architecture of the model, one of the types of transformer is considered, the task of which is to classify images based on the popular “ImageNet-1000” dataset. This type of transformer is also called vision transformer and can serve either as a standalone model or as part of a more complex architecture. The explainability methods included activation maps of classes, which were calculated by applying algorithms based on forward and backward propagation of image tensors through the components of the transformer: multi-head attention layers and fully connected multilayer networks. The aim of the work is to increase the explainability of the internal processes of the functioning of the vision transformer by analyzing the obtained activation maps and calculating a metric to evaluate their explainability. The results of the study reveal patterns that reflect the mechanisms of operation of the vision transformer in solving the image classification problem, as well as evaluating the importance of the identified classification features through the use of the explainability metric.</p>
      </abstract>
      <kwd-group xml:lang="en">
        <kwd>machine learning model</kwd>
        <kwd>explainability</kwd>
        <kwd>visual transformer</kwd>
        <kwd>encoder</kwd>
        <kwd>attention mechanism</kwd>
        <kwd>class activation maps</kwd>
        <kwd>back propagation activation maps</kwd>
      </kwd-group>
    </article-meta>
  </front>
</article>
