<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "https://jats.nlm.nih.gov/publishing/1.3/JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xml:lang="ru">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Computing, Telecommunication and Control</journal-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Информатика, телекоммуникации и управление</trans-title>
        </trans-title-group>
      </journal-title-group>
      <issn pub-type="epub">2687-0517</issn>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">3</article-id>
      <article-id pub-id-type="doi">10.18721/JCSTCS.19303</article-id>
      <title-group>
        <article-title>Kernel Regression with Tree-Induced Metrics</article-title>
        <trans-title-group xml:lang="ru">
          <trans-title>Ядерная регрессия с индуцированными деревьями метриками</trans-title>
        </trans-title-group>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <contrib-id contrib-id-type="orcid">0009-0005-6368-0001</contrib-id>
          <name>
            <surname>Goriunov</surname>
            <given-names>Maksim</given-names>
          </name>
          <email>goryunov_myu@spbstu.ru</email>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Konstantinov</surname>
            <given-names>Andrei</given-names>
          </name>
          <xref ref-type="aff" rid="aff1"/>
          <email>andrue.konst@gmail.com</email>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Novikov</surname>
            <given-names>Fedor</given-names>
          </name>
        </contrib>
      </contrib-group>
      <aff id="aff1">Peter the Great St. Petersburg Polytechnic University</aff>
      <pub-date publication-format="electronic" date-type="pub" iso-8601-date="2026-09-30">
        <day>30</day>
        <month>09</month>
        <year>2026</year>
      </pub-date>
      <volume>19</volume>
      <issue>3</issue>
      <fpage>29</fpage>
      <lpage>45</lpage>
      <abstract xml:lang="en">
        <p>This paper presents a family of kernel models based on metrics derived from the structure of decision trees and random forests. Unlike the standard random-forest proximity, which only checks whether two observations fall into the same leaf, the proposed metrics also use information about decision paths, the depth of their divergence, and the depth of their lowest common ancestor. A weighted variant with trainable depth-dependent parameters is also introduced. The tree-based metrics are aggregated over a forest and used to construct exponential kernels. They can also be combined with the Euclidean distance in the original feature space. We prove that the resulting kernels are positive semidefinite and can therefore be used in standard kernel methods, including Nadaraya–Watson regression and kernel ridge regression. The proposed models are evaluated on classification datasets and in a controlled source-to-target regression experiment with a limited number of labeled target observations. The results show that information about the internal structure of decision trees can be useful beyond simple leaf coincidence. Several proposed variants improve upon conventional geometric kernels and the standard random-forest proximity kernel, while the best-performing metric depends on the prediction method and the dataset.</p>
      </abstract>
      <kwd-group xml:lang="en">
        <kwd>kernel regression</kwd>
        <kwd>decision trees</kwd>
        <kwd>tree-based metrics</kwd>
        <kwd>random forests</kwd>
        <kwd>Nadaraya–Watson estimator</kwd>
        <kwd>kernel ridge regression</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <back>
    <ref-list>
      <title>References</title>
      <ref id="ref1">
        <mixed-citation publication-type="journal">Breiman L. Random Forests. Machine Learning, 2001, Vol. 45, Pp. 5–32. DOI: 10.1023/A:1010933404324</mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation publication-type="journal">Biau G., Scornet E. A random forest guided tour. TEST, 2016, Vol. 25, Pp. 197–227. DOI: 10.1007/s11749-016-0481-7</mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation publication-type="journal">Geurts P., Ernst D., Wehenkel L. Extremely randomized trees. Machine Learning, 2006, Vol. 63, Pp. 3–42. DOI: 10.1007/s10994-006-6226-1</mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation publication-type="journal">Scornet E. Random Forests and Kernel Methods. IEEE Transactions on Information Theory, 2016, Vol. 62, No. 3, Pp. 1485–1500. DOI: 10.1109/TIT.2016.2514489</mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation publication-type="journal">Lin Y., Jeon Y. Random Forests and Adaptive Nearest Neighbors. Journal of the American Statistical Association, 2006, Vol. 101, No. 474, Pp. 578–590. DOI: 10.1198/016214505000001230</mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation publication-type="journal">Biau G., Devroye L., Lugosi G. Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research, 2008, Vol. 9, Pp. 2015–2033.</mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation publication-type="journal">Biau G. Analysis of a random forests model. Journal of Machine Learning Research, 2012, Vol. 13, Pp. 1063–1095.</mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation publication-type="journal">Iakovidis I., Arcozzi N. Improved convergence rates for some kernel random forest algorithms. Mathematics in Engineering, 2024, Vol. 6, No. 2, Pp. 305–338. DOI: 10.3934/mine.2024013</mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation publication-type="journal">Shi T., Horvath S. Unsupervised Learning with Random Forest Predictors. Journal of Computational and Graphical Statistics, 2006, Vol. 15, No. 1, Pp. 118–138. DOI: 10.1198/106186006X94072</mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation publication-type="journal">Mantero A., Ishwaran H. Unsupervised random forests. Statistical Analysis and Data Mining, 2021, Vol. 14, No. 2, Pp. 144–167. DOI: 10.1002/sam.11498</mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation publication-type="journal">Sage A.J., Genschel U., Nettleton D. Tree aggregation for random forest class probability estimation. Statistical Analysis and Data Mining, 2020, Vol. 13, No. 2, Pp. 134–150. DOI: 10.1002/sam.11446</mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation publication-type="journal">Meinshausen N. Quantile Regression Forests. Journal of Machine Learning Research, 2006, Vol. 7, No. 35, Pp. 983–999.</mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation publication-type="journal">Rhodes J.S., Cutler A., Moon K.R. Geometry- and Accuracy-Preserving Random Forest Proximities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, Vol. 45, No. 9, Pp. 10947–10959. DOI: 10.1109/TPAMI.2023.3263774</mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation publication-type="journal">Laabs B.-H., Westenberger A., König I.R. Identification of representative trees in random forests based on a new tree-based distance measure. Advances in Data Analysis and Classification, 2024, Vol. 18, Pp. 363–380. DOI: 10.1007/s11634-023-00537-7</mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation publication-type="journal">Xiong C., Johnson D., Xu R., Corso J.J. Random forests for metric learning with implicit pairwise position dependence. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discove-ry and Data Mining, 2012, Pp. 958–966. DOI: 10.1145/2339530.2339680</mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation publication-type="journal">Bicego M., Cicalese F. Computing Random Forest-distances in the presence of missing data. ACM Transactions on Knowledge Discovery from Data, 2024, Vol. 18, No. 7, Art. no. 180. DOI: 10.1145/3656345</mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation publication-type="journal">Lee S., Lee C., Mun K.G., Kim D. Decision Tree Algorithm Considering Distances Between Classes. IEEE Access, 2022, Vol. 10, Pp. 69750–69756. DOI: 10.1109/ACCESS.2022.3187172</mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation publication-type="journal">Bicego M., Cicalese F. An Interesting Property of Random Forest Distances with Respect to the Cur-se of Dimensionality. In: Structural, Syntactic, and Statistical Pattern Recognition (eds. A. Torsello, L. Rossi, L. Cosmo, G. Minello), 2025, Vol. 15444, Pp. 188–198. DOI: 10.1007/978-3-031-80507-3_19</mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation publication-type="journal">Bicego M., Cicalese F. On the Good Behaviour of Extremely Randomized Trees in Random Forest-Distance Computation. Machine Learning and Knowledge Discovery in Databases: Research Track: European Conference (ECML PKDD), 2023, Vol. IV, Pp. 645–660. DOI: 10.1007/978-3-031-43421-1_38</mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation publication-type="journal">Schoenberg I.J. Metric spaces and positive definite functions. Transactions of the American Mathe-matical Society, 1938, Vol. 44, Pp. 522–536. DOI: 10.1090/S0002-9947-1938-1501980-0</mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation publication-type="journal">Kelly M., Longjohn R., Nottingham K. The UCI Machine Learning Repository [online]. Available: https://archive.ics.uci.edu (Accessed 30.08.2026).</mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation publication-type="journal"> </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>
