EHN update figures & draft

Co-Authored-By: joshua vogelstein <jovo@jhu.edu>
Co-Authored-By: Jayanta Dey <52499217+jdey4@users.noreply.github.com>
Co-Authored-By: Will LeVine <levinewill@icloud.com>
Co-Authored-By: Kaleab A Kinfu <51483466+kaleab-k@users.noreply.github.com>

CITATION.cff

@@ -1,12 +1,29 @@
 # YAML 1.2
 ---
-abstract: "Random forests (RF) and deep networks (DN) are two of the primary machine learning methods in current literature and are known to yield differing levels of performance on different data modalities. In particular, RF is one of the leading methods for classifying tabular data, while DN excels at classifying structured data. We wish to further explore and establish the conditions and domains in which each approach excels, particularly in the context of sample size and feature dimension. To address these issues, our team is analyzing the performance of these models across tabular, image, and audio settings using varying model parameters and architectures. For our tabular domain, we used OpenML-CC18, a collection of benchmark datasets for machine learning analysis. For image data, we used CIFAR-10, CIFAR-100, and SVHN datasets. For audio data, we used FSDD and employed spectrogram construction as a preprocessing step. The goal of this project is to observe novel trends in model classification accuracy and wall time across a range of sample sizes and domains. In general, DN would surpass RF at higher class numbers and higher sample sizes."
+abstract: "Deep networks and decision forests (such as random forests and gradient boosted trees) are the leading machine learning methods for structured and tabular data, respectively. Many papers have empirically compared large numbers of classifiers on one or two different domains (e.g., on 100 different tabular data settings). However, a careful conceptual and empirical comparison of these two strategies using the most contemporary best practices has yet to be performed. Conceptually, we illustrate that both can be profitably viewed as “partition and vote” schemes. Specifically, the representation space that they both learn is a partitioning of feature space into a union of convex polytopes. For inference, each decides on the basis of votes from the activated nodes. This formulation allows for a unified basic understanding of the relationship between these methods. Empirically, we compare these two strategies on hundreds of tabular data settings, as well as several vision and auditory settings. Our focus is on datasets with at most 10,000 samples, which represent a large fraction of scientific and biomedical datasets. In general, we found forests to excel at tabular and structured data (vision and audition) with small sample sizes, whereas deep nets performed better on structured data with larger sample sizes. This suggests that further gains in both scenarios may be realized via further combining aspects of forests and networks. We will continue revising this technical report in the coming months with updated results."
 authors:
   -
     affiliation: "Johns Hopkins University, Baltimore, MD"
     family-names: Xu
     given-names: Haoyin
     orcid: "https://orcid.org/0000-0001-8235-4950"
+  -
+    affiliation: "Johns Hopkins University, Baltimore, MD"
+    family-names: Kinfu
+    given-names: Kaleab
+  -
+    affiliation: "Johns Hopkins University, Baltimore, MD"
+    family-names: LeVine
+    given-names: Will
+  -
+    affiliation: "Johns Hopkins University, Baltimore, MD"
+    family-names: Panda
+    given-names: Sambit
+    orcid: "https://orcid.org/0000-0001-8455-4243"
+  -
+    affiliation: "Johns Hopkins University, Baltimore, MD"
+    family-names: Dey
+    given-names: Jayanta
   -
     affiliation: "Johns Hopkins University, Baltimore, MD"
     family-names: Ainsworth
@@ -20,15 +37,22 @@ authors:
     family-names: Kusmanov
     given-names: Madi
   -
-    affiliation: "Johns Hopkins University, Baltimore, MD"
-    family-names: Panda
-    given-names: Sambit
-    orcid: "https://orcid.org/0000-0001-8455-4243"
+    affiliation: "Harvard University, Cambridge, MA"
+    family-names: Engert
+    given-names: Florian
+  -
+    affiliation: "Microsoft Research, Redmond, WA"
+    family-names: White
+    given-names: Christopher
   -
     affiliation: "Johns Hopkins University, Baltimore, MD"
     family-names: Vogelstein
     given-names: Joshua
     orcid: "https://orcid.org/0000-0003-2487-6237"
+  -
+    affiliation: "Johns Hopkins University, Baltimore, MD"
+    family-names: Priebe
+    given-names: Carey
 cff-version: "1.1.0"
 identifiers:
   -
@@ -44,6 +68,6 @@ keywords:
 license: MIT
 message: "If you use this software, please cite it using these metadata."
 repository-code: "https://github.com/neurodata/rf-dn-paper"
-title: "When are Deep Networks really better than Random Forests?"
+title: "Decision Forests vs. Deep Networks: Conceptual Similarities and Empirical Differences at Small Sample Sizes"
 version: "0.0.1"
 ...

README.md

@@ -1,9 +1,9 @@
-# When are Deep Networks really better than Random Forests at small sample sizes?
+# Decision Forests vs. Deep Networks: Conceptual Similarities and Empirical Differences at Small Sample Sizes
 
 [![arXiv](https://img.shields.io/badge/arXiv-2108.13637-red.svg?style=flat)](https://arxiv.org/abs/2108.13637)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![License](https://img.shields.io/badge/License-MIT-blue)](https://opensource.org/licenses/MIT)
 
 **This is preliminary work. More details will be available.**
 
-Random forests (RF) and deep networks (DN) are two of the most popular machine learning methods in the current scientific literature and yield differing levels of performance on different data modalities. We wish to further explore and establish the conditions and domains in which each approach excels, particularly in the context of sample size and feature dimension. To address these issues, we tested the performance of these approaches across tabular, image, and audio settings using varying model parameters and architectures. Our focus is on datasets with at most 10,000 samples, which represent a large fraction of scientific and biomedical datasets. In general, we found RF to excel at tabular and structured data (image and audio) with small sample sizes, whereas DN performed better on structured data with larger sample sizes. Although we plan to continue updating this technical report in the coming months, we believe the current preliminary results may be of interest to others.
+Deep networks and decision forests (such as random forests and gradient boosted trees) are the leading machine learning methods for structured and tabular data, respectively. Many papers have empirically compared large numbers of classifiers on one or two different domains (e.g., on 100 different tabular data settings). However, a careful conceptual and empirical comparison of these two strategies using the most contemporary best practices has yet to be performed. Conceptually, we illustrate that both can be profitably viewed as ''partition and vote'' schemes. Specifically, the representation space that they both learn is a *partitioning* of feature space into a union of convex polytopes. For inference, each decides on the basis of *votes* from the activated nodes. This formulation allows for a unified basic understanding of the relationship between these methods. Empirically, we compare these two strategies on hundreds of tabular data settings, as well as several vision and auditory settings. Our focus is on datasets with at most 10,000 samples, which represent a large fraction of scientific and biomedical datasets. In general, we found forests to excel at tabular and structured data (vision and audition) with small sample sizes, whereas deep nets performed better on structured data with larger sample sizes. This suggests that further gains in both scenarios may be realized via further combining aspects of forests and networks. We will continue revising this technical report in the coming months with updated results.