Posted in r/MachineLearning by u/tsssuh • 207 points and 29 comments
This is an automated archive made by the Lemmit Bot.

The original was posted on /r/machinelearning by /u/tsssuh on 2024-10-21 14:52:17+00:00.

I recently came across a paper titled “Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products” by Shengjie Luo, Tianlang Chen, Aditi S. Krishnapriyan (which I’ll call GTP). After reading it, I have some very strong suspicions about this paper. It appears that the core proposed methodologies of this paper are strikingly similar to those in an older paper in computer graphics, titled “Fast and Accurate Spherical Harmonics Products” by Hanggao, Xin et al (which I’ll call FSHP). This is concerning because the GTP paper does not cite the FSHP paper.

The GTP authors have been notified of this issue nearly 5 months ago, but they have completely ignored the concerns raised.

Paper links:

  • GTP:
  • FSHP:

I wanted to share this with the ML research community to get some of your thoughts on this matter. Below, I provide just a few examples that illustrate the similarities between the two papers (although there are many more). In each case, I will show a side-by-side comparison of the relevant text snippets from both papers, with the key similarities highlighted with colored boxes.

  • Identical Core Methodology: Both papers propose a method to compute triple products (called tensor products in GTP) of spherical functions represented as spherical harmonics basis expansions. They both convert the SH functions to the 2D Fourier domain (blue highlight), where they can use the Fast Fourier Transform (FFT) to compute the triple products efficiently (yellow highlight). They then convert the results back to SH space (blue highlight). See the picture below for a side-by-side comparison of the core methodology in both papers.

  • Change-of-basis to the 2D Fourier domain: The details of the conversion from the SH basis to the 2D Fourier domain are nearly identical in both papers. The equations are identical (green highlight), and both papers make the observation that the conversion coefficients between these two bases are sparse, non-zero only when m=+/-t (red highlight).

  • Time complexity of computing triple products using FFT: As mentioned before, both papers use the Fast Fourier Transform (FFT) to compute the triple products efficiently. They then both dedicate a subsection to deriving the time complexity of this operation. The derivation is identical (green highlight), and both papers arrive at the same O(L3) complexity (blue highlight).

  • Computing higher-order products using a divide-and-conquer approach: Both papers then tackle the product of higher order products (i.e., the product of 3 or more spherical functions). They both begin by observing the associativity of the 2D convolution (red highlight), and they use this fact to develop the same divide-and-conquer approach to compute these higher-order products (green highlight). Finally, they both come up with the same time complexity for this operation (green highlight). In addition, they both derive the same time complexity for the naive approach to computing these products (blue highlight).

  • The GTP authors have been notified of this issue nearly 5 months ago, but they have completely ignored the concerns raised. This is quite alarming. GitHub issue:

Additional Concerns:

  • The GTP authors promised to publicly release the code and model weights, but it has now been 5 months since ICLR, and they have not done so. This raises concerns about the reproducibility and transparency of their work. Taking a look at the primary authors’ GitHub repositories for their previous works, it seems that this is a pattern of behavior.
  • The table 1 experimental results are a bit misleading. The authors claim that their method performs better than the state-of-the-art methods. However, their model simply takes the previous state-of-the-art model (EquiformerV2) and adds their new layer to every block. This would be analogous to some efficient linear attention method (e.g., Linformer) claiming to outperform the state-of-the-art transformer model by simply adding their method to every block of the transformer. That model would strictly be slower and less efficient than the original transformer model.
  • To make things even worse (regarding the additional concern above), the authors’ efficiency comparison figures seemingly do not account for the overhead of EquiformerV2; only the new layer they add. This is misleading (to the point of being dishonest) as the new layer is only a small part of the model, and the overhead of EquiformerV2 is not accounted for in the efficiency comparison figures.

I believe that this type of behavior should not be tolerated in the ML research community. Not only does it do a disservice to the original authors of the FSHP paper, but it also undermines the integrity of the research community as a whole. It is also worth noting that this type of behavior may inadvertently harm senior authors who are advising numerous concurrent projects. While senior authors bear the ultimate responsibility for the work produced in their labs, it can be challenging to thoroughly scrutinize every detail of each project. Nevertheless, this incident highlights the importance of establishing robust processes to ensure the integrity and originality of research outputs, even in large and complex research environments.

TL;DR: The core methodologies and insights of the GTP paper are strikingly similar to those in the FSHP paper, but the GTP paper does not cite the FSHP paper. The GTP authors have been notified of this issue nearly 5 months ago but have completely ignored the concerns raised. This raises serious concerns about the integrity and transparency of the GTP paper.