Can Hyperbolic Fine-Tuning Close the Gap Between What LLMs Say and Do? A Null Result

A Test of Hyperbolic Low-Rank Adaptation for Self-Report Training

Zakiriya Gladney¹

Faculty Mentor: R. Michael Alvarez²

¹Harvard University, Cambridge, MA, USA

²California Institute of Technology, Pasadena, CA, USA

1Introduction

When we want to understand how individuals behave, social scientists often pose surveys, asking respondents to describe themselves. Individuals’ self-reported traits predict what they do. LLMs can answer the same questionnaires, and after alignment training they give consistent, human-like answers. But those answers do not predict the models’ behavior.¹ One can imagine a model rating itself as unlikely to tell people what they want to hear, only to then agree with a user who asserts that the Earth is flat.

Self-CTRL is one recent attempt to close this gap. It trains a model with reinforcement learning, rewarding self-reports that are consistent with the model’s behavior.² This project asks what might cause the gap in the first place, and what might limit methods like Self-CTRL. I approach that question through geometry.

Personality measures are hierarchical. The BFI-2 inventory groups 60 items into 15 facets and the facets into 5 broad domains.³ Hierarchies are hard to represent faithfully in flat, Euclidean space: the number of nodes in a tree grows exponentially with depth, while the room available in Euclidean space grows only polynomially. Hyperbolic space grows exponentially, so trees fit into it with much less distortion.⁴ Concepts inside LLMs already show hierarchical structure.⁵ If a model’s representation of personality is squeezed into flat space, the link between its trait self-reports and its behavior could be distorted, contributing to the say-do gap.

Most fine-tuning uses low-rank adaptation (LoRA), which adds a small trainable update to each weight matrix in ordinary Euclidean space.⁶ HypLoRA performs the same update on a hyperbolic surface, the Lorentz hyperboloid, before mapping the result back.⁷ My research question was: does Euclidean space distort this psychological structure, and does letting an adapter work in hyperbolic space restore it?

I found that HypLoRA, as published, does not train at the learning rate and step budget I used. Its gradients are about 38 times smaller than LoRA’s, for reasons I have not yet identified. When I corrected the update size, the hyperbolic adapter trained, but its results could not be distinguished from LoRA’s. I also show that in this design the hyperbolic and Euclidean updates always point in the same direction, so geometry can change only how large each update is.

2Results

2.1 Published HypLoRA barely trains

The simplest measure of whether an adapter is learning is the size of its gradient. Figure 1 shows the median gradient norm for each run. LoRA and size-matched HypLoRA both fall between about 1.7 and 2.7. Published HypLoRA falls between 0.056 and 0.064 in all three seeds, about 38 times lower. Its largest value is more than 25 times smaller than the smallest value in either other condition.

The divergence from the starting model tells the same story (Figure 2). Self-CTRL includes a penalty, measured as Kullback-Leibler divergence, that keeps the model close to its starting point. All three conditions began at the same KL value, 0.313. From the first 50 training steps to the last 50, LoRA reduced it 18.6-fold. Published HypLoRA reduced it only 1.4-fold. Size-matched HypLoRA followed LoRA’s path through step 250; its last two blocks are raised by a checkpoint-resume artifact described under Limitations.

Because the adapter barely changed, the model’s self-reports barely changed either. Trait spread, the variation in the model’s trait self-reports, moved by at most 0.011 in every published-HypLoRA seed (Figure 3). The effective rank of the BFI-2 item embeddings rose in all three LoRA seeds and all three size-matched seeds, and changed by at most 0.015 in published HypLoRA (Appendix Table A2).

2.2 With update size matched, HypLoRA cannot be told apart from LoRA

The size-matched condition rescales each hyperbolic update to the size the Euclidean update would have had. This removes the overall scale difference and leaves only the shape of the update. This condition trained normally: its gradient norms and its early KL curve match LoRA’s (Figures 1 and 2).

But it did not differ from LoRA in behavior. Alignment-%, the share of trait–task relationships whose sign matches the literature, moved inconsistently in every condition. Its 95% confidence intervals were 21 to 31 percentage points wide, wider than any difference between conditions (Appendix Table A3). Trait spread rose sharply in one seed and barely in the other two, for both LoRA and size-matched HypLoRA (Figure 3).

To check that I had not simply chosen the wrong measures, I compared each condition with LoRA on all 80 numeric quantities recorded at each checkpoint. Size-matched HypLoRA differed from LoRA in the same direction across all three seeds on 8 of the 80, fewer than the 20 expected by chance. Published HypLoRA did so on 29 of the 80.

2.3 Weight-space comparisons were inconclusive

I also compared the trained adapters directly, using the cosine similarity between runs’ weight updates. Runs of the same method with different seeds had nearly orthogonal updates (cosine about 0.0003 for LoRA). This is expected. LoRA’s A matrix starts random, depends on the seed, and stays close to its starting value during training,^8, 9 so each run’s update lies in a different random subspace. Two adapters in different subspaces can still affect the model’s outputs in similar ways, so weight similarity cannot tell us whether the conditions learned the same thing (Appendix Table A5).

3Discussion

Why published HypLoRA barely trains is unresolved. The submitted version of this report explained the small gradients with a numerical clamp in HypLoRA’s return map, which would pin the adapter’s output near zero until the update grew large enough to escape it. That explanation does not apply to the code I ran. My implementation computes the return map as [asinh(u)/u] · x_space (see Methods), which needs no clamp near zero, so the adapter’s output tracks its input at every scale. The suppression itself is real: the gradient norms and the KL divergence measure it directly. Its cause is not yet identified. Curvature is a learnable parameter in HypLoRA, but it changed by less than 0.1% in every run, so in practice I tested a fixed curvature.

Geometry can change update size, not direction. In HypLoRA, the update is scaled by a positive factor on the way into hyperbolic space and scaled again on the way out. Because the LoRA matrices are linear, the direction of the update does not change. I measured a cosine similarity of 1.0000 between the hyperbolic and Euclidean updates for every token. What the geometry does change is how each token’s update is scaled: tokens with very large hidden states are compressed compared with what LoRA would apply. Once the overall size of the update is matched, this per-token scaling is the only remaining difference between HypLoRA and LoRA.

What the null result means. I found no detectable difference between size-matched HypLoRA and LoRA, but this design had limited power. Three seeds are too few for a significance test: with three seeds, the smallest possible p-value from a paired sign test is 0.25. The reward signal was also sparse, with a median reward per step of zero in eight of nine runs. And outcomes depended strongly on the seed, as the trait spread results show. A real but modest effect of geometry could be hidden in that variation. The result should be read as no detectable difference at this power.

Limitations.

• Each condition had three seeds, and only one curvature value was tested.
• The hyperbolic runs exceeded the computing cluster’s time limit and were resumed from checkpoints; the LoRA runs were not. Four of the six resumed runs show a jump in KL at the resume point. This cannot explain the gradient gap, which is present from the first steps, but it adds variance to the hyperbolic conditions.
• HypLoRA could differ from LoRA for reasons other than geometry, such as different optimization dynamics. The size-matched control addresses the largest of these, update scale, but does not rule out the rest.
• Only the adapter is hyperbolic. This is not a comparison between a fully hyperbolic LLM and a Euclidean one; existing hyperbolic LLMs are small, especially if training data must be held constant.

Future directions.

• Trace published HypLoRA’s gradients through each step of the hyperbolic layer to find why they are so small, and sweep the learning rate and training length to test whether it trains, and behaves differently from LoRA, given more budget.
• Use more seeds and a denser reward signal, and compare conditions by their outputs or activations on a fixed set of prompts rather than by their weights.
• Test the per-token scaling directly by applying it to an ordinary LoRA adapter, which isolates the geometric effect without the hyperbolic machinery.

4Methods

Model and adapters

All runs adapt allenai/OLMo-7B-Instruct-hf¹⁰ in bfloat16 on a single NVIDIA A100 80 GB GPU. Adapters use rank 32, α = 64, no dropout, and target the q_proj, k_proj, v_proj, up_proj, and down_proj modules. These settings are identical across all nine runs.

HypLoRA implementation

The hyperbolic adapter follows the equations in the HypLoRA paper rather than its released code. The Lorentz re-projection follows the paper’s Eq. 9, and the logarithmic map is computed as [asinh(u)/u] · x_space. Curvature is learnable and kept positive by a softplus; its raw value is initialized at 1.0, giving an effective curvature of 1.313.

Training

All conditions are trained with Self-CTRL² using group-relative policy optimization for 400 steps. Each group has 8 rollouts sampled at temperature 1.0 with top-p 0.9 and a 150-token limit, with 32 items per rollout. The batch size is 10 with gradient accumulation 2 and micro-batch size 1. The optimizer is AdamW with learning rate 10⁻⁵, betas (0.9, 0.95), and no weight decay. A KL penalty toward the reference policy has coefficient 0.1 and uses 8 reference samples.

Conditions

Each of the three conditions was run with seeds 0, 1, and 2. LoRA is the Euclidean baseline. HypLoRA, published uses the method with no correction. HypLoRA, size-matched multiplies each layer’s hyperbolic update by one scalar per forward pass so that its Frobenius norm equals that of the Euclidean update.

Measures

Checkpoints are evaluated at steps 0, 100, 200, 300, and 399. Alignment-% is the share of 30 trait-by-task regression coefficients (6 traits by 5 tasks) whose sign matches the direction expected from the literature, with bootstrap confidence intervals; about 50% is chance.¹ Trait spread is the variation in the model’s 1–5 trait self-reports. Geometry is measured on embeddings of the 60 BFI-2 items as bare text: tree distortion measures how far pairwise distances between items depart from the domain–facet–item hierarchy;⁴ Gromov δ-hyperbolicity measures how tree-like the representation is, using the four-point condition on sampled quadruples;⁷ and effective rank measures how many dimensions the embeddings use.

Analysis

At step 0 the adapter is untrained, so all three conditions evaluate the same model and agree exactly within a seed. Variation between seeds is therefore larger than variation between conditions, and every comparison is made within a seed, as the change from step 0 to step 399. I report the direction of each per-seed change and its size, not p-values. A change of exactly zero counts as neither positive nor negative. The 80-measure comparison in Section 2.2 is descriptive: with many related measures and three seeds, single entries carry little weight, and only the overall pattern is interpreted. Runs resumed from a checkpoint re-run the steps after that checkpoint; the duplicate records are removed before analysis, keeping the later one.

Reproducibility

All nine runs used identical configuration and identical training and evaluation code, verified by comparing each run’s recorded configuration and code version. The analysis scripts compare_arms.py and adapter_similarity.py are included in the project repository.

References

1. P. Han et al., “The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs,” arXiv:2509.03730, Sep. 2025.
2. I. Pres, L. Ruis, M. Ghebreselassie, B. Z. Li, and J. Andreas, “Self-CTRL: Self-Consistency Training with Reinforcement Learning,” arXiv:2606.18327, Jun. 2026.
3. C. J. Soto and O. P. John, “The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power,” Journal of Personality and Social Psychology, vol. 113, no. 1, pp. 117–143, 2017.
4. C. De Sa, A. Gu, C. Ré, and F. Sala, “Representation Tradeoffs for Hyperbolic Embeddings,” Apr. 2018. [Online]. Available: https://arxiv.org/abs/1804.03329v2.
5. K. Park, Y. J. Choe, Y. Jiang, and V. Veitch, “The Geometry of Categorical and Hierarchical Concepts in Large Language Models,” arXiv:2406.01506, 2025.
6. E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv:2106.09685, 2021.
7. M. Yang et al., “Hyperbolic Fine-Tuning for Large Language Models,” arXiv:2410.04010, Feb. 2026.
8. S. Hayou, N. Ghosh, and B. Yu, “LoRA+: Efficient Low Rank Adaptation of Large Models,” 2024.
9. J. Zhu et al., “Asymmetry in Low-Rank Adapters of Foundation Models,” 2024.
10. D. Groeneveld et al., “OLMo: Accelerating the Science of Language Models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand: Association for Computational Linguistics, 2024, pp. 15789–15809, doi: 10.18653/v1/2024.acl-long.841.
11. Anthropic, “Claude Opus 5 and Claude Opus 5.5,” large language models accessed through Claude Code, Anthropic, 2026. [Online]. Available: https://www.anthropic.com/claude

Acknowledgements

I thank my faculty mentor, R. Michael Alvarez, for mentorship and for providing access to computing resources. I thank Rafal Kocielnik for technical assistance and guidance throughout this project, and Danny Ebanks for help accessing computing resources. This work was supported by the Caltech WAVE Fellows Program. The computations in this paper were run on the FASRC Cannon cluster supported by the FAS Division of Science Research Computing Group at Harvard University.

I used Claude (Anthropic), through the Claude Code coding assistant, to help write and debug the training, cluster, and analysis code, to help analyze results, and to help draft and format this report. I directed the research, reviewed all code and analyses, and take responsibility for the content.

ASupplementary tables

Table A1: Training signal per run. “Carried gradient” is the share of steps that were neither skipped nor in a group with identical rewards.

Table A2: Mean trait spread and change in effective rank, step 0 to step 399.

Table A3: Alignment-% with 95% bootstrap confidence intervals at step 399. Interval widths of 21 to 31 points exceed every difference between conditions.

Table A4: Geometry: change from step 0 to step 399 in each condition minus the same change in LoRA, paired by seed. “Signs” counts seeds above and below zero.

Table A5: Cosine similarity between adapter weights. ΔW = BA is the full low-rank update; A and B are its two factors. At a fixed seed all conditions start from the same random A, which barely changes during training; this shared starting point, not agreement between methods, accounts for the higher same-seed similarity.