We demonstrate that even under these minimal conditions, model performance degrades as input length increases, often in surprising and non-uniform ways. Chroma is HiringIt is common for modern LLMs to have input context lengths in the millions of tokens. In this case, increasing input length means increasing the size of the graph to traverse through, which increases task difficulty as a result. It is difficult to disambiguate increasing task complexity from input length, which makes it difficult to isolate the impact on performance due to input length alone. We find that as needle-question similarity decreases, model performance degrades more significantly with increasing input length.