The Tragedy of the Uncrossed Campus

I need to start with something that has been bothering me since I started researching this story.

The Müller paper that proposed the spider-inspired depth sensor was published in 2017 in a journal called Biomimetics. It is a good paper. The authors built a physically accurate optical model of a jumping spider’s principal eye, validated the depth-from-defocus algorithm against it, and explicitly stated that the resulting sensor would be cheaper and more robust than existing approaches. They suggested specific applications: UAVs, MAVs, CubeSats, any system where size, weight, power, and cost matter more than millimeter precision.

The Schreiber paper that proved insect neural circuits can run on neuromorphic hardware was published in 2024 in Neuromorphic Computing and Engineering. It demonstrates a complete closed-loop system, honeybee path integration running on a single BrainScaleS-2 core at 1000 times biological speed. It proves the methodology works.

The Nagata paper that first demonstrated the spider’s depth-from-defocus mechanism was published in 2012 in Science. It is one of the most elegant behavioral experiments I have ever read.

None of these papers bridge each other’s communities. Müller (2017) published in Biomimetics, which sits in the optics and computer vision literature. It cites Nagata (2012) because they share the same subject — spider vision. But Schreiber (2024) was published in Neuromorphic Computing and Engineering and does not cite Müller or Nagata. The spider vision researchers publish in Journal of Experimental Biology and Animal Cognition. The neuromorphic hardware people publish in IEEE Transactions on Biomedical Circuits and Systems and attend ISSCC. The optics and computer vision community publishes in Optics Express and CVPR and attends SPIE.

These groups do not overlap. They do not go to the same conferences. They do not submit to the same funding programs. And as a result, a sensor that multiple lines of evidence suggest is feasible has sat on a table for nine years while each community continued working on its own problems, unaware that the missing piece for their own work was sitting in a journal the next building over.

The Three Tribes#

Let me be specific about who we are talking about.

The first group is arthropod ethologists. These are the biologists who study animal behavior, particularly the cognitive capabilities of spiders, insects, and other invertebrates. They are the ones who ran the Portia detour planning experiments, the mental rotation studies, the self-recognition tests. They publish in Animal Cognition, Journal of Experimental Biology, Frontiers in Psychology. They present at ARVO (the Association for Research in Vision and Ophthalmology), SICB (the Society for Integrative and Comparative Biology), and ISBE (the International Society for Behavioral Ecology). Their funding comes from the NSF BIO directorate and similar biology-focused agencies.

They know what Portia can do. They have been studying it for decades. But they are not hardware people. The question of whether a spider’s optical principles could be turned into a manufactured sensor is not their question. It never occurs to most of them to ask it.

The second group is neuromorphic engineers. These are the people who design and build chips that mimic neural computation. They work at Intel’s Loihi group, at SynSense, at IBM Research, at IMEC. They publish in IEEE Transactions on Biomedical Circuits and Systems, Neuromorphic Computing and Engineering, Nature Electronics. They attend ISSCC (the International Solid-State Circuits Conference), IEDM, the Telluride Neuromorphic Workshop. Their funding comes from NSF CISE and ENG directorates and from DARPA’s Microsystems Technology Office.

They know how to build energy-efficient neural inference hardware. They demonstrated the honeybee path integration proof of concept. They have chips that can run spiking neural networks at milliwatt power. But they do not read biology journals. The question of whether a spider’s depth perception algorithm could be mapped onto their hardware never occurs to them.

The third group is computer vision and optics researchers. These are the people who build depth sensors: stereo cameras, LIDAR systems, structured light projectors, time-of-flight sensors. They publish in CVPR, ICCV, Optics Express, and yes, Biomimetics (where the Müller paper lives). They attend CVPR, ICCV, SPIE Photonics. They work in industrial R&D labs at Sony, ams-OSRAM, Lumentum, and in academic optics departments. Their funding comes from DARPA, from industrial research budgets, and from NSF ENG.

They publish in Biomimetics, which means the Müller paper was nominally in their literature. But depth-from-defocus is a niche within a niche in computer vision, and a single-lens uncorrected DFD sensor is not something the community has actively pursued. The paper has been cited, but not by anyone who builds neuromorphic hardware.

Here is the concrete example that made this real for me.

An IMEC researcher who publishes on low-power vision sensors and stacked CMOS imagers attends ISSCC and IEDM, the chip design conferences. The Australian National University group that builds optical models of arthropod visual systems and publishes on spider depth perception attends ARVO and SICB, the biology and vision science conferences. These groups publish in different journals, present at different conferences, and apply to different funding programs.

I searched for any citation link between the Müller 2017 Biomimetics paper and the neuromorphic hardware literature. There is none. I searched for any paper connecting jumping spider visual processing to neuromorphic implementation. There is none. The closest bridge is the Schreiber 2024 honeybee work, which is one taxonomic class away and does not reference the spider vision literature.

This is not a claim about what could happen. It is a documented fact about what has not happened.

The Evidence#

This is not a vague claim about disciplinary silos. It is empirically verifiable.

The Müller et al. 2017 paper was published in Biomimetics. Its citation pattern, as of my search, shows no uptake in the neuromorphic hardware literature. The Schreiber et al. 2024 paper was published in Neuromorphic Computing and Engineering. It does not cite the Müller paper or the Nagata spider vision work. The Nagata et al. 2012 paper in Science does not discuss hardware implementations.

A systematic search across arXiv, Google Scholar, and Semantic Scholar reveals exactly zero published works that directly connect jumping spider cognition with neuromorphic computing or bio-inspired hardware design. Zero. The closest parallel is the Schreiber honeybee work, which involves a different invertebrate with a 10x larger neural budget and does not bridge the gap to salticids.

This gap is not a search failure. I ran independent searches across Google Scholar, Semantic Scholar, and arXiv across multiple queries and two levels of recursive investigation. The result was clear: the seam between these fields is genuinely unexplored.

The Tragedy#

The tragedy is not that the research is hard. The tragedy is that it is not.

A single-lens chromatic depth-from-defocus sensor could be built from components that already exist in commercial fabs. A molded plastic lens. Two photodetector arrays at slightly different focal depths, implemented in a stacked CMOS process that Sony and IMEC have already qualified. A small inference engine, maybe a few thousand parameters, trained on synthetic blur data and running at sub-milliwatt power on an existing neuromorphic or conventional microcontroller.

The optical physics is understood (Nagata 2012). The computational model is validated (Müller 2017). The methodology for running invertebrate neural circuits on neuromorphic hardware is proven (Schreiber 2024). Seven different insect neural circuit families have already been implemented in silicon: honeybee path integration, locust collision avoidance, fruit fly olfactory processing, dragonfly visual pursuit, locust tympanic ear, fly photoreceptor models, cricket phonotaxis.

Every physical primitive exists. Every proof of concept has been demonstrated. What does not exist is a single funded project that puts an optics researcher, a neuromorphic hardware engineer, and a spider biologist in the same room.

Nine years after the sensor was proposed, it has not been built. Not because it cannot be built. Because the institutional architecture of science is optimized for depth within disciplines, not for synthesis between them.

What Would It Take#

A program structure with parallel tracks and convergence gates. Not the sequential academic pipeline where biologists study the spider for five years, publish a paper, then neuromorphic engineers build a chip for five years, publish a paper, then optics researchers design a sensor for five years, publish a paper. That pipeline reliably kills bio-inspired projects at the handoff point between stages. It is how promising ideas go to die.

Parallel tracks: the optics team builds the sensor frontend while the neuromorphic team builds the blur decoder while the biologists continue mapping the spider’s neural circuits. Each track proceeds independently. At regular convergence gates, the teams compare results and adjust.

A single person who reads across the three literatures. This is the scarcest resource in the entire system. Someone who can attend ISSCC and ARVO and CVPR and recognize that the paper presented at one conference solves the problem described at another. These people exist. They are just not funded to do this work.

The Pattern#

I have been thinking about what this story means beyond spiders.

The coordination failure between these three communities is not rare. It is the default state of organized knowledge production. The system incentivizes depth. You advance in your career by knowing more about less. The journals, the conferences, the funding programs, the tenure criteria all reward specialization. This is not a bug. It is a feature. It produces rigorous, well-validated work within each discipline.

But it also produces gaps. Seams between fields where the missing connection is not a discovery but a conversation. Where the research has been done, the primitives exist, the proof of concept is demonstrated, and nobody has put them together because nobody was looking at both sides.

The gap between what we know about jumping spider vision and what we could build from it is one such seam. It is not the only one.

There are sensors that could exist. Drugs that could be developed. Software architectures that could be deployed. Solutions to problems that are currently being worked on independently by people who do not know that the answer to their question appeared last month in a journal they do not read.

The sensor that could cost pennies and consume microwatts. The spider in your garden that already knows how to build it. The paper that shows you how. The chip that could run it. All of these exist. The connection between them does not.

Sometimes great collaborations are just a short walk across campus. In this case, the people who understand the spider’s eyes are in Christchurch, New Zealand. The people who built the first insect-scale neuromorphic proof of concept are in Heidelberg, Germany. The people who modeled the spider’s optics computationally are in the Netherlands and Berlin. The people who make the event-based vision sensors that come closest to Portia’s efficiency are in Zurich and Paris. The first neural recordings from a jumping spider brain were done at Cornell.

This is not a matter of different buildings on the same campus. It is different buildings on different continents. The coordination failure is not geographical laziness. It is structural: no shared conferences, no shared journals, no shared funding programs, no institution that employs a spider vision researcher and a neuromorphic chip designer under the same roof. The distance between these fields is not measured in meters. It is measured in the absence of any institutional mechanism that would cause these people to encounter each other’s work.

That is the tragedy. Not that the sensor hasn’t been built. That it hasn’t been built, and it should have been years ago, and the only thing standing in its way is the way we organize knowledge.

References#

• Müller, A.N. et al. (2017). Depth Estimation from a Single Camera Image Using a Spider-Eye-Inspired Depth-from-Defocus Sensor. Biomimetics, 2(1), 3. https://doi.org/10.3390/biomimetics2010003
• Schreiber, K. et al. (2024). Emulating insect brains for neuromorphic navigation. Neuromorphic Computing and Engineering. https://arxiv.org/abs/2401.00473
• Nagata, T. et al. (2012). Depth Perception from Image Defocus in a Jumping Spider. Science, 335, 469-471. https://doi.org/10.1126/science.1211667
• Land, M.F. (1969). Structure of the retinae of the principal eyes of jumping spiders. Journal of Experimental Biology, 51, 443-470.
• Cross, F.R. & Jackson, R.R. (2014). Specialised use of working memory by Portia africana. Animal Cognition, 17(2), 435-444. https://doi.org/10.1007/s10071-013-0676-2
• Tarsitano, M.S. & Jackson, R.R. (1997). Detour route discrimination by Portia. Animal Behaviour. https://doi.org/10.1006/anbe.1997.0505