Last time, we met Portia, a jumping spider that plans hour-long hunting routes, learns by trial and error, and manipulates mental images with fewer than 100,000 neurons. We ended on a provocation: one of Portia’s capabilities, its depth perception system, has been studied at the optical level, computationally modeled, and explicitly proposed as a sensor template. And nine years later, nobody has built it.

This part is about what that sensor would look like, why it should exist, and what’s standing in the way. Part three will answer the harder question, which is also the simpler one: why hasn’t this happened?

Let me start with a confession. I am not an optical engineer. I am not a neuromorphic computing researcher. I am an engineering leader who became fascinated by a 2012 paper in Science showing that a jumping spider estimates distance by the amount of blur in its vision. Not stereo vision. Not LIDAR. Blur. That led me to a 2017 paper in Biomimetics where researchers built a computer model of the spider’s eye, validated that blur-to-distance math works, and proposed building a sensor based on it. And then I spent a week trying to find out if anyone had actually built the thing.

Nobody has. And after reading everything I could find, I think I understand why. But first, let me show you what I found.

The Optical Trick#

Hold a finger close to your eye. It is sharp. Now hold it at arm’s length and bring it slowly toward your nose. At some point it goes blurry. Your eye has a lens, and the lens can only focus light from one distance at a time. Everything else falls on your retina at varying degrees of blur.

That blur is information. It tells your brain how far away things are, relative to what your eye is focused on. Humans use this cue weakly, because our two eyes give us stereo vision and that is stronger. But a jumping spider has no stereo vision. Its two principal eyes work independently, each one a long narrow tube with a single thick uncorrected lens at the front.

An uncorrected lens has a property that lens designers normally treat as a flaw: chromatic aberration. Different colors of light bend by different amounts when passing through the lens. Red light focuses farther away. Blue light focuses closer. A corrected lens eliminates this. The spider’s lens does not.

Now add the retina. Behind the lens, at the back of the eye tube, the photoreceptors are arranged in four vertically stacked layers. Only the two deepest layers, called L1 and L2, are used for depth estimation. They sit about 10 to 20 micrometers apart.

Here is what happens. Light enters the lens, splits into its component colors because of chromatic aberration, and comes to focus at different depths depending on wavelength and distance. The image on L1 will have a different amount of blur than the image on L2, because they sit at different focal depths. The ratio of blur between the two layers encodes the distance to the object.

This is not speculation. Nagata et al. (2012, Science) proved it behaviorally. They occluded all but one of the spider’s anterior median eyes and had it jump at prey under different lighting conditions. Under green light, the spiders jumped accurately. Under red light, they consistently jumped too short. The spider was using the chromatic blur differential as its depth cue, and when the color balance shifted, the cue broke.

Cross-section diagram of a jumping spider's principal eye: white light enters through a thick curved lens, splits into rainbow color bands inside the eye, and converges at two distinct focal planes labeled L1 and L2, with a magnified callout showing the blur differential between the two layers
A single uncorrected lens, chromatic aberration as a feature, and two photoreceptor layers at slightly different focal depths. No stereo vision. No LIDAR. Just blur.

The Model That Proved It Works#

In 2017, a team led by researchers at the European Space Agency’s Advanced Concepts Team and Humboldt University built a physically accurate 3D computer model of the jumping spider’s principal eye using Blender and LuxRender ray tracing. They modeled the lens parameters (radii of curvature, thickness, refractive index), the posterior chamber fluid, and the layered retina from published anatomical measurements.

They generated a dataset of images as they would appear on L1 and L2 at different distances, then applied a well-known computer vision algorithm, Subbarao’s depth-from-defocus algorithm, to the simulated retinal images. The algorithm recovered distances with a median error of 9.7 percent. Good enough for a spider to land a precision jump. Good enough for many engineering applications.

What made this study remarkable was not the algorithm. DFD is a mature computer vision technique. What made it remarkable was that the algorithm assumed an ideal thin lens. The spider uses a thick uncorrected lens with heavy spherical and chromatic aberration. The algorithm should not have worked well. But it did, because the positions of L1 and L2 relative to the lens happen to cancel out the worst of the aberrations.

The authors did not stop at modeling the spider. They explicitly proposed building a sensor based on the same principle.

“A DFD sensor mimicking the spider eye setup by using only a single uncorrected lens and two photo sensors would be cheaper and more robust than the expensive and fragile corrected lens systems commonly used for DFD.”

They noted that such a sensor would require no moving parts, consume minimal power, and be suitable for UAVs, MAVs, and CubeSats. They published this in 2017.

That was nine years ago.

The Precedent That Proves the Method#

You might be thinking: this sounds like one of those bio-inspired ideas that looks good on paper but never works in hardware. Normally I would share that skepticism. There is a long graveyard of “we looked at a bat and made a chip” proposals.

But in this case, someone has already proven the methodology works with a different invertebrate.

In 2024, Schreiber et al. published “Emulating insect brains for neuromorphic navigation” in Neuromorphic Computing and Engineering. They emulated the honeybee path integration neural circuit on the BrainScaleS-2 mixed-signal neuromorphic processor. The entire system, environment, sensory organs, brain, actuators, and virtual body, ran autonomously on a single neuromorphic core at 1000 times biological speed. Over 4800 consecutive bee journeys across 320 generations completed in 30 minutes.

The bee brain has about 1 million neurons. Portia has less than 100,000. The optical component of Portia’s depth perception is simpler than the bee’s path integration circuitry. The methodology is proven. The question is not whether insect neural circuits can run on neuromorphic hardware. They can. The question is why nobody has tried with a spider.

Portia would not even be the first non-honeybee invertebrate on neuromorphic silicon. There are at least seven documented insect neural circuit families that have been translated into hardware: honeybee path integration, locust collision avoidance, fruit fly olfactory processing, dragonfly visual pursuit, locust ear-on-a-chip, fly photoreceptor models, and cricket auditory pattern recognition. Every single one is an insect. None are arachnids. The gap is not methodological. It is taxonomic.

What a Built Thing Would Look Like#

Picture a sensor the size of a grain of rice. A single molded plastic lens, no moving parts, bonded directly to a stacked CMOS die with two photodetector layers at slightly different depths. The die includes a small inference engine, maybe 10,000 parameters, running a compact neural network trained on synthetic blur data from the Müller model. Total system power, lens to depth map: low milliwatts at most, potentially microwatts for a sparse output. Cost in volume: cents per die.

What would that enable?

Collision avoidance on micro-UAVs small enough to fit in your palm, navigating without GPS or LIDAR. Always-on depth sensing for AR and VR headsets, running on a power budget that current depth cameras cannot touch. Insect-scale robots that navigate environments too small or dangerous for larger machines. Redundant depth sensors for autonomous vehicles as a fallback to LIDAR, costing nothing in system integration terms. Cheap depth cameras for accessibility applications, helping people with visual impairments navigate indoor spaces.

Every physical primitive for this sensor exists in a fab today. Sony and IMEC have qualified stacked CMOS photodetector processes. SynSense has the Speck chip, integrating an event-based vision sensor and SNN processor on a single die at milliwatt power. Intel’s Loihi 2 demonstrated the feasibility of low-power neuromorphic inference. The molded plastic lens is the cheapest component in the system. The neural decoder can be trained entirely on synthetic data from the existing optical model, no connectome required.

The sensor does not need to be perfect to be useful. The Müller model achieved 9.7 percent median error. That is too noisy for autonomous driving at highway speeds. It is entirely sufficient for collision avoidance on a drone moving at walking pace. The first generation does not need to outperform LIDAR. It just needs to be cheap enough and low-power enough to go where LIDAR cannot.

What Is Still Missing#

Here is the honest version. The optical model is solved. The manufacturing primitives exist. The neuromorphic precedent is proven. What is missing is the neural decoder. We know the input to Portia’s depth perception system (blur patterns on L1 and L2) and we know the output (accurate jumping distance). We do not know the circuit in between.

That could matter. Or it could not. A compact convolutional neural network trained on synthetic ray-traced data could learn the blur-to-depth mapping without understanding how Portia’s neurons do it. The resulting sensor would be a functional approximation, not a biological simulation. First-generation products do not need to match the spider’s efficiency. They just need to be good enough to ship.

The actual bottleneck is not technical.

The Real Obstacle#

Here is what I keep coming back to. The Müller paper proposed this sensor in 2017 with enough detail that you could start building it from the published figures. The optical principle was confirmed behaviorally in 2012. The CMOS processes needed to build it were already in development. The honeybee neuromorphic proof of concept was published in 2024. Seven insect neural circuit precedents exist.

Every piece is in place. Nothing has moved.

The reason is not physics. It is not engineering readiness. It is not funding. The reason is that the people who understand the biology, the people who understand the hardware, and the people who understand the optics do not work together. They attend different conferences. They publish in different journals. They answer to different program managers. The spider vision papers are in biology venues that hardware researchers never encounter. The neuromorphic papers are in engineering venues that biologists do not read. The optics papers are in yet a third literature, cited by neither group.

A 2017 paper in Biomimetics proposed building a copper-and-silicon version of a spider’s eye, and as far as I can tell, the neuromorphic hardware community has never seen it.

That is the real story. And it is the subject of part three.

Next in this series: The Tragedy of the Uncrossed Campus, exploring the coordination failure that keeps promising research from becoming hardware and the pattern it reveals about how science organizes itself.

References#

  • Nagata, T. et al. (2012). Depth Perception from Image Defocus in a Jumping Spider. Science, 335, 469-471. https://doi.org/10.1126/science.1211667
  • Müller, A.N. et al. (2017). Depth Estimation from a Single Camera Image Using a Spider-Eye-Inspired Depth-from-Defocus Sensor. Biomimetics, 2(1), 3. https://doi.org/10.3390/biomimetics2010003
  • Schreiber, K. et al. (2024). Emulating insect brains for neuromorphic navigation. Neuromorphic Computing and Engineering. https://arxiv.org/abs/2401.00473
  • Land, M.F. (1969). Structure of the retinae of the principal eyes of jumping spiders. Journal of Experimental Biology, 51, 443-470.
  • Dalgaty, T. & Vianello, E. Insect-inspired neuromorphic computing. Semantic Scholar.