Orbbec Gemini 336L: outdoor stereo depth sensing for IP67-rated robotics and agricultural automation

A perception stack that performs well on a warehouse floor can fall apart within minutes of rolling into a strawberry tunnel or a gravel yard. Usually nothing is broken. The camera was specified for conditions that don’t exist outside. Integrators building agricultural robots, outdoor AMRs, and field service platforms need to read a depth camera datasheet with different questions in mind, and the Orbbec Gemini 336L outdoor stereo depth sensor makes a good worked example because its published specs draw a clear line around where it belongs in the field.

What changes when depth sensing moves outdoors

Indoors, the environment is very helpful. The light levels are stable, the surfaces are generally flat and dry and there’s not a lot of dust in the air. Once you take the robot out onto the street however, none of those three assumptions are valid. Each of those is reflected in a different section of the spec sheet.

Sunlight competes with the projector

Active stereo cameras project an infrared dot pattern onto the scene which the matching algorithm can then use as texture on blank walls, floors and boxes. The Gemini 330 series cameras project at 850nm. Sunlight has a lot of energy in that near-infrared band so in direct sun the projected pattern suffers from a loss of contrast. This starts off at longer distances and gets worse as the sun gets stronger.

Structured light cameras are completely reliant on their projected pattern and for that reason are generally only used in semi-outdoor situations. Stereo cameras have the advantage of a fallback. With two imagers, you can still get depth from the natural textures of the surface (soil, bark, gravel, leaves, etc.), and these textures get much more prominent in the sunlight. The Gemini 330 series utilises both active and passive stereo simultaneously. The projected pattern is the one that kicks in when the surface is low texture (in the shade, at dusk etc) and passive matching does the bulk of the work during the day. Always ask the vendor what their camera does when the projector isn’t doing its job. That’s the information that actually matters.

Glare is another lighting issue. Wet leaves, irrigation lines, greenhouse panels and even stainless steel tools can all cause reflections which will saturate the pixels and disrupt matching. The optical filter in front of the depth sensors determines which wavelengths get to the sensors and this has a direct impact on how the camera deals with reflective scenes.

Dust, water, and condensation

Field robots operate in pollen, chaff, soil dust, rain and irrigation spray and many of them get hosed down at the end of the shift. The IP ratings as defined in IEC 60529 are used as a shorthand for this. The first digit relates to solids and the second to water. A 6 in the first position means dust-tight. A 5 in the second position means protection against water jets and a 7 means the enclosure is capable of withstanding temporary immersion.

Humidity has its own place in your requirements. Most cameras are rated for non-condensing humidity. If a robot goes from a cool shed out into a warm, wet morning, condensation can form on the cold surfaces, and an IP rating won’t stop it on the optical window. How you mount, how you shield and how you warm up the camera is just as important as the enclosure rating.

Range comes down to geometry

Stereo depth error increases with the square of the distance and decreases as the baseline between the two imagers increases. Outdoor robots tend to be moving faster and over rougher ground than the indoor AMRs so they need reliable depth information at greater ranges to plan their stops and avoid obstacles. That’s why outdoor platforms typically use long baseline cameras. It’s also why accuracy at 2m and 4m is more important than the theoretical maximum range.

Gemini 336L specifications for field platforms

The Gemini 336L is the IR-filtered version of the Gemini 335L and has the same long baseline design. The Orbbec Gemini 336L product page and the Gemini 330 series spec sheet give the following figures.

Stereo baseline is 95mm, projector wavelength is 850nm. Depth range is from 0.17m to over 20m (theoretical max is 65m according to Orbbec). The optimal depth range is 0.25m to 6m. Spatial accuracy is stated as being ≤0.8% at 2m and ≤1.6% at 4m at 1280 x 800. In practice that’s about 16mm of error at 2m and about 64mm at 4m. That’s the range where the majority of the obstacle avoidance and row-following decisions are made.

Depth output is 1280 x 800 at 30fps over a 90-degree x 65-degree field of view. The RGB sensor can deliver up to 1280 x 800 at 60fps over 94 degrees x 68 degrees. In the long baseline models, both the depth and color sensors are global shutter, which prevents the skew that a rolling shutter can induce when a robot is traversing over uneven ground. The camera also has an IMU and supports hardware triggering.

The key feature of the 336L is its IR-pass filter. The 335L has a filter that passes both visible and near-infrared light, the 336L has a filter that blocks visible light before it reaches the depth sensors. Orbbec recommends their IR-enhanced models for scenes with glare from reflective surfaces, bright light with hard shadows, and strong outdoor lighting. In the agri context that translates to wet leaves, greenhouse coverings and shiny metal on the tools and harvesters.

Camera is 124 x 29 x 27.7mm and weighs 135g. Can be mounted via a 1/4-20 UNC thread on the base or two M4 holes on the back. Data and power via a single USB 3.0 Type-C connection. Average power draw is under 3W.

Robust design

The Gemini 336L has an IP65 rating (dust-tight and protected against water jets). Orbbec’s documentation includes a detail which is one that integrators can easily miss: the rating only applies if the camera is used with cables of an equivalent IP rating. The USB-C connection must be specified as part of the sealed unit, not just as a standard connection.

The rated operating range is −10°C to 50°C at 5% to 90% RH (non-condensing) and Orbbec list the operating environment as indoor and outdoor.

That range covers the vast majority of real-world field work, just not all of it. Orbbec’s range makes the limitations very clear. The Gemini 345Lg is their primary outdoor model. It’s IP67 rated, operates from -20°C to 65°C, has been vibration tested from 10 to 2000Hz at 6.4 grms and connects over GMSL2 via a FAKRA-C connector. The Gemini 335Le and 338Le are also IP67 rated and use Gigabit Ethernet with PoE. If your platform is going to be subject to washdown, parked in the summer sun or require long locking cable runs then one of those latter two models is probably the better choice.

Matching the camera to the platform

Agricultural robots

Greenhouse and high tunnel robots are the ideal fit for the 336L. The temps are generally within the rated range, the water exposure is predominantly spray rather than immersion and the glare from wet leaves and coverings is a daily occurrence that the IR-pass filter is very good at dealing with. Harvest assist platforms, crop scouting carts and spraying robots all operating in rows will spend the majority of their time within the 0.25m to 6m ideal range.

Open field machines are a different proposition. A sensor housing on a tractor-mounted tool in July can be hotter than the air, and these machines are routinely pressure washed and stored in unheated sheds over winter. That’s the environment you’re looking at so you need the 345Lg’s IP67 sealing and wider temperature range.

Outdoor AMRs

Campus delivery bots, yard tuggers and security patrol platforms all require depth sensing at a distance for obstacle detection and they need it to work in the changing light from dawn through to midday. The 336L’s 95mm baseline and its ≤1.6% precision at 4m provides planners with useful data at the distances where the braking decisions are actually being made. Before you go and buy one, though, check two things: firstly, that the camera will be within the −10°C to 50°C temperature range on your platform in your climate, and secondly, that the USB path from camera to computer is compatible with your chassis. Any platform that’s going to have a long cable run or require a locking connector will typically end up going with the GMSL2 or Ethernet version.

Field service and inspection platforms

Robots used for utility inspection, infrastructure surveying and site mapping typically have sensors on their masts or arms and will be moving in and out of the building during the course of a single day. The 336L’s 135g weight, standard tripod thread and very low power consumption make it ideal for such applications. Orbbec also list 3D reconstruction as a primary application of the camera with the IR-pass filter maintaining excellent depth quality regardless of the lighting conditions. That’s a useful feature if you’re doing a scan that starts inside and finishes outside.

Integration on the robot

Depth is calculated on the camera by Orbbec’s MX6800 depth engine ASIC, and the depth-to-color alignment is also done within the camera. The host computer receives a fully processed depth stream rather than a pair of raw images which saves CPU and GPU resources for navigation, segmentation and crop detection. On a battery-powered field robot, this extra compute budget translates directly into longer runtime.

The camera is based on the Orbbec SDK v2 (open source and actively maintained), has ROS and ROS2 wrappers and Python support on Windows, Ubuntu, macOS and Android. The Gemini 330 series have been validated with the NVIDIA Jetson Thor and work with the NVIDIA Isaac Perceptor so that’s a big help for any team already using that platform. Hardware timestamping and multi-device sync allow integrators to combine multiple cameras for greater coverage (a very common requirement on wide boom sprayers and large AMRs).

Frequently asked questions

Is the Gemini 336L rated IP67?

No. Orbbec lists the Gemini 336L as IP65 (dust-tight and protected against water jets if you’re using cables of the same rating). For IP67 in the Gemini range you’ve got the Gemini 345Lg which is Orbbec’s primary outdoor model, or the Ethernet versions of the Gemini 335Le and 338Le.

Should I choose the Gemini 336L or the Gemini 335L for an outdoor robot?

They’re the same in terms of baseline, range, precision and IP65 rating so the only real difference is the filter. The 336L only passes infrared light to the depth sensors, which helps with glare and in bright lighting. The 335L passes both visible and near infrared. If you’re operating your robot in an environment with wet leaves, reflective panels, or shiny metal, then go for the 336L. If you actually need full-spectrum stereo images rather than just IR, then take the 335L.

How far can the Gemini 336L see in practice?

Orbbec state a depth range of 0.17m to over 20m with a theoretical maximum of 65m, the best range being 0.25m to 6m. Plan your safety-related actions around that best range and the published accuracy (≤0.8% at 2m, ≤1.6% at 4m). Always test on your own target and in your own lighting before setting any speed limits.