Light and Weather in Microsoft Flight Simulator

Fans of flight simulators often talk about how “immersive” a sim is. It’s about how much the sim experience makes you feel like you’re flying a real airplane. Visuals are obviously a key component, and VR can take this to a higher level in terms of feeling truly surrounded by a 3D environment made up of the structures of your aircraft as well as all the natural and human-built objects in the simulated world around you. Sound is also very important, as are physical controls like yokes and rudder pedals (controlling an airplane with a keyboard is definitely an immersion killer). Realistic ATC (Air Traffic Control) communications are important especially for airliners and military flight sims.

Microsoft Flight Simulator 2020 addresses all these components, but to me, lighting and weather are the pieces that really pull it all together. Things we take for granted in real life like shadows, reflections, transparency, refraction, and color shifts are all well-simulated in MSFS. Making things look real also requires careful attention to materials and to how light interacts with them, including their colors and how shiny or rough they appear to be, which also affects color.

MSFS uses techniques called PBR (physically based rendering) to pull this off. Notice how the yellow features in this Stearman paint scheme (above) are reflected and distorted in the shiny blue fabric panels of the wings. The colors are shifted by these reflections as well as by shadows and by the angle from the sun at which you are viewing the surface. PBR techniques are also used to make buildings, runways, and other surface objects appear more realistic, often in combination with photogrammetry. 

Photogrammetry uses photographic imagery for measurement purposes, and in MSFS, this real-world data allows many cities to appear nearly photorealistic. Based on the resolution of the captured data, objects as small as signs on buildings may be readable, depending on your position and altitude. Technologies such as radar and lidar can also be used to accurately measure the height of terrain, and such data is often available to simulation modelers. Although MSFS recreates the entire surface of the Earth, the level of available detail varies by location, with many major cities covered by detailed photogrammetry (like New York City, shown in a VR stereo pair below), while many others depend on AI-based “autogeneration” of 3D objects including buildings, mountains, trees, etc. 

The appearance of objects in a simulation naturally depends on light sources, and MSFS incorporates multiple light sources starting with the sun and the moon (both accurately positioned based on the location, date, and time of day). I believe starlight is also modeled. Human-made light sources include streetlights, vehicle lights, light from building interiors, airport lighting, etc., and what amazes me is  when light sources interact with surface properties (reflecting or scattering) as well as with atmospheric properties like clouds, mist, rain, and snow. The resulting scenes can be uncannily realistic.

Speaking of rain and snow, weather is the other big piece of “immersiveness” I wanted to briefly discuss here. It’s a critical subject in real life aviation and in the sim. Microsoft and Asobo and their partners have placed huge emphasis on making weather look and feel right, and the system they developed is incredibly detailed, flexible, and easy to control.

Not only can you adjust any location's time and weather to your choosing - from clear skies to heavy clouds, to snow and storms – with “Live Weather,” you can even set it to match real conditions taking place in that part of the world. But if you don’t like the time of day, the wind, the height of the clouds, the height of the waves, or the depth of the snow cover, you can change any of that without even stopping your flight, using the interface shown above. 

Rather than drone on and on about this, allow me to recommend two brief developer videos that clearly explain how they managed to make the weather so real, it can get scary at times. Fortunately I can switch to clear skies anytime and (usually) not hit the mountain that was behind that cloud!

Feature Discovery Series, Episode 2: Weather 

Partnership Series: Meteoblue - Weather Forecast System