Introduction
Rendering the ocean in video games has occupied the mind-space of programmers, researchers and dreamers alike for decades. Making realistic waves on water surfaces in real-time is now commonplace in today’s AAA games. Even lower-end games played on mobile devices demonstrate realistic looking water surfaces. So, the problem of rendering waves is more or less solved… on a local scale. But what isn’t commonplace is seeing an animated ocean surface on a planetary scale. In this blog post I talk about where I am at on this topic with Bending Time as well as outline the ocean features that will be investigated in the future.
Ocean Height
As soon as we step back to rendering the whole planet, the ocean rendering problem space changes. It’s no longer just a matter of generating the wave geometry, shading it and adding other effects like reflections, etc. One of the main reasons is the large-scale coordinates you need to model things planet-wide, which I discussed in my Earth Coordinate Systems blog post. Further, spherical-based coordinates make it more difficult to frame the problem space of generating the wave geometry. If one models the planet in the ECEF frame, one quickly sees the math is non-trivial to simulate the ocean surface. Computing the surface accurately, with high fidelity, involves the forward conversion of geodetic coordinates to ECEF for every vertex of the ocean surface mesh. On top of this, implementing, e.g., the simplest Gerstner waves as described in the classic Effective Water Simulation from Physical Models written by Mark Finch and published in NVIDIA’s original GPU Gems book, can be challenging primarily because Mark, like most if not all water simulation developers, treat the ocean as having Z-heights above an XY plane in a 3D Cartesian coordinate system.
Rendering the ocean in video games has occupied the mind-space of programmers, researchers and dreamers alike for decades. Making realistic waves on water surfaces in real-time is now commonplace in today’s AAA games. Even lower-end games played on mobile devices demonstrate realistic looking water surfaces. So, the problem of rendering waves is more or less solved… on a local scale. But what isn’t commonplace is seeing an animated ocean surface on a planetary scale. In this blog post I talk about where I am at on this topic with Bending Time as well as outline the ocean features that will be investigated in the future.
Ocean Height
As soon as we step back to rendering the whole planet, the ocean rendering problem space changes. It’s no longer just a matter of generating the wave geometry, shading it and adding other effects like reflections, etc. One of the main reasons is the large-scale coordinates you need to model things planet-wide, which I discussed in my Earth Coordinate Systems blog post. Further, spherical-based coordinates make it more difficult to frame the problem space of generating the wave geometry. If one models the planet in the ECEF frame, one quickly sees the math is non-trivial to simulate the ocean surface. Computing the surface accurately, with high fidelity, involves the forward conversion of geodetic coordinates to ECEF for every vertex of the ocean surface mesh. On top of this, implementing, e.g., the simplest Gerstner waves as described in the classic Effective Water Simulation from Physical Models written by Mark Finch and published in NVIDIA’s original GPU Gems book, can be challenging primarily because Mark, like most if not all water simulation developers, treat the ocean as having Z-heights above an XY plane in a 3D Cartesian coordinate system.
Gerstner waves in Mark Finch’s “Effective Water Simulation from Physical Models” in GPU Gems
The immediate thought of most game developers will be to “just hack it;” do the wave simulation in a separate 3D Cartesian coordinate frame and place/orient it on the Earth’s surface so it looks appropriate. This perennial, favourite, game developer strategy is effective to solve this problem but it’s not a seamless solution. For example, it would be difficult to do this for the whole planet all at once. One might argue that you don’t need individual waves at the whole-planet scale, which is true but there are other considerations. To start, the question of whether to render the ocean as a “mega mesh” or to use tiles like is done in the mapping world comes to mind.
For non-geospatially-oriented developers, using a mega mesh with “sea level at height zero” might be their first tack. This works okay for viewing the entire planet at once but, when you are close to the surface, one sees that height zero, which typically is an ellipsoid height of zero, doesn’t always align with the land. In fact, it rarely does. This is why 3D globes don’t try and render the land-ocean interface because the coordinates don’t line up. In addition to the land-ocean interface issue, the ocean isn’t a perfect circle.
The ocean’s surface is affected by several factors but the biggest is Earth’s variable gravitational pull across the planet. These variations cause the ocean surface to be lower or higher depending on whether there is greater or less gravitational force respectively. Scientists refer to the shape of the ocean surface under the influence of the gravity of Earth, not accounting for other factors such as wind and tides, as the geoid.
For non-geospatially-oriented developers, using a mega mesh with “sea level at height zero” might be their first tack. This works okay for viewing the entire planet at once but, when you are close to the surface, one sees that height zero, which typically is an ellipsoid height of zero, doesn’t always align with the land. In fact, it rarely does. This is why 3D globes don’t try and render the land-ocean interface because the coordinates don’t line up. In addition to the land-ocean interface issue, the ocean isn’t a perfect circle.
The ocean’s surface is affected by several factors but the biggest is Earth’s variable gravitational pull across the planet. These variations cause the ocean surface to be lower or higher depending on whether there is greater or less gravitational force respectively. Scientists refer to the shape of the ocean surface under the influence of the gravity of Earth, not accounting for other factors such as wind and tides, as the geoid.
Earth Gravitational Model from Wikipedia
There are many models of the Earth’s geoid used for a variety of purposes. When defining the global height of the ocean surface, the Earth Gravitational Model of 1996 (EGM96) still serves as a lightweight model (lightweight in terms of computer memory space) used in modern devices like handheld GPS receivers. (The height displayed by a handheld GPS device is typically above mean sea level (AMSL), which is a height relative to the geoid, but this is tangent to the discussion here.) Beyond EGM96, there is EGM2008 and soon to be released is EGM2020, which provide enhanced accuracy and precision to the model.
In the cases of EGM96 and EGM2008, NASA has processed the models into tiles. EGM96 is available in 15’ x 15’ (1/4 degree) tiles and EGM2008 is available in 2.5’ x 2.5’ tiles from https://earth-info.nga.mil/index.php?dir=wgs84&action=wgs84.
In the cases of EGM96 and EGM2008, NASA has processed the models into tiles. EGM96 is available in 15’ x 15’ (1/4 degree) tiles and EGM2008 is available in 2.5’ x 2.5’ tiles from https://earth-info.nga.mil/index.php?dir=wgs84&action=wgs84.
EGM2008 raster tiles from NGA
As can be seen, the format of the tiled EGM model looks like a heightmap. The heights are encoded as floating point values above or below the reference ellipsoid, WGS84 in this case, making it a type of Digital Elevation Model (DEM). From here we can deduce the height of the ocean surface is the ellipsoid height of zero plus or minus the geoid height as depicted in the following figure.
Relationship between orthometric height, ellipsoid height, and geoid height
This global model of ocean surface height is still an approximation. If we were standing on a virtual beach looking at the waves come in, this representation of the height of the surface would still not be accurate enough to make for a nice-looking scene. A few meters up or down could result in the virtual area being flooded or our virtual avatar looking at a fairly withdrawn beachfront. In order to achieve beach-level quality for the ocean surface height, we look to nautical charts.
Nautical (or marine) charts define the height of the ocean’s surface at a much more accurate scale as you can imagine is needed for the safe navigation of vessels traveling the ocean. Nautical charts list various heights for the area the chart covers relative to a chart datum. Chart (or tidal) datums are reference points for geographical regions to define the height of the ocean’s surface at a particular date and time.
Looking at an example chart, 3538 of the Canadian Hydrographic Service (CHS) BSB Navigational Charts, which covers the Desolation Sound area, it looks like a typical nautical chart where the land is depicted in a tan colour and the deeper areas are coloured white while the shallower areas are coloured deeper shades of blue the shallower the water gets.
Nautical (or marine) charts define the height of the ocean’s surface at a much more accurate scale as you can imagine is needed for the safe navigation of vessels traveling the ocean. Nautical charts list various heights for the area the chart covers relative to a chart datum. Chart (or tidal) datums are reference points for geographical regions to define the height of the ocean’s surface at a particular date and time.
Looking at an example chart, 3538 of the Canadian Hydrographic Service (CHS) BSB Navigational Charts, which covers the Desolation Sound area, it looks like a typical nautical chart where the land is depicted in a tan colour and the deeper areas are coloured white while the shallower areas are coloured deeper shades of blue the shallower the water gets.
CHS RNC Chart 3538 – Desolation Sound
In addition to the pictorial view of land and water depth, nautical charts always list their datum information.
CHS RNC Chart 3538 – Chart Datum Information
As we can see from the chart datum information, height references like HHW ([Mean] Higher High Water) and LLW ([Mean] Lower Low Water) to name a couple are shown. These height references are important to determine exactly where the ocean surface is located vertically.
A chart datum height reference of Mean Lower Low Water is often used to provide a conservative view for mariners so they can rely on the water level not going below this reference point.
For the purpose of rendering the ocean surface without simulating the tides, we can use Mean High Water. This will provide a view of the land-ocean interface where the underwater features, e.g., seaweed, crabs, etc. are generally not exposed. I consider simulating low tide, with all the exposed features that were underwater hours before, a future consideration for the lifelike virtual world.
Let’s use an example to calculate the ECEF coordinates of the ocean surface in meters relative to the WGS84 ellipsoid. Using the same chart as above, let’s zoom into a particular region, Hernando Island.
For the purpose of rendering the ocean surface without simulating the tides, we can use Mean High Water. This will provide a view of the land-ocean interface where the underwater features, e.g., seaweed, crabs, etc. are generally not exposed. I consider simulating low tide, with all the exposed features that were underwater hours before, a future consideration for the lifelike virtual world.
Let’s use an example to calculate the ECEF coordinates of the ocean surface in meters relative to the WGS84 ellipsoid. Using the same chart as above, let’s zoom into a particular region, Hernando Island.
Stag Bay RNC Chart Example
The numbers shown in the image above are “soundings,” measurements hydrographers take to determine the depth at a particular location. As we saw in the chart datum information, the soundings are in meters.
Let’s use the sounding to the immediate right of the STAG BAY label, ‘40’, and compute the coordinates of the ocean surface at that point.
The 40-meter sounding is the height at the Lowest Normal Tide. To convert to Mean High Water as I suggested to do, we look back to the chart. We can see the closest tidal station to Stag Bay is Lund. The tidal information on the chart only has Mean Higher High Water so we’ll use that value from the corresponding column in the Lund row, which is 4.8 meters. Now we have a Mean Higher High Water depth of 40 + 4.8 = 44.8 meters. This is referenced to the Lund tidal station. But how do we know where this station sits relative to the WGS84 ellipsoid? For that, we need to get information about the tide station from an external resource. These are typically managed by the government and provided via a website. Below we see the information about the Lund tide station.
Let’s use the sounding to the immediate right of the STAG BAY label, ‘40’, and compute the coordinates of the ocean surface at that point.
The 40-meter sounding is the height at the Lowest Normal Tide. To convert to Mean High Water as I suggested to do, we look back to the chart. We can see the closest tidal station to Stag Bay is Lund. The tidal information on the chart only has Mean Higher High Water so we’ll use that value from the corresponding column in the Lund row, which is 4.8 meters. Now we have a Mean Higher High Water depth of 40 + 4.8 = 44.8 meters. This is referenced to the Lund tidal station. But how do we know where this station sits relative to the WGS84 ellipsoid? For that, we need to get information about the tide station from an external resource. These are typically managed by the government and provided via a website. Below we see the information about the Lund tide station.
We see there are various vertical data codes listed. These different data references vary in purpose and accuracy. For our purpose, we will use the NAD83_CSRS datum, which shows an elevation of -8.016 meters. To simplify the process in this blog post, we will use this value directly as the height relative to WGS84, however, this isn’t quite correct as the Earth’s tectonic plates are constantly moving. For the super keen, have a look at https://natural-resources.canada.ca/maps-tools-and-publications/geodetic-reference-systems/canadian-spatial-reference-system-csrs/9052 to get a better understanding of the details of the NAD83_CSRS datum. Adding the elevation of the Lund tide station, we have a height of 44.8 + (-8.016) = 36.78 meters. Now that we have the elevation, we need the horizontal geographic coordinates.
Again, looking at the chart information, the chart uses the Mercator Projection based on NAD83. We must project the Mercator coordinate back to geographic.
Note if the chart was referenced to a different ellipsoid other than WGS84, we would need to perform a datum transformation (or datum shift) as well.
Using Global Mapper, a GIS software application that can read RNC charts, the location of the 40-meter sounding has Mercator coordinates of X= -13905279.486 and Y = 6413303.196. I talked about map projections in my Earth Coordinate Systems blog post but I will just skip straight to the answer here. The geographic coordinates for that point are 49° 59' 55.3849" N, 124° 54' 47.7033" W.
So now we have a geodetic point for that location:
Lat: 49.99871802
Lon: -124.91325092
Elevation: 36.78 meters
Converting from geodetic to ECEF using the formula from my previous blog post, we get:
X: -2351.153 km
Y: -3368.638 km
Z: 4862.726 km
We can now position that point in our WGS84 ECEF world coordinate system. Next we need to scale the solution.
If we step back and look at the chart as a whole, we can see that it could be converted to a heightmap of sorts. We could use a best-fitting, 3D surface model and create a heightmap from each chart file. This is exactly what Bending Time intends to do. If the charts end up being too large on their own, they will be tiled for more efficient download and runtime processing.
Circling back, it’s clear that we need nautical charts when we are close to the planet’s surface. However, these very specific ocean heights will likely not make a difference to an observer flying in a virtual plane at 37,000 ft for example. This is where the geoid comes in.
We use the geoid heights for planet-scale ocean heights and nautical charts for when we are close to the planet’s surface. The geoid also serves as a way for the simulation to calculate AMSL heights, which could be used in GPS or aircraft simulation.
Wave Geometry
Okay, now that we can model the correct height of the ocean, we need to add waves when we are near the surface.
One of the main reasons Bending Time chose to use the Local ENU coordinate frame when the observer is close to the surface is for ocean rendering. By using ENU coordinates, we can emulate a flat service and get round-Earth coordinates at the same time. This lets us use the ENU z-axis as “up” for the ocean. This will be exactly true at the ENU origin but the error in the up direction increases the farther away from the origin the observer goes.
Again, looking at the chart information, the chart uses the Mercator Projection based on NAD83. We must project the Mercator coordinate back to geographic.
Note if the chart was referenced to a different ellipsoid other than WGS84, we would need to perform a datum transformation (or datum shift) as well.
Using Global Mapper, a GIS software application that can read RNC charts, the location of the 40-meter sounding has Mercator coordinates of X= -13905279.486 and Y = 6413303.196. I talked about map projections in my Earth Coordinate Systems blog post but I will just skip straight to the answer here. The geographic coordinates for that point are 49° 59' 55.3849" N, 124° 54' 47.7033" W.
So now we have a geodetic point for that location:
Lat: 49.99871802
Lon: -124.91325092
Elevation: 36.78 meters
Converting from geodetic to ECEF using the formula from my previous blog post, we get:
X: -2351.153 km
Y: -3368.638 km
Z: 4862.726 km
We can now position that point in our WGS84 ECEF world coordinate system. Next we need to scale the solution.
If we step back and look at the chart as a whole, we can see that it could be converted to a heightmap of sorts. We could use a best-fitting, 3D surface model and create a heightmap from each chart file. This is exactly what Bending Time intends to do. If the charts end up being too large on their own, they will be tiled for more efficient download and runtime processing.
Circling back, it’s clear that we need nautical charts when we are close to the planet’s surface. However, these very specific ocean heights will likely not make a difference to an observer flying in a virtual plane at 37,000 ft for example. This is where the geoid comes in.
We use the geoid heights for planet-scale ocean heights and nautical charts for when we are close to the planet’s surface. The geoid also serves as a way for the simulation to calculate AMSL heights, which could be used in GPS or aircraft simulation.
Wave Geometry
Okay, now that we can model the correct height of the ocean, we need to add waves when we are near the surface.
One of the main reasons Bending Time chose to use the Local ENU coordinate frame when the observer is close to the surface is for ocean rendering. By using ENU coordinates, we can emulate a flat service and get round-Earth coordinates at the same time. This lets us use the ENU z-axis as “up” for the ocean. This will be exactly true at the ENU origin but the error in the up direction increases the farther away from the origin the observer goes.
ENU coordinate system
Looking back to Finch’s “Effective Water Simulation from Physical Models,” we now implement the Gerstner waves as the first step. I use three waves with additive properties resulting in the wave geometry being calculated in an ocean shader in the ENU frame. This is where I left off.
One of the challenges I was facing was, even at the ENU surface level, the scale of the scene is very large, around 100 km x 100 km. I will circle back to ocean rendering after I complete the enhanced terrain shading work.
An alternative to the Gerstner approach is to use a Fast Fourier Transform (FFT) to simulate waves. There are several solutions I’ve seen that do this with fantastic results. However, the planetary-scale issues remain regardless of the approach you take to wave geometry.
Further, I highly recommend Jasper Flick’s Catlike Coding website for easy-to-understand rendering techniques. Specifically, his Waves article is a very nice introduction to wave geometry.
One of the challenges I was facing was, even at the ENU surface level, the scale of the scene is very large, around 100 km x 100 km. I will circle back to ocean rendering after I complete the enhanced terrain shading work.
An alternative to the Gerstner approach is to use a Fast Fourier Transform (FFT) to simulate waves. There are several solutions I’ve seen that do this with fantastic results. However, the planetary-scale issues remain regardless of the approach you take to wave geometry.
Further, I highly recommend Jasper Flick’s Catlike Coding website for easy-to-understand rendering techniques. Specifically, his Waves article is a very nice introduction to wave geometry.
Let’s move onto ocean shading and lighting.
Ocean Shading
As most developers do, I approached the shading of the ocean using the simplest setup possible: a simple sine wave. Better to get the basic shading in place before adding advanced effects. Moving vertices using a sine wave is fairly straightforward once you have the vertex in the ENU frame. The next step is deciding what colour the pixel should take. I’ve seen several ocean simulations use textures as part of determining pixel colour, which has the bonus of being able to add effects like foam/wash. However, I didn’t want to use a texture at the beginning as I figured I should be able to shade the surface using just the light direction vector, colour and intensity. At the time, I had heard about Physically-Based Rendering (PBR) so I thought I should at least learn a little more. Well, I fell down the PBR rabbit hole. So much so, I developed a standalone PBR test tool in Unity, which I will publish in my next blog post.
Bending Time models the Sun position accurately based on the day and time and maintains a light direction vector as well as an intensity value. The light colour is dependent on the atmosphere, which is the next blog post after the PBR test tool. To cast the sunlight on the ocean, the ECEF sunlight direction vector is transformed to our ENU frame and passed to the ocean shader. From here we calculate the usual PBR vectors and apply ambient, diffuse and specular properties. The result was “meh.” At this point, I turned my attention to the colour of the sea.
As someone who is very familiar with NASA’s products, I started at their Ocean Color page at https://oceancolor.gsfc.nasa.gov/. I was surprised to see the effort on NASA’s part in tracking the phytoplankton biomasses across the planet. Turns out the chlorophyll concentration in the plankton is a major contributor to the colour of the ocean’s surface because the plankton live at the surface to collect the sun’s light for their photosynthesis process.
Ocean Shading
As most developers do, I approached the shading of the ocean using the simplest setup possible: a simple sine wave. Better to get the basic shading in place before adding advanced effects. Moving vertices using a sine wave is fairly straightforward once you have the vertex in the ENU frame. The next step is deciding what colour the pixel should take. I’ve seen several ocean simulations use textures as part of determining pixel colour, which has the bonus of being able to add effects like foam/wash. However, I didn’t want to use a texture at the beginning as I figured I should be able to shade the surface using just the light direction vector, colour and intensity. At the time, I had heard about Physically-Based Rendering (PBR) so I thought I should at least learn a little more. Well, I fell down the PBR rabbit hole. So much so, I developed a standalone PBR test tool in Unity, which I will publish in my next blog post.
Bending Time models the Sun position accurately based on the day and time and maintains a light direction vector as well as an intensity value. The light colour is dependent on the atmosphere, which is the next blog post after the PBR test tool. To cast the sunlight on the ocean, the ECEF sunlight direction vector is transformed to our ENU frame and passed to the ocean shader. From here we calculate the usual PBR vectors and apply ambient, diffuse and specular properties. The result was “meh.” At this point, I turned my attention to the colour of the sea.
As someone who is very familiar with NASA’s products, I started at their Ocean Color page at https://oceancolor.gsfc.nasa.gov/. I was surprised to see the effort on NASA’s part in tracking the phytoplankton biomasses across the planet. Turns out the chlorophyll concentration in the plankton is a major contributor to the colour of the ocean’s surface because the plankton live at the surface to collect the sun’s light for their photosynthesis process.
NASA’s “Chlorophyll a” global data product
This data, like most other remotely sensed data, is quite large hence it is tiled for easier download and processing. Bending Time loads this data in and the texels from the chlorophyll image are provided to the ocean shader. This is then summed with a “base colour” which is just hard-coded for now. Admittedly, the results were far from perfect and the whole ocean rendering work was left in this halfway state. I look forward to getting back to this work as it’s challenging and the end “product,” an animated ocean rendered on a true-scale 3D world, will be quite novel.
Future
There are many more sub-topics when rendering the ocean:
Developing the overall solution for planetary ocean rendering has already been expensive and I’m not even done yet! So, the additional features above must be tied to business if there’s a hope of funding their development. Having said that, the first three items, essentially ocean waves coming to shore, make a big difference to the visuals especially the first one. You don’t want your waves coming to shore on an off-kilter angle because it would look obviously unnatural to the observer.
I investigated wave refraction and the concept is fairly simple: waves refract as they enter shallower water. The folks at the University of Hawaii have put together an excellent resource all about the ocean called Exploring Our Fluid Earth, which can be found at https://manoa.hawaii.edu/exploringourfluidearth/, which includes a page about wave refraction among a plethora of other pages about the ocean.
Future
There are many more sub-topics when rendering the ocean:
- Wave refraction
- Shoaling/surf
- Shoreline/beaches
- Tides
- Wind effects
- Whitecaps
- Joint North Sea Wave Observation Project (JONSWAP)
- Sea foam
- Boat wash
- Underwater
- Ocean bottom
- Depth/transparency
- God rays
- Marine life
- Coral reefs
Developing the overall solution for planetary ocean rendering has already been expensive and I’m not even done yet! So, the additional features above must be tied to business if there’s a hope of funding their development. Having said that, the first three items, essentially ocean waves coming to shore, make a big difference to the visuals especially the first one. You don’t want your waves coming to shore on an off-kilter angle because it would look obviously unnatural to the observer.
I investigated wave refraction and the concept is fairly simple: waves refract as they enter shallower water. The folks at the University of Hawaii have put together an excellent resource all about the ocean called Exploring Our Fluid Earth, which can be found at https://manoa.hawaii.edu/exploringourfluidearth/, which includes a page about wave refraction among a plethora of other pages about the ocean.
Ocean Wave Refraction
https://manoa.hawaii.edu/exploringourfluidearth/physical/coastal-interactions/wave-coast-interactions
https://manoa.hawaii.edu/exploringourfluidearth/physical/coastal-interactions/wave-coast-interactions
While the concept is easy to understand, implementing ocean wave refraction and surf is difficult especially on a planetary scale.
I leave this topic and the others listed above to a future where Bending Time is making money on ocean-related visualizations/simulations.
Conclusion
This blog post ended up being quite a bit more involved than I had originally anticipated so let me try and synthesize it down to a few salient points:
I hope you were able to make it this far. Personally, I find the topic of planetary ocean rendering fascinating and I really look forward to getting back to work on it.
Explorarent maris.
--Sean
I leave this topic and the others listed above to a future where Bending Time is making money on ocean-related visualizations/simulations.
Conclusion
This blog post ended up being quite a bit more involved than I had originally anticipated so let me try and synthesize it down to a few salient points:
- A global geoid defines the height of the ocean surface at planetary scales
- Nautical charts are needed to understand the height of the ocean at local scales
- Generating the wave geometry has been solved in XYZ and we need to be creative to apply these solutions to a round Earth
- Shading the ocean needs to consider things like chlorophyll concentrations to accurately depict colour
- Funding realistic planetary ocean rendering must be tied to business
I hope you were able to make it this far. Personally, I find the topic of planetary ocean rendering fascinating and I really look forward to getting back to work on it.
Explorarent maris.
--Sean
RSS Feed