Home/Technical Information/Technical Notes
Technical Notes
A summary of the Lumerical FDTD GPU solver: NVIDIA CUDA driver and GPU hardware requirements, how licensing is counted from SM count, the conditions a simulation must meet, and the current limitations on solvers, sources and monitors.
Since the 2023 R2 release, Lumerical FDTD has supported GPU computation.
This page summarises the requirements and current limitations of the Lumerical FDTD GPU solver.
Running a Lumerical FDTD simulation on a GPU requires NVIDIA CUDA driver version 450.80.02 or later (Linux) or version 452.39 or later (Windows). In addition, the NVIDIA GPU must meet the following requirements:
To monitor GPU usage, use the “GPU-Util” value reported by the NVIDIA System Management command-line utility. Windows users should note that Windows Task Manager reports only graphics-related GPU utilisation.
GPU solver usage is similar to CPU solver usage (see Ansys optics solve, accelerator and Ansys HPC license usage). The license usage calculation treats Streaming Multiprocessors (SM) the same way as CPUs. With an Ansys Standard or Business license, for example, every 16 SM requires one Lumerical Accelerator (engine license). There is no partial counting: a 40 SM GPU, for example, requires three licenses to run any job.
An FDTD job uses every available SM in the GPU, which means the number of SM per job is not user-configurable. You therefore need enough licenses to cover all the SM in the GPU.
When running several jobs we recommend running them in series rather than in parallel. Parallel execution requires as many licenses as there are jobs but takes the same time as serial execution. Running two jobs simultaneously on a GPU of 16 SM or fewer, for example, requires two licenses, whereas running them one after the other on the same machine with only one license finishes in roughly the same time.
You can find the number of SM in a GPU from the NVIDIA documentation, from third-party websites, or by running the GPU resource test in Job Manager Configuration (localhost only).
Log file from a run of the FDTD GPU engine
start loading CUDA query DLL... load CUDA query DLL successfully. GPU streaming multiprocessors (SMs): 16
“total gpu sms” in the FDTD results after running the FDTD GPU engine
Note: as with CPUs, memory bandwidth matters more to performance than core count (see the FDTD benchmarks on CPU).
The Lumerical FDTD GPU solver can only run 3D FDTD simulations. The “express mode” option must be enabled in the FDTD object properties (advanced options tab).
The GPU solver is suited to narrowband simulations, or to those using non-dispersive materials.
All movie monitors are disabled.
In the Resource settings, switch the job manager from “CPU” to “GPU”.
Turning on the FDTD solver property “express mode”
setnamed("FDTD", "express mode", true);
FDTD resource manager
setresource("FDTD","GPU", true);
setresource("FDTD", 3, "GPU Device", "Auto");
setresource("FDTD", "GPU", false);
setresource("FDTD", "CPU", true); //< equivalent to previous line
So that a GPU on a remote host can be used, the “GPU device” value is not validated. The default is “Auto”; you must set an integer value.
Only PML boundary conditions are supported. Bloch, periodic, symmetric, antisymmetric, PEC and PMC boundary conditions are not supported.
Mode sources and ports
Total-field scattered-field (TFSF) sources
Time monitors
Frequency-domain monitors
Layer builder
Products and software
Analysis technologies
From confirming that the settings in this article are sound, through to applying them to a problem of your own.