This may very well have nothing really to do with video memory but rather can be caused by an overload of addons. If that's not your problem, however, read on.
This most often happens on systems with graphics adapters with less than 32MB of memory. It sometimes occurs specifically when using more graphically complex island addons.
To attempt to resolve this problem, try lowering your resolution, both in size and from 32 bit to 16 bit. Make sure you have the latest or most stable version of your graphic adapter's drivers.
If you've tried everything including reducing all other graphic related preference and option settings that OFP makes available to you, it's time to seriously consider getting a better graphics adapter with more video memory onboard.