A recent report from Secunia states that popular Windows applications don't use the DEP and ASLR protections. It is true for VLC up to 1.0: the latest version at the moment, 1.1, supports permanent DEP mode, and ASLR on all of its DLLs.
One thing the report could have shown is the difference between applicatins built with MSVC or GCC. Adding DEP and ASLR in Visual Studio means adding /NXCompat and DynamicBase to the compilation options. With MinGW, there is a different trick. This article on my old blog is slightly outdated: ld in binutils 2.20 supports the --nxcompat and --dynamicbase options. So, now, the developers using GCC have no more excuse!
Let's sum up the state of the security of VLC:
- 1.0.5 is NOT SAFE on Windows. 1.0.6 brings a lot of security fixes, but this version was not released on Windows. And security features are not used.
- 1.1.0 supports permanent DEP and ASLR (with DllCharacteristics flag, only on Vista/7) and termination on heap corruption
- 1.1.1 supports the same as 1.1.0, and adds DEP on XP SP3 with SetProcessDEPPolicy
- SafeSEH and stack cookies are not yet used
The developers using LibVLC should check their software: DEP won't be activated if their executable doesn't support it.