I found the solution. I originally thought that it worked like that also however it doesn't, at least on the later versions. Basically you have to modify the <Directrory> Directive here is how I configured it for my virtual host.
The Options FollowSymLinks needs Indexes to follow it.
Options FollowSymLinks Indexes