Full post about my first round of testing can be found here: http://ffxiv.zam.com/forum.html?forum=176&mid=1286865364175002057&page=1#m1286865364175002057
Used this parser for the testing: http://www.ffxivcore.com/topic/11874-pseudos-ffxiv-log-parser/
I plan to do more testing with different set ups, such as using all level and class appropriate armor and accessories, then only changing the weapon to lower level and higher level weapons, etc. This is not nearly a large enough sample size to draw conclusions from, especially since it is possible to change various other stats so easily to balance out gear or weapons usage. I also have no idea how monster level and your current gear/weapon level relate to each other as of yet. You may, for example, do better with a +5 level weapon when fighting red level monsters due to some check performed when you hit them, much more testing will be needed.
However, even during these early test, I feel that there is significant penalties to wearing higher level armor and using higher level weapons. It stands to reason that the more of a level gap you have between optimal rank for the item and your current rank, the worse the penalties, but I will test that more in future parsing. Many people have observed that upgrading from Weathered weapons to the Lv 7, 9, 10 items made huge improvements to their ACC and DMG. It looks like higher level items than your rank may have a similar effect.