The maximum quality, if you will (I-IV) is determined by the gear's optimal rank (40-50 => IV, 30-29 => III, 20-29 ==> II, everything else I).
However, for any given level, there is a chance you will generate non-maximum materia. So a rank 50 item can yield Grade I materia.
So, max threshold determined by rank of gear, but there's always a probabilistic element.
That as well as:
Each Grade has a min,mid,max range, meaning a IV hp+ can have 50,55,60 HP.
Having armor fixed to 100% before braking to materia is said to give you better odds of better result.
+1.+2,+3 play a role though unconfirmed. Same with moon phase.
There are a few rumors as well BUT, i those are more of a superstition.