Yes, indeed you can't use the raw modularity scores to say whether it is a "good" partition, because a similarly high score may also be obtained in a random graph. Compare it to the traditional clustering coefficient: if the two variables are independent (the 'null' condition), the correlation is zero, so any correlation away from zero is "high". For modularity however, this value is not zero for random graphs (the 'null' condition), so that it's difficult to say whether modularity is 'high' and therefore a 'good' partition.
One way to asses the significance of a partition (how different from the 'null' condition is it), would be the technique I introduced here (http://bit.ly/1aDFatJ
There are by the way also ways to overcome the resolution limit (see http://bit.ly/1amD2Jj
for technical details), but these are not implemented (yet) in Gephi. However, one would also need to choose a particular resolution value, for which it might helpt to construct a 'resolution profile': what do partitions look like for various resolutions (some details are provided in http://bit.ly/1aDFatJ