I posted the basics of how the poison stacks in the general discussion thread a while ago, what happens is each standard ovi hit does 0.3% per tick for a total of 1.5% over 5 ticks (if the ticks are already going when you hit, that 1.5% will be distributed between the remaining ticks, thus seeming like more damage than it really is). Each successive standard ovi hit during the 5 seconds (it is exactly 5 seconds, 1 per tick) will add another 1.5% to the total DOT. Additionally, each extra poison tick gains some extra damage based on how many hits you have stacked already, such that hit number 2 will actually add 2%, hit 3 will add 2.5% and so on. The numbers change a little if you use njp or the second hit of f22, however, and if you mix them then they start to add up differently to how you'd expect, so clearly the formula isn't quite so simple.
Basically, each hit adds the standard poison damage of that hit plus a bonus multiplied by the number of poison hits currently in the counter, the remaining poison damage is distributed between the remaining ticks and when the 5 ticks are over, the damage and number of poison hits both reset to zero.
