Total Rounding Amount option located on menu Configuration - Order - Order Details.
Use Total Rounding Method to specify rounding formula:
For example, set Total Rounding Amount field to 0.05 and Use Total Rounding Method set to Nearest Then total will be rounded to nearest 5 cents.
Set this field to 0 if you don't want to do rounding.
See also: