Apparently, if you post one time about ArcMap field calculator, you’re bound to get additional questions.  After my recent post about using field calculator to convert text values to numeric, someone asked about a problem they were having with another calculation they were having.

The underlying problem was that python 2.6, which is installed with ArcGIS 10, uses integer division when both the numerator and denominator are integers. The result of integer division is an integer rounded towards negative infinity.

If you’re not aware of this (or forget) then you open yourself up to some unexpected results–they can be especially hard to catch if you are using it within a larger block of code.

In this example, I’m calculating a percentage but the result is 0 for all the records because of the rounding.

The easiest thing to do in this simple example is just convert one of the two value to a non-integer value. This can be done by multiplying by 1.0 (not just ‘1’, you need to include the ‘.0’) which is a float datatype. Multiplying one of the values by a float makes that value a float and integer division no longer applies & we end up with a happy GISer.

Another option if you are using a Codeblock is to include ‘from __future__ import division’ in your code block. Python is slowly moving away from using integer division and the _\_future__ module overrides the default behavior.