ArcMap on Node Dangles

Using ArcPy.Mapping to update Definition Queries

Tue, 08 May 2018 17:56:09 -0500

Working with a routine process today that I normally only do once in awhile but today needed to do it several times. It requires changing the definition query on several features classes. Being the ‘lazy’ GIS guy that I am (owner of a company I used to work at called me that once as a sort of compliment for my tendency to script a lot of what I did), I decided to finally script it instead of changing definition queries about 42 times.

The quick & dirty ~~script~~ code that I wrote & can be run in the ArcMap python window consists of two parts. The first three lines just need to be run once per session to get a variable set with the list of layers. The loop I ran multiple times, updating the ‘.replace’ settings each time. I ran the loop, did my other processes, and re-ran the loop as needed and was a happy GISer.

#These first 3 lines only need to be run once.
mxd = arcpy.mapping.MapDocument("CURRENT")
l = arcpy.mapping.ListDataFrames(mxd)[0] #Note the hard-code [0], meaning the first dataframe. YMMV
ll = arcpy.mapping.ListLayers(l)

#Everytime I wanted to update the definition queries just need to update the replace parameters.

for l in ll:
     try:
        print(l.name)
        print(l.definitionQuery)
        z = l.definitionQuery.replace("_011","_012")
        l.definitionQuery = z
     except:
        print("")

ArcMap Field Calculator: Number of parts in multi-part feature

Thu, 05 Sep 2013 14:04:40 -0500

In the last week, I have looked for multi-part features a couple of times. Today, I was looking for multi-part polygons after dealing with the fall-out of a case of Clip Gone Wild as shown below.

I have not found a way to write a query to find these but Field Calculator does allow you to calculate a field’s value to the number of parts.

Using the Python parser, just write the formula (note that case matters): !shape!.partCount

Quick & Dirty arcpy: Autopan ArcMap using arcpy

Wed, 02 May 2012 05:19:40 -0500

Question: How do I get ArcMap to automatically pan through an area.

As I mentioned in a previous post, I recently had the need to have ArcMap automatically pan through a project area. My first attempt was to print a series of data-driven pages (using a fishnet polygon layer as the index) this but that did not accomplish what I needed so I switched to arcpy, which made the task simple enough. Nothing special or tricky about this code, but just did not find it anywhere else.

The one thing to note is that I have a 1 second pause between pans–this was to allow image tiles to download. You will need to adjust the delay to meet your needs. The toolbox and code can also be downloaded.

import sys,arcpy,datetime
inLayer = sys.argv[1]

def printit(inMessage):
    print inMessage
    arcpy.AddMessage(inMessage)

mxd = arcpy.mapping.MapDocument("CURRENT")

arcpy.MakeFeatureLayer_management(inLayer, "indexLayer")
cur=arcpy.SearchCursor("indexLayer")

df = arcpy.mapping.ListDataFrames(mxd)[0]
newExtent = df.extent

iCount = 0
iTotal = (arcpy.GetCount_management("indexLayer").getOutput(0))

for row in cur:
    thisPoly = row.getValue("Shape")
    newExtent.XMin, newExtent.YMin = thisPoly.extent.XMin, thisPoly.extent.YMin
    newExtent.XMax, newExtent.YMax = thisPoly.extent.XMax, thisPoly.extent.YMax
    df.extent = newExtent
    time.sleep(1)
    iCount+=1
    printit("Panned to feature {0} of {1}".format(iCount,iTotal))

del row
del cur

ArcMap Field Calculator: Identifying Unique Cases, Single Field

Sat, 28 Apr 2012 05:55:34 -0500

Seems like a lot of people are finding the ArcMap Field Calculator examples that I have posted useful so I will make an effort to post more of them. Most posts are generated after I do something and think that others might want to know how to do it. (Or so I can go back and remember how I did something without re-inventing it).

Something I did today was create a field (!Case!) and then populated this with a unique identifier for each different value (case) that occurred in a different field (!Feature!).

Note: python’s index statement is a 0-based search so the first case will have the value 0, the second will have 1, and so on. If you want to start the results at 1, you can make the last line: ‘return caseList.index(inValue) + 1’.

The basic structure for this is shown:

caseList = [ ]

def returnCase(inValue):
   global caseList

   if not inValue in caseList:
      caseList.append(inValue)

   return caseList.index(inValue)

ArcMap Field Calculator: Identifying Unique Cases, Multiple Fields

Mon, 23 Apr 2012 06:06:25 -0500

You may have noticed that this post–ArcMap Field Calculator: Identifying Unique Cases, Single Field–specifies ‘Single Field’. Yes, that was my version of a cliff-hanger post.

The basic structure I listed in that post can be expanded on to satisfy your needs. The example in my earlier post was case sensitive for example, you could modify it so it treats ‘a’ the same as ‘A’.

Today’s example groups records into different cases based off the values of two fields, !county_c! and !feature! and required only minor modifications.

The calling line was modified from:

From:

   returnCase(!feature!)

To:

   returnCase(!county_c!,!feature!)

to accommodate passing both values.

The function definition likewise was modified to accept two values, this:

   def returnCase(inValue1):

to:

  def returnCase(inValue1, inValue2)

And this line was added, creating a list from the two values passed in:

   inValue = [inValue1, inValue2]

**(Note: The same results could have be achieved by using the original function by creating the list in the calling statement: returnCase([!county_c!,!feature!] )
**

caseList = [ ]

def returnCase(inValue1, inValue2):
   inValue = [inValue1, inValue2]
   global caseList

   if not inValue in caseList:
      caseList.append(inValue)

   return caseList.index(inValue)

ArcMap Field Calculator: ArcPy Date to Decimal Function

Thu, 06 Oct 2011 21:08:35 -0500

One of the standards in our databases is to store dates as 8-digit integer values in the format of yyyymmdd. This requires us to occasionally convert values from date fields into this format.

We can do this in the ArcMap Field Calculator using this arcpy function:

def datetodouble(inNum):
   splitList = str(inNum).split("/")
   return  splitList [2]  +("0"+ splitList [0])[-2:]  +("0"+ splitList [1])[-2:]

ArcGIS Add-In Custom Mouse Cursor

Tue, 19 Jul 2011 20:47:03 -0500

I was working on a project and wanted my own custom mouse cursor and did not easily find a way to make your own in ESRI’s instructions. But, once you know how to do it, it is pretty easy. In Visual Studio, Add a New Item:

Add a Cursor File:

You can edit your cursor with the editor program in Visual Studio. Once you satisfied with how it looks, make sure that the Build Action on the cursor is ‘Embedded Resource’.

Then you can set your cursor with two lines of code. Note that my cursor is in my QDI.QdiAddIn Namespace:

Dim pCursorStream As System.IO.Stream = Me.GetType.Assembly.GetManifestResourceStream("QDI.QdiAddIn.NewCursor.cur")
MyBase.Cursor = New System.Windows.Forms.Cursor(pCursorStream)

44.852994-93.55073

Change Detector arcpy Script

Fri, 08 Apr 2011 10:48:59 -0500

During a process I was working on, I needed to compare a feature class before and after some edits. I did not quickly find anything in ArcToolbox but searching ArcResources led me to Change Detector script by Bruce Harold. After making a couple of tweaks–for some reason in one of my feature classes, the Shape field had an upper case ‘S’ and in the other it was a lower case ’s'. I also discovered that it needs to export to the same format (personal geodatabase, file geodatabase, shapefile) as the source data (or at least one that uses the same field name deliminator).

After minor adjustments, though, it worked like a charm. I’ll be submitting the changes I made to Bruce and let him incorporate the changes into the official code.

FOLLOW-UP: Mr. Harold quickly responded to my email & made the change (although I haven’t checked it). Way to go Bruce! Thanks for a handy script.

44.852994-93.55073

ArcMap Field Calculator: Calculating Geometry using arcpy

Tue, 08 Mar 2011 09:30:40 -0600

One of the things I had not gotten around to doing in ArcGIS 10 was figure out how to directly manipulate the geometry of a record using the Field Calculator. When I stumbled upon a bug in the way the Extract Values to Points tool handles Null geometries, I figured it was time to figure it out. Setting the X, Y to 0,0 was sufficient for my needs.

I set the Parser to Python and the formula was simple once I figured out the syntax:

pPoint = arcpy.Point(0,0)

Then, to extend my knowledge, I wanted to see how to calculate the geometry based off of two fields (X and Y). The only real trick is knowing the bracket field names using exclamation points instead of brackets:

arcpy.Point(!X!,!Y!)

So turns out everything it pretty easy and straight-forward. Whew!

ArcMap Copy and Paste Features

Wed, 13 Oct 2010 16:28:39 -0500

Sometimes it seems like I could write the data down by long hand faster than ArcMap can copy & paste features from one feature class to another. geeeeezzz.

44.852994-93.55073

Another TopoToRaster Error

Wed, 15 Sep 2010 13:36:34 -0500

Subtitled: Why error messages are good.

Came up with another error while running TopoToRaster but this time ArcGIS gave an error message that led to a solution. Turned out all my contour lines had an elevation of 16 which TopoToRaster did not like. I had intended to increase the elevation and inadvertently set them all to sixteen. I had saved the previous values before editing so it turned out to be a simple fix and I didn’t have to spend a day trying figure out what was wrong.

TopoToRaster Error

Mon, 13 Sep 2010 20:25:01 -0500

In running an automated process, I had a TopoToRaster repeatedly fail on me. The only input theme was a contour theme. The process ran fine when I used the envelope of the contour theme as the output extent but when I changed it to the envelope of a polygon theme, it would bomb. The polygon’s envelope was smaller than the contour theme.

ArcCatalog would bomb out without presenting any sort of useful message.

I determined eventually that If I set the left extent as 435210, the process would work if set the right extent to 446655 or greater but would bomb if I used 446654 or less. If the the left extent was 435209, the right extent had to be 446654 or greater.

The image below shows the contours (with vertexes) and the polygon theme I was using. The two circle graphics show the approximate extent I was clipping to.

I eventually wrote python script to do a series of tests with the intent of determining the pattern of what does and does not work. The unexpected benefit was I actually got some sort of error message back. Turns out it had to do with my lack of vertices–I densified my polylines and it ran fine. Something I should have thought of earlier but it was difficult lacking any feedback.

ArcBruTile

Thu, 09 Sep 2010 14:38:46 -0500

Stumbled across ArcBruTile, an ArcMap application that allows you to display map tile services–such as OpenStreetMap, Google Maps, Bing, Spatial Cloud, and Tile Map Service (TMS)–in your ArcMap documents. It works pretty well from my preliminary testing of it. I have found that at times that the graphical tiles are distorted but I am guessing it is because they are optimized for display in a specific projection, at specific scales and, in ArcMap, you can use any projection/scale combination you want.

This is a free, open-source project that is still in Beta but definitely an application I plan to keep an eye on. As with any open-source project, the cool thing is the code is available for you to look at, learn from, and hopefully contribute to.

Workstation ArcInfo Interchange Import Error

Tue, 07 Sep 2010 19:08:39 -0500

I have been working on updating some data that was published using Arc/Info Workstation Export (.e00) files, converting them to shapefiles, or in the case of annotation, to a geodatabase feature class.

While converting some INFO tables, I have received this error:

It turns out that Workstation can create export files that contain values that did not fit within their field–I’m guessing that somehow you are actually able to enter invalid values into those fields but I did not confirm that.

The error, ‘Unable to encode the 38th item in the 65th record…’ was only partially helpful. The error was in the 65th record but I was not able figure out which field contained the error. The nature assumption that it was the 38th column was incorrect, at least if you are counting from left to right. Whatever logic that is used, is consistent, at least within a specific table, because I had numerous records with problems and when the error messages indicated it was the same item, it did turn out to be the same field.

What I ended up having to do was open the export file in my favorite text editor, locating the info for problematic record and go through the columns one-by-one, comparing the value in the text file with that in the table. In this case, 14.1 was suppose to be the value in the WP_TIO2 field.

One thing I noticed, and this is consistent with behavior I have observed in ArcObjects in other cases, is that the record with the invalid value in a field ends up with the same value in that field as the previous record. So in this case, the bad record has a value of 1.12 in the WP_TIO2 field because that is what previous record has. That leads me to believe that ArcObjects is re-using a variable as it does the import, just updating each value, field-by-field.

Just to make sure that I had diagnosed the problem right, I attempted to change the value to 14.1 in an ArcMap edit session and got this message:

A look at the field properties confirmed that the field would not accept 14.1 as a value. The precision (3) means the field can hold 3 digits with 2 (the Scale) of them being right of the decimal point–leaving only one digit available to the left of the decimal point.

Once I identified the problem, I used two methods to correct it. The majority of my cases were actually cases where a psuedo-null value was being used to indicate that a field was really Null. In some cases, either -999.9 or 100 were being used. In these cases, I just substituted a different psuedo-null value (-99.9) that conformed to the field limitations. The data I was working with was not tightly controlled in the past and there were other existing records that used these value for the same fields, so I was able to do this without much concern.

Some values, however, were not nulls and I felt needed to be retained. These, I had to add a new field that could contain the values, copy the values over and then edit the ones that had caused problems during the import. Once complete, I exported the data to a new table with the new table structure I created.

Utah Geological Survey

Wed, 04 Aug 2010 16:50:04 -0500

The Utah Geological Survey has a good supply of Maps and data available online and some online maps. They also have an online Blog.

The GIS data is a fairly complete package–the sample I downloaded included .pdfs, shapefiles, layer files, images and ArcMap project (.MXD) files used to generate the .pdfs. File sizes are larger because of the completeness. Feature classes have a moderate level of attribution.

The highlight, however, are the sample Keyhole Markup (KMZ) files of the of the St. George 30′ x 60′ quadrangle that can be viewed in Google Earth. Too cool.

Create Geologic Cross Sections–eXacto Section v. 2.0, ArcMap 9.3

Fri, 23 Jul 2010 13:59:44 -0500

The Create Geologic Cross Sections–eXacto Section v. 2.0, ArcMap 9.3 written by Jennifer Carrell of the Illinois State Geological Survey is a handy tool for creating cross sections.

It requires ArcGIS and a 3D Analyst Extension license.

You can create profiles against multiple DEMs at once, define the vertical exaggeration, and have it include contact points. It is well documented and comes with sample data to use with the tutorial.

I did not see it mentioned but it also include a personal geodatabase that include a grid that can be used along with the profiles you create.

If you make profiles, think this is definitely an option you should look at. The fact that it is a VBA application is a bit worrisome in that its lifespan is limited.

Hogeweg: ArcGIS Editor for OpenStreetMap

Wed, 07 Jul 2010 19:10:35 -0500

As someone who lives in the boon-docks whose street (both my current and previous one) does not appear in the data for either NAVTEQ or TeleAtlas and someone who used both of those data sets extensively in his previous job, I’ve watched OpenStreetMap for awhile, although have not contributed.

Marten Hogeweg, ArcGIS Server Geoportal Extension Product Manager, ESRI announce in his blogs an ArcGIS Editor for OpenStreetMap. Sounds like a great idea, giving GIS professionals–some whom have the source materials the ability to easily contribute should only help the project.

Unfortunately, it is only available at 10.0 but that gives me another reason to install it on a virtual machine and start playing more.

Sticky Move Tolerance

Thu, 01 Jul 2010 16:51:42 -0500

While I am pretty experienced in editing data in previous generations of ESRI’s software, I have not done a lot in ArcGIS. So in my new position where I am supporting a group of geologists who do a lot of data creation in ArcMap, I am learning some of the intricacies of the ArcGIS platform.

One of the potential gotchas that a coworker informed me about that I was vaguely aware of was the issue of sticky move tolerances. What can happen, when editing existing features, is that each time you select a feature it can easily get moved very slightly if the mouse moves while the feature is selected. While the movement itself may be well within any allowable accuracy requirements, it can make a huge mess if you are working with a shapefile polygon coverage for example.

ArcMap has a setting, Sticky Move Tolerance, that helps prevent these tiny, accidental moving of features. A feature will not be moved until the mouse moves at least as many screen pixels as this setting. The default setting is 0, disabling this feature.

This setting can be set changed under Options on the Editor Toolbar.

The Sticky move tolerance is on the General tab. In this example, I have my setting at 25 pixels which may be a bit high but I do not do a lot of data editing myself so it is workable. The downside of having it set this high is if I want to move a feature just a tiny bit, I have to first drag the mouse relatively far away and then bring it tighter into where the feature is.

Not sure how well this is documented although it can be found in the help system.

ArcMap-Excel Join Error

Fri, 28 May 2010 02:02:37 -0500

Came across a new ArcMap bug today. A staff member was trying to join an Excel 2007 file (.xlsx) to a point shapefile in ArcMap and it did not appear to be working. She only had Office 2003 on her machine so I tried it on my spanking new machine that has 2007 installed. Same result–the fields get appended but they are all blank.

I discovered, however, that when I had the table open, if I switched form showing ALL records to only the selected, Poof!!! The values filled in. From there I was able to successfully export the data to a new shapefile that always showed the values.

I have an inkling that the file name, which contained space and other special characters, may have been a factor.

August 11, 2010 Addendum:

I was just working with some more and, after joining a table to an Excel table, having all the fields look blank, I discovered that if I close and then re-open the table all looks good–all fields have values.

Another problem I had is that even after successfully joining a table to the Excel table and creating an XY Event Theme from it, I was unsuccessful in exporting it to either a shapefile or a personal geodatabase feature class. My personal opinion at this point would be to convert the Excel file to a different format before working with it in ArcGIS.