Setting up your development environment for Python and ArcGIS Desktop

By: Christopher Fricke

Setup Aptana

  1. Download Aptana and Install <http://www.aptana.com/>
  2. Open up Aptana

    Pic1

  3. Configure your workspace.  This will be where all of your project files will be stored.  I recommend throwing this in C:\Users\YourUsername\Source.  You can hit that checkbox so that it doesn’t ask you ever again.

    Pic2

Configure Python

  1. Once Aptana is open, you will need to specify your Python Interpreter.  This process allows you to specify  the version of Python that ships with the ArcGIS for Desktop or ArcGIS for Server installed on your machine.  [Note: ArcGIS for Server ships with x64 bit python.  You can add in both 32 and 64 bit interpreters if you have desktop and server on the same machine.]
  2. First open up Aptana preferences

    Pic3

  3. Open PyDev / Interpreter – Python

    Pic4

  4. Click the button labeled New…
  5. Give the interpreter a unique name.   I am labeling it ArcGIS Python 2.7 x32, because this is for the default python location for ArcGIS for Desktop 10.1.  If I were doing server, I would label it ArcGIS Python 2.7 x64.
  6. Also, add in the path to the python executable:
    – The default path for 10.1 desktop is C:\Python27\ArcGIS10.1\python.exe
    – The default path for 10.1 server is C:\Python27\ArcGISx6410.1\python.exe
    – The default path for 10.0 is C:\Python26\ArcGIS10.0\python.exe

    Pic5

  7. Click OK a few times.
  8. The python interpreter is now ready for action.

Your first project!

  1. First select the button labeled “Create Project” in the right hand window of Aptana.

    Pic6

  2. Select PyDev / PyDev Project

    Pic7

  3. Give your project a cool name, like “Example” and make sure to specify the Python interpreter you created in the previous section.  Click Finish.

    Pic8

  4. Create a python package by right clicking on your folder and selecting New / PyDev Package.  This is the folder you will be creating all of your code in.  Name it something clever like, example.  A python module is a folder with a __init__.py file inside.  This allows your python files you create as a module rather than just one off scripts.
  5. Now right click on the example folder and create a new python file by selecting New / PyDev Module.  Give it a fun name like “Example” and select the “Module: Main” template.  Finally click finish.
    – More advanced users can play around with the other templates.  For right now the main one should work perfectly for our needs.

    Pic9

  6. The following code prints out the name for all of the toolboxes available in ArcMap.  Type it in to see if your Python Environment is setup and ready to roll in Aptana.

    import arcpy
    if __name__ == ‘__main__’:
    toolboxes = arcpy.ListToolboxes() # Query for a list of toolboxes
    for toolbox in toolboxes: # Loop through the toolboxes
    print toolbox # And Print the name out

  7. Click the play button in the main toolbar on the top of the screen.

    Pic10

  8. Once you hit play, toolbox names should start popping up in the console section of Aptana.

    Pic11

  9. You are ready for lift off!
Advertisements

120 Hours in the Python’s Den: PyCon 2011

Starting in Version 10, Python is the de facto geoprocessing scripting language for the ESRI suite of software.  I primarily use python to automate the data processing work I perform.  I also use Python to create geoprocessing scripts for the ArcServer environment.  This year I attended PyCon, the Python languages community convention, to further develop my Python skills and see how others are using this powerful language.

I started my excursion to Atlanta for PyCon with three questions in mind:

1)      How can I utilize the multiprocessing capabilities of the python language?  What obstacles have others faced in implementing this functionality with ESRI’s ArcPy package?

2)      I would like to create intuitive input dialogs for my geoprocessing tasks.  How can I leverage other GUI frameworks (Qt, TK, wxPython) to create input dialogs with more built in functionality?

3)      How can we utilize the cloud to reduce the time for complex and time intensive geoprocessing tasks?

I took four classes to start the conference: Python 101, Google App Engine, Creating GUI Applications in Python using Qt I, and Deploying web applications to the cloud.  I learned a few cool tidbits from these four classes.

  • The print statement will be replaced with a print function in Python 3.  Syntactically this change makes a lot of sense.  In case you are wondering, you can actually backport this functionality into the Python 2.6 that comes with Version 10.  Quick code sample:

<code>

# Old

>>> Print “%s is a very fun language for %s” % (‘python’, ‘geoprocessing’)

# New

>>> Print (“{0} is a very fun language for {1}”.format(‘python’,’geoprocessing’))

>>> Print (“{langauge} is a very fun language for {task}”.format(language = ‘python’, task = ’geoprocessing’))

</code>

  • Google App Engine is cool, but I do not have a reason to use it yet.  I feel in the future, using this service, I will be able to quickly stand up a site without having to worry about infrastructure.
  • Qt can run on almost everything and every operating system.  This is a huge plus, however I am going to hold judgment until the dust settles with Nokia.
  • In a little under three hours I was able to setup a distributed, load balanced django application on the Amazon Cloud pulling data from their Relational Database Server and distributed using their CloudFront CDN.  I definitely see this as a resource I will invest more time in learning.

It is amazing to see what people are doing with python, primarily in the distributed computing and web application end of the python development spectrum.  My new goal for the year is to finally get around to learning to develop with Django.  It seems every other presentation was a huge django proponent.  There were also a few interesting talks on the forecast of Python language development and IDE’s for writing your code. Fortunately, for you the Python Foundation videotaped most of the presentations this year, http://pycon.blip.tv/posts?view=archive.

Alright and onto a few takeaway lessons I learned from my experience at PyCon.

  • ESRI is investing tons of resources into building Python as a quick, efficient geoprocessing language for their tools.  PyCon ran concurrently with ESRI’s developer summit; however, they were still able to send out 7+ representatives to PyCon.  I spent a couple nights with the ESRI folks, and to be honest I am incredibly pumped about version 10.1.  Tons of cool new features are coming down the pike.
  • Microsoft is gradually pushing into the Python/Open Source community.  I am in the process of downloading Microsoft’s Python extension for Visual Studio.  The latest and greatest release promises to debug both cPython (what ESRI uses) and IronPython (a .Net variety).  Currently I am using WinPython, the default with ArcGIS, but I see the Visual Studio version quickly superseding its use.
  • The python community could learn a lot from the GIS side of the house.  Almost every presentation involved crunching data, processing it in some way and displaying it graphically.  I definitely see an opportunity to step in and show a more intuitive method for displaying spatial data.

Esri Developer Summit: Closing Thoughts

Dan Levine

Well the 2011 Dev Summit wrapped up with a bunch of folks walking around with sore arms from the dodgeball tournament and you could tell Jim McKinney was disappointed his team wasn’t  in the finals. As has been the pattern over the last 6 years, Esri raises the bar a little bit each year making this better and better. The sessions we better, they were nimble in moving rooms around to accommodate crowds, and the food was better.  This is by far and away the best technical show they offer, and with the direct access to the product teams, it really is a must attend.

One of the big observations I have from the show overall is that we as an industry are still trying to figure out the mobile market and how to provide Enterprise solutions in that space. What I saw was a lot of sessions on “how to” with iOS or Android or Windows Mobile, which were well attended and necessary. But I am starting to suspect that the attendance in each of these was largely along personal favorites (iOS, Android, Windows Mobile) and perhaps not so much on need.  I was surprised at the low attendance at the couple of sessions about how to pick which one is appropriate or if you want to do multiple platform deployments, how do you decide.  Esri actually did a real nice job in these sessions.

Trying to put that observation or perception in context of the Keynote speakers point that the mobile market is incredibly fragmented right now, and the end users are demanding solutions working on multiple platforms is where I am struggling.  Are we as technologists still so caught up in the coolness factors of each of the individual platforms and the Marketing hype that drives separation (mine is better than yours) that we aren’t seeing the bigger picture? Or is it that we are still in the early part of the technology curve where we have to understand each of the technologies first then figure out how to tie them together?  That is not to say that there aren’t tools that allow us to start doing just that; it just seems like these are just getting out of the gate.

During the week before the Dev Summit I must have received 2 emails a day from some vendor, integrator, or techno think tank group offering to help me as a CTO figure out what my corporate mobile strategy was going to be.  This is obviously a big emerging problem for corporate IT departments that we need to be smart about. Do they standardize on a single platform (a strategy our friend from Gartner would seem to indicate is doomed), do they allow, and therefore have to support, multiple platforms across their IT infrastructure, or do they create some sort of Enterprise Mobile Service BUS that abstracts the layer between core IT systems and whatever mobile platform you want?

Hmmm, that last one sounds a lot like the AGS Server architecture model, and by the way the same model that the run time solution is going to follow……..

Okay slight change of topic, but still related.

So I have heard a little bit about the QT development environment in the last month or two. It’s always around developing cross platform solutions for mobile phones. It has just started to show up as one of the last bullets on Esri slides about developing multi-platform solutions in mobile. But for the first time I see that Esri has fully embraced this as part of a product line in the development of SDKs for the run time solution they will release as part of 10.1. On digging into the resource center, I am seeing a modest amount of whitepapers- like instructions and getting started material for using Qt for engine development- in 10.x. Man how do you keep up with all of it!

Qt is a cross-platform application and UI framework. Using Qt and C++, you can write web-enabled applications once and deploy them across desktop, mobile and embedded operating systems without rewriting the source code. http://qt.nokia.com/.  The guys presenting the run-time special session mentioned that it has been used for large scale enterprise solutions particularly when high performance is required.  Some of the real-time map refresh capability they showed during the demonstrations apparently will be available first with the Qt SDK and follow with the other 2 SDKs (JAVA and Silverlight WPF).  Patrick sat in a session for developing military solutions and they showed a Qt based application with military operations, moving maps, real-time comms, etc.  I was also a bit surprised at the show of hands during the run-time talk, apparently there are a bunch of C++ developers in attendance. I don’t remember seeing that represented much previously.

Well over the next couple of weeks I will be doing a lot of thinking about what we saw this week and trying to figure out what it means to the industry and what we as a company want to invest in. Already looking forward to next year and what we can do until then.

Steve Mulberry

The Dev Summit came to a close today but not without its share of informative sessions. I sat through one this morning that made me think more about the comment, “ArcGIS Desktop is not meant for the web”.  We keep hearing how GIS is moving to the web with the many open API’s, free templates, tools for hosting and managing data,  applications, etc. I also hear Esri and others say, “Desktop will always be around and there’s no way you can do the things in desktop over the web”. Not so fast; in the last year or so we’ve seen BA Desktop become BA Server with an open API, and then evolve into BA Online.

This opened the door for Community Analyst.

Early this year we saw ArcLogistics desktop become web enabled.

Now I learn that the desktop redistricting tool we’re all familiar with has moved to the web.

Each one of these tools is arguably a full featured GIS with editing data, analyzing information, map production, and dissemination. What’s next, an online property management system? Desktop over the web…not such a pipe dream any more.

Patrick Barnhart

I’m sitting in the airport on my way home from the summit and it seems a good time to wrap up.  It’s been a great week but it’ll be nice to get home.  I’ve taken in so much stuff that it’s starting to blur together but since this will be my last post for a while, I thought I’d focus on a task that’s common to a lot of the projects going on at GISi – printing high quality maps from a web application.  I mentioned this one on day 3 and promised to attend some sessions on Python to learn a little more about it since that seems to be the easiest way to go about printing to a PDF from ArcGIS 10.

Python has been part of ArcGIS since release 9 and has been successful enough that support for it has been increasing with every release until now, at 10.1, it is fully integrated into the application – intellisense, code completion, and integrated help; they’ve even added a Python console to the main interface and gave it a name – ArcPy.  I’ve messed with Python before, mainly by creating tools using Model Builder and exporting them to scripts that I could modify, but I’ve always used it inside of ArcMap, which isn’t very helpful when it comes to printing from the web.  One of yesterday’s sessions entitled ‘Using Geoprocessing Services in a Web Application’ showed me that there’s a lot more that can be done with it, and the follow up sessions I attended on Python make it look pretty easy.  Python is a very clean, easy to read language, and while I’ll have to learn a new syntax, it looks fun.

Okay, so we all know you can publish a Geoprocessing task through ArcGIS Server and make it available to other ArcMap users via the toolbox.  What I didn’t know, or at least hadn’t really focused on before, is that you can also access these tools via REST.  As we’ve seen in my previous posts, there’s a number of ways to make REST call to ArcGIS Server, either directly using a URL from a back end web service or from the front end using one of the Web APIs (JavaScript, Flex, Silverlight).  Any of these will work; I haven’t done much Flex or Silverlight, but me and JavaScript go way back, so that’s the approach that appeals to me.

As we should with all projects, let’s first define our scope.  What I’m after is the ability to allow the user to select a template from a list; this will set the locations of the map elements on the page, as well as the size and orientation.  So the title, legend, scale bar, north arrow, comments, etc will be placed where we want them to go so we can make a nice clean map that’s easy to customize if we need to adjust the template.  Then all we need to do is create the elements, put them in where they need to go, and output the whole thing into a PDF for the user to print at a high resolution.  Piece of cake, right?  If you’re doing it from scratch, maybe not so much.

ArcPy, however, makes things easier.  There are a ton of existing scripts that we can use and it has the ability to open an .mxd to access the layout and retrieve map elements based on their name.  For example, a text area named Title can be retrieved and populated by ArcPy with relative ease.  To do this, all I really need is to create layouts in ArcMap (one layout per .mxd, named appropriately so I can find them later) and make a JavaScript front end with a couple of menu items and a print button.  Esri is all about Dojo (some folks like JQuery, I’m more of an ExtJS guy), but it doesn’t matter as long as we use the JavaScript API to create a handler on the print button, we’re good.

Our handler needs to get the state of our map (extents, layers, etc) and the layout selected by the user, then make a REST call to a Python Geoprocessing tool that will do the work.  Of course, I haven’t actually done this part yet, but since we’re dealing with ArcGIS templates, there’s existing code we can use to generate the map and export the whole thing to a pdf.  The example I saw, which should be available on ESRI’s resource page in a couple of weeks if it isn’t up already, was only a couple of pages worth of code, and compared to how I’ve done this in the past using open source libraries like ITextSharp, it looked really simple.

So, as you can see, my head is still spinning from all the possibilities I’ve seen over the last couple of days, and want to play with all the new toys all at once, but the reality is there’s only so many hours in the day.  Hopefully I’ll get a chance to take a closer look at some of these things in the days to come and will pass along what I learn.  And finally, if you’re like me and have never had a chance to attend the developer summit before, start bugging your manager about it for next year; it’s a good time.  Traveling is a pain and there’s a lot of sitting still involved (which is tough for me), but you’ll see a lot of great stuff and Esri puts on a great show.

Ben Taylor

I attended a session today geared toward the defense and intel community. The majority of the presentation centered around creating maps for military use and 2525C symbolization, however, there was a cool demo that utilized the upcoming 10.1 runtimes, specifically the C++ Qt SDK. The demo included messaging between multiple simulated vehicles. The performance of the application running on the new runtime was quite impressive.

And speaking of the new runtime, I also attended a session highlighting their upcoming release in 10.1. There are some pretty significant changes coming that will yield a smaller overall footprint, increase performance, and simplify deployment. However, low level access to ArcObjects will not be supported in the runtime. So there will be a shift in development paradigms mainly toward WPF. A word of advice I picked up in the presentation is to get familiar with the Silverlight Web API, because if you are comfortable with that, you’ll have a good head start on working with the new runtimes.

Tommy Bramble

I started the final day of the conference with the ‘Building RIA Clients Using ArcGIS Workflow Manager REST Services.’  Esri is calling AGS Server Workflow Manager the ‘complete GIS management system’ and recommends using the product to create and manage GIS project workflows.  The product can be used by all stake holders including clients, managers, analysts, and developers.   Workflow Manager will allow you to author and publish project workflows, centralize project information and documentation, control access to data (including spatial data in your SDE repository), and integrate GIS into business processes.  The main intent of this session was to demonstrate how you can build a custom workflow manager solution using the new REST endpoints available.  The presenters showed examples of a custom Flex viewer that was able to query users, jobs, reports, and workflows through attribute criteria or by spatial area-of-interest.

My next session of the day was titled ‘Redesigning Desktop Applications for the Web.’  The session was basically a talk on why you would (and wouldn’t) want to move your applications to the web.  The web has become a pervasive, accessible, and proven platform and now might be the best time to consider moving desktop apps over, but, ultimately, you will need good project planning and a sound understanding of the user audience to make the best decision.   Some of the key information I took away from this session included:

  • The web is a fast and cheap infrastructure that provides a platform for easy collaboration and sharing.
  • Application development is easier now than ever before (with WebAPI’s and Application Builder tools).
  • The web interfaces are richer and smoother, allowing a desktop-like experience.
  • The user experience on the web is highly important.  You need a simple, but powerful application.
  • You must know the user audience and application requirements.  In some situations, a desktop application will still be the best choice.
  • Must consider security implications on the web.
  • Planning is key!  Planning the design, development, and deployment phases are critical to any application’s success.

My last session of the conference was ‘Advanced Development with ArcGIS API for MS Silverlight/WPF’.   Having no development experience with Silverlight, my goal was to learn what is possible with the framework and how it may differ from the other WebAPI’s.  The presenters here, like in most all other web sessions, stressed the importance of the user experience and reminded the audience that the application does not always have to be map-centric.  The application examples presented here demonstrated the grid/map interaction (i.e. hover over a data record and an assortment of animated effects highlight the related map feature) and other visually rich ways of interacting with the map and attribute data.  Some basic editing examples were also shown.  From the standpoint of a beginner Silverlight developer, I was most interested in the new Silverlight Application Builder (a WYSIWYG Silverlight editor) and the Visual Studio Silverlight templates.   Apparently, you can easily switch back and forth between the Application Builder and your custom Visual Studio-based Silverlight projects, which could make development in teams easier.

This was the end of the conference for me.  Esri did a great job and presented a lot of useful information regarding their products and future trends.

Charming the Snake : Python Tips

While fearing a real python might be justified, you shouldn’t be intimidated by the Python language. Esri has embraced Python as the language that fulfills the needs of its user community. Programming with Python using the new ArcPy site-package in ArcGIS 10 makes GIS programming accessible for the average GIS professional. For longtime GIS users who yearn for the simpler days of Avenue (the scripting language used with ArcView 3.x), Python has some of the same attractive features: a manageable number of well documented classes and functions, scripts that can be easily loaded into a document and executed, code shared as simple text files, and (perhaps best of all) no need to know what QueryInterface means! Python is a widely used, nonproprietary language, so learning Python can be beneficial for non-GIS applications.

To help you get started with Python, this article will highlight a few basic techniques that are commonly required when writing

Python scripts for ArcGIS, such as how to:

  • Reference additional libraries (i.e., import functionality) for use in a script.
  • Accept user-provided arguments.
  • Get a reference to the current map document.
  • Create, open, and write to files.
  • Show messages that report the script’s progress.
  • Execute a geoprocessing tool.
  • Access objects in a map document (data frames, layers, tables).

Importing Additional Modules

Python modules are files used to organize sets of additional functionality so your script can access it. The import statement is one way to bring a module into a script and make the functionality it defines available. To import a module, simply type an import statement at the top of the script window, like those shown below:

import os

import sys

import arcpy

import arcpy.mapping as mapping

This example shows some modules you may commonly work with. The os and sys modules contain functions for working with files on disk. The ArcPy site-package allows access to geoprocessing tools. The ArcPy mapping module provides functionality for working with the map document and the objects it contains.

Reading User Inputs

Commonly, a script will need some input from the user in order to run, and when creating a script, parameters can be defined to specify what information will be entered by the user. The order of parameters defined for the script is important, because parameter values are read from an ordered list of inputs. The following code illustrates how to read user inputs. Note that the

first parameter value is referenced by index position 0:

outDir = arcpy.GetParameterAsText(0)

packageMap = arcpy.GetParameter(1)

checkBrokenLayers = arcpy.GetParameter(2)

GetParameterAsText reads user input as a text string. GetParameter reads user input and returns an object (e.g., Boolean).

Getting a Reference to the Current Map

When scripting in any environment, the current document is often the key object. Once a reference to the current document is obtained, the script can drill down into the additional objects it contains. When scripting for ArcMap, a reference to the map document will provide access to the data frames, layers, and tables the map contains. The example below uses the MapDocument function to get a reference to the current map document and store it in a variable called mxd.

mxd = mapping.MapDocument(‘Current’)

The required parameter for the MapDocument function is either a path to a map document on disk or the keyword Current to get the map in which the script is executing.

Working with a File on Disk

Use the open function (imported in the os package) to open a file on disk:

reportFile = open(reportPath, ‘w’)

The open function takes two arguments: the file name and the file mode. Valid file modes are w for write, r for read, and a for append. If the file exists, it will be overwritten. If it does not exist, it will be created.

Use the write function to write information to a file (if it was opened in either write or append mode).

reportFile.write(reportText)

reportFile.close()

Make sure to close a file when you are done working with it.

Showing Status Messages as a Script Runs

Use the AddMessage function (from the ArcPy package) to display a message as your script runs. The code below produces a message that will appear in the script’s output window as it executes.

arcpy.AddMessage(‘Writing report to ‘ + reportPath)

Executing a Geoprocessing Tool

Any geoprocessing tool that is available from ArcToolbox may be executed programmatically using ArcPy. In fact, each tool is available as a function in ArcPy. Consult the ArcGIS documentation for more information regarding available geoprocessing tools. The example below will execute the Package Map geoprocessing tool (in the Data Management toolbox).

arcpy.PackageMap_management(mxdFilePath, packagePath)

PackageMap_management requires the path to the map document and the path to the output package file.

Get Objects in the Map (Data Frames and Layers, e.g.)

Use mapping.ListDataFrames to get a list of data frames from the map.

dataFrames = mapping.ListDataFrames(mxd, ‘’)

The ListDataFrames function takes two arguments: a map document (or a path to one) and a wildcard string for filtering data frames based on name (e.g., La*). The function returns a Python list of data frames. Similar to the ListDataFrames function, ListLayers returns a Python list object containing layer objects.

layers = mapping.ListLayers(mxd, ‘’, frame)

Conclusion

With the ArcPy site-package, Esri has integrated Python into ArcGIS 10. Python’s relative simplicity and power make it the scripting language of the future for ArcGIS.

For more information, contact Thad Tilton at ttilton@gisinc.com.

ArcPy + ArcGIS Server = Excellent, fast, high quality prints

In a previous blog, Bryan Baker discussed how it is possible to create high-quality printed maps through ArcGIS Server (using a webservice and ArcObjects combo).  Now that 10 has been released, with all of its new libraries and other pieces to make our lives easier, we saw an opportunity to create a web-based printing solution without using ArcObjects.  With the release of 10, ESRI bundled a new python site package with ArcGIS Server (and ArcMap) called ArcPy, which gives developers access to items once only obtainable through ArcObjects. Read more of this post